Data index construction method and device and storage medium

Abstract

The invention discloses a data index construction method and device and a storage medium, and belongs to the field of storage. The method is applied to a client, the client is used for managing a plurality of storage nodes arranged on an infrastructure, at least one storage node in the plurality of storage nodes stores data of a user, and cloud services of the user also run on the infrastructure. The method comprises the following steps: acquiring an index operator input by a user, wherein the index operator comprises field parameters of multi-level indexes; sending the index operator to a first storage node in the at least one storage node, wherein the first storage node is any one storage node in the at least one storage node; and configuring the first storage node to enable the first storage node to generate a multi-level index according to the index operator. The data storage efficiency can be improved.

Description

[0001] This application claims priority to Chinese patent application No. 202310760309.X, filed on June 26, 2023, and entitled “Method for Generating KV Operator”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of storage, and in particular to a data index construction method, device and storage medium. Background Art

[0003] A cloud storage system is a system for storing data online. The cloud storage system includes a cloud service and multiple storage nodes. The cloud service is used to generate records that need to be stored. The records include multiple index values ​​and data content. The multiple storage nodes are used to store the records generated by the cloud service. When the data content needs to be queried, the records including the multiple index values ​​can be queried from the multiple storage nodes based on the multiple index values, and the data content can be read from the records.

[0004] When a cloud service generates a record that needs to be stored, the cloud storage system selects a storage node for the record from the multiple storage nodes, sends the record to the storage node, and the storage node receives and stores the record. However, the current process of storing records in storage nodes is relatively complicated and time-consuming, which reduces the efficiency of storing data. Summary of the invention

[0005] The present application provides a data index construction method, device and storage medium to improve the efficiency of storing data. The technical solution is as follows:

[0006] In the first aspect, the present application provides a data index construction method, which is applied to a client, and the client is used to manage multiple storage nodes set on an infrastructure, at least one of the multiple storage nodes stores the user's data, and the user's cloud business is also running on the infrastructure. In the method, an index operator input by the user is obtained, and the index operator includes field parameters of a multi-level index. The index operator is sent to a first storage node in at least one storage node, and the first storage node is any storage node in at least one storage node. The first storage node is configured so that the first storage node generates a multi-level index according to the index operator.

[0007] Since the index operator includes field parameters of the multi-level index, the index operator is sent to the first storage node, so that the first storage node can be configured so that the first storage node generates a multi-level index in one transaction according to the index operator, which can reduce the time required to generate the multi-level index. In this way, data is stored in the multi-level index, which improves the efficiency of storing data.

[0008] In one possible implementation, a first-level retrieval request is sent to the first storage node, and the first-level retrieval request is used to enable the first storage node to generate a multi-level index in one retrieval transaction. The multi-level index can be generated in one transaction, reducing the time required to generate the multi-level index.

[0009] In another possible implementation, the cloud service includes multiple first records, the multi-level index is an index tree, the field parameters of the multi-level index include at least one first range set, the first range set includes the i-th level index value range and at least one i+1-th level index value range, i=1, 2, 3,..., among the multiple first records, there are records including the i-th level index value belonging to the i-th level index value range and the i+1-th level index value belonging to the i+1-th level index value range, and the first-level retrieval request is used to enable the first storage node to establish an index tree based on the at least one first range set and the multiple first records.

[0010] Since each first range set includes the i-th level index value range and at least one i+1-th level index value range, each first range set indicates the i-th level index node and at least one i+1-th level index node of the i-th level index node, and the at least one i+1-th level index node is a child node of the i-th level index node. In this way, the first storage node can establish multiple levels of index nodes in the index tree based on each first range set in one transaction, which can reduce the time required to establish the index tree, thereby improving the efficiency of storing data.

[0011] In another possible implementation, the user inputs multiple index operators, and sends multiple index operators to the first storage node so that the first storage node generates multiple multi-level indexes according to the multiple index operators. In this way, multi-level indexes can be generated, thereby improving the efficiency of generating multi-level indexes.

[0012] In another possible implementation, a successful response sent by the first storage node after generating the multi-level index is received. An index operator is sent to a second storage node, where the second storage node is at least one storage node other than the first storage node. The second storage node is configured so that the second storage node generates the multi-level index according to the index operator. In this way, multiple storage nodes can generate the multi-level index.

[0013] In another possible implementation, after a multi-level index is established in at least one storage node, an index result is sent to the cloud service. The index result is used to indicate that at least one storage node has successfully generated a multi-level index, so that the cloud service can save data in the multi-level index of the storage node after receiving the index result.

[0014] In a second aspect, the present application provides a data index construction device, which is used to execute the method in the first aspect or any possible implementation of the first aspect. Specifically, the device includes a unit for executing the method in the first aspect or any possible implementation of the first aspect.

[0015] In a third aspect, the present application provides a data index construction device, comprising at least one processor and a memory, wherein the at least one processor is used to couple with the memory, read and execute instructions in the memory, so as to implement the method in the first aspect or any possible implementation manner of the first aspect.

[0016] In a fourth aspect, the present application provides a computer program product, which includes a computer program stored in a computer-readable storage medium, and the computer program is loaded by a processor to implement the method in the above-mentioned first aspect or any possible implementation manner of the first aspect.

[0017] In a fifth aspect, the present application provides a computer-readable storage medium for storing a computer program, wherein the computer program is loaded by a processor to execute the method in the above-mentioned first aspect or any possible implementation manner of the first aspect.

[0018] In a sixth aspect, the present application provides a chip comprising a memory and a processor, the memory being used to store computer instructions, and the processor being used to call and run the computer instructions from the memory to execute the method in the above-mentioned first aspect or any possible implementation of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 It is a structural diagram of a cloud storage system provided by an embodiment of the present application;

[0020] Figure 2 It is a schematic diagram of the structure of an index tree provided in an embodiment of the present application;

[0021] Figure 3 is a schematic diagram of the structure of another cloud storage system provided in an embodiment of the present application;

[0022] Figure 4 It is a flow chart of a data index construction method provided in an embodiment of the present application;

[0023] Figure 5 is a schematic diagram of the structure of another index tree provided in an embodiment of the present application;

[0024] Figure 6 It is a structural schematic diagram of a data index construction device provided in an embodiment of the present application;

[0025] Figure 7is a schematic diagram of the structure of a computing device provided in an embodiment of the present application;

[0026] Figure 8 It is a schematic diagram of the structure of a cluster provided in an embodiment of the present application;

[0027] Fig. 9 It is a schematic diagram of the structure of another cluster provided in an embodiment of the present application. DETAILED DESCRIPTION

[0028] The implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0029] The following is an introduction to the concepts that appear in the embodiments of the present application.

[0030] The client is a management system of the infrastructure. For example, the client is used to manage multiple storage nodes set up on the infrastructure.

[0031] Index operators are configuration parameters configured by users on the client for establishing multi-level indexes. Index operators include field parameters of multi-level indexes.

[0032] A multi-level index is an index tree including multi-level index nodes, where the i-th level index node includes the i-th level index value range, i=1, 2, 3, ...

[0033] The field parameters of the multi-level index include configuration parameters for establishing an index tree. For example, the field parameters of the multi-level index include at least one first range set, each first range set includes an i-th level index value range and at least one i+1-th level index value range, which are used to establish an i-th level index node and at least one i+1-th level index node of the index tree.

[0034] The first record is the data to be stored generated by the cloud business, including multi-level index values ​​and data content. The leaf node of the index tree is used to save the first record, and the multiple index value ranges to which the multi-level index values ​​in the first record belong are the index value ranges included by each index node on the path from the leaf node to the root node.

[0035] See also Figure 1 The embodiment of the present application provides a cloud storage system 100, which includes a client 101, a cloud service 102, and M storage nodes 103, where M is an integer greater than or equal to 1. The client 101 communicates with each storage node 103 of the M storage nodes 103, and the client 101 also communicates with the cloud service 102.

[0036] The cloud service 102 is used to obtain records to be stored.

[0037] The client 101 is used to provide an interface to the user. Thus, the user configures the index operator through the interface, and the index operator includes field parameters of the multi-level index. The client 101 is also used to receive the index operator through the interface and send the index operator to each storage node 103.

[0038] For each storage node 103, the storage node 103 is used to receive the index operator and generate a multi-level index based on the index operator.

[0039] In some embodiments, the cloud service 102 is used to generate records, and the records to be stored are records generated by the cloud service 102 .

[0040] In some embodiments, the record includes a multi-level index value and data content, etc. Optionally, the record is a key-value pair, the key in the record includes the multi-level index value, and the value in the record includes the data content.

[0041] For each storage node 103, the multi-level index generated by the storage node 103 can be an index tree. The storage node 103 uses the index tree to store the records obtained by the cloud service 102. Optionally, the storage node 103 saves the records obtained by the cloud service 102 in the leaf nodes of the index tree.

[0042] See also Figure 2 The index tree includes multiple non-leaf nodes and multiple leaf nodes. The multiple non-leaf nodes in the index tree can also be called index nodes. The index nodes included in the index tree are divided into Q levels, that is, the index tree includes Q-level index nodes, and Q is an integer greater than or equal to 2.

[0043] The root node of the index tree is the 0th level index node, and at least one child node of the root node is the 1st level index node of the index tree. For each 1st level index node, at least one child node of the 1st level index node is the 2nd level index node of the index tree, ..., at least one child node of the Q-1th level index node is the Qth level index node.

[0044] The root node of the index tree may be a virtual node. Alternatively, the root node of the index tree may not include any content.

[0045] The i-th level index node of the index tree includes the i-th level index value range, i=1, 2, ..., Q. At least one child node of the i-th level index node is the i+1-th level index node, and each i+1-th level index node includes the i+1-th level index value range.

[0046] At least one child node of the Q-th level index node is a leaf node, that is, the multiple leaf nodes of the index tree include the child nodes of each Q-th level index node in the index tree.

[0047] For each leaf node, the leaf node is used to store at least one record, and the index value range of the index value included in the record stored in the leaf node is the index value range included in each level of index nodes on the path between the leaf node and the root node of the index tree. In other words, the i-th level index value range included in the record stored in the leaf node is the index value range included in the i-th level index node on the path between the leaf node and the root node of the index tree.

[0048] For example, see Figure 2 , assuming that the index tree includes three levels of index nodes and multiple leaf nodes, that is, Q=2, the 0th level index node of the index tree is the root node, and the root node is a blank virtual node. The two leaf nodes of the root node are the first level index nodes, namely the first level index node 11 and the first level index node 12. Among them, the first level index node 11 includes the first level index value range "age:9-11", indicating that the age is greater than or equal to 9 and less than or equal to 11 years old, and the first level index node 12 includes the first level index value range "age:12-15", indicating that the age is greater than or equal to 12 and less than or equal to 15 years old.

[0049] The two leaf nodes of the first-level index node 11 are second-level index nodes, namely, second-level index node 21 and second-level index node 22. Among them, second-level index node 21 includes a second-level index value range "name:af", and the names included in the second-level index value range are a, b, c, d, e or f, and second-level index node 22 includes a second-level index value range "name:xz", and the names included in the second-level index value range are x, y or z. A leaf node of the first-level index node 12 is a second-level index node 23. Among them, second-level index node 23 includes a second-level index value range "name:gk", and the names included in the second-level index value range are g, h, j or k.

[0050] The two child nodes of the second-level index node 21 are two leaf nodes, namely leaf node 31 and leaf node 32. Among them, the first record included in leaf node 31 is: the first-level index value "age=10", the second-level index value "name=a" and the data content "id=1", and the second record is: the first-level index value "age=11", the second-level index value "name=b" and the data content "id=3". The first record included in leaf node 32 is: the first-level index value "age=10", the second-level index value "name=e" and the data content "id=2", and the second record is: the first-level index value "age=11", the second-level index value "name=f" and the data content "id=5".

[0051] The two child nodes of the second-level index node 22 are two leaf nodes, namely leaf node 33 and leaf node 34. The first record included in leaf node 33 is: the first-level index value "age=10", the second-level index value "name=x" and the data content "id=6", and the second record is: the first-level index value "age=11", the second-level index value "name=y" and the data content "id=4". The first record included in leaf node 34 is: the first-level index value "age=10", the second-level index value "name=z" and the data content "id=7".

[0052] A child node of the second-level index node 23 is a leaf node 35. The first record of the leaf node 35 is: the first-level index value "age=12", the second-level index value "name=g" and the data content "id=8", and the second record is: the first-level index value "age=13", the second-level index value "name=h" and the data content "id=9".

[0053] In some embodiments, cloud service 102 is used to obtain records to be stored and send the records to client 101. Client 101 uses a consistent hashing algorithm to send the records to a storage node 103. The storage node 103 receives the records and saves the records in a leaf node of the index tree.

[0054] In some embodiments, see Figure 3 The cloud storage system 100 also includes a metadata center (MDC) 104 and N virtual nodes, where N is an integer greater than M. The N virtual nodes are distributed in the M storage nodes 103, and the MDC 104 is used to store the consistent hashing algorithm.

[0055] The client 101 is used to receive the record sent by the cloud service 102, obtain the hash value of the record, and obtain the consistent hash algorithm from the MDC 104; based on the hash value of the record and N, the consistent hash algorithm is used to determine the virtual node to which the record belongs, and send the record to the storage node 103 where the virtual node is located.

[0056] The storage node 103 receives the record and stores the record in a leaf node of the index tree in the storage node 103. Next, the process of storing the record is described in detail through any of the following embodiments.

[0057] See also Figure 4 The present application embodiment provides a data index construction method 400, which is applied to Figure 1 or Figure 3 The cloud storage system 100 shown includes the following process.

[0058] Step 401: The client receives an index operator, which includes field parameters of a multi-level index.

[0059] In some embodiments, the client displays an interface through which a user configures an index operator and through which the client receives the index operator.

[0060] Optionally, the interface is an interface interface, the client displays the interface interface, the user can configure the index operator in the interface interface, and the client obtains the index operator from the interface interface.

[0061] The field parameters of the multi-level index include at least one first range set, that is, the index operator includes the at least one first range set. Each first range set includes the i-th level index value range and at least one i+1-th level index value range corresponding to the i-th level index value range, i=1, 2, 3, ..., Q-1.

[0062] In some embodiments, for each first range set, the first range set is a key-value pair, the key in the key-value pair includes the i-th level index value range in the first range set, and the value in the key-value pair includes the at least one i+1-th level index value range in the first range set.

[0063] For example, assuming Q=2, the field parameters of the multi-level index in the index operator include a first range set, the first range set includes a first-level index value range "age:9-11", and two second-level index value ranges corresponding to the first-level index value range "age:9-11", and the two second-level index value ranges are "name:af" and "name:xz" respectively.

[0064] In some embodiments, the user can configure multiple index operators in the interface, and the client can obtain the multiple index operators, each index operator is used to generate a different index tree.

[0065] Step 402: The client sends a first-level search request to a first storage node, where the first-level search request includes the index operator. The first storage node is a storage node among M storage nodes, where M is an integer greater than or equal to 1.

[0066] For example, in the above Q=2, the client sends a first-level retrieval request to the first storage node, and the first-level retrieval request includes a first range set, the first range set includes a first-level index value range "age:9-11", and two second-level index value ranges corresponding to the first-level index value range "age:9-11", and the two second-level index value ranges are "name:af" and "name:xz" respectively.

[0067] In some embodiments, when the client obtains multiple index operators, the first-level retrieval request includes the multiple index operators, so that the client concurrently sends the multiple index operators to the first storage node, so that the first storage node generates multiple multi-level indexes based on the multiple index operators, that is, generates multiple index trees.

[0068] Step 403: The first storage node receives the first-level search request, where the first-level search request includes the index operator.

[0069] That is to say, the first-level retrieval request includes at least one first range set in the index operator, each first range set includes the i-th level index value range and at least one i+1-th level index value range corresponding to the i-th level index value range, i=1, 2, 3,..., Q-1.

[0070] In some embodiments, the primary search request includes multiple index operators.

[0071] Step 404: The first storage node generates a multi-level index based on the index operator.

[0072] In step 404, the first storage node establishes a plurality of index nodes included in the index tree based on the at least one first range set included in the index operator.

[0073] The multiple index nodes included in the index tree are multiple non-leaf nodes included in the index tree.

[0074] For each first range set including the i-th level index value range and at least one i+1-th level index value range corresponding to the i-th level index value range, the multiple index nodes include the i-th level index node corresponding to the i-th level index value range and at least one i+1-th level index node corresponding to the at least one i+1-th level index value range, the i-th level index node includes the i-th level index value range, each i+1-th level index node respectively includes the i+1-th level index value range corresponding to each i+1-th level index node, and the at least one i+1-th level index node is a child node of the i-th level index node.

[0075] In step 404, the first storage node generates a retrieval transaction based on the primary retrieval request; in the retrieval transaction, a plurality of index nodes included in an index tree are established based on the at least one first range set included in the primary retrieval request.

[0076] Optionally, the first-level retrieval request may include multiple index operators, and the first storage node generates multiple multi-level indexes, that is, generates multiple index trees, according to the multiple index operators.

[0077] For each index operator, the first storage node may generate a multi-level index according to the index operator through 4041-4045 as follows, that is, generate multiple index nodes included in the index tree through 4041-4045 as follows.

[0078] 4041: The first storage node first establishes the root node of the index tree.

[0079] Optionally, the root node may be a blank node and also the 0th level index node of the index tree, and the first-level retrieval request may not include the 0th level index value range.

[0080] 4042: The first storage node obtains at least one first range set including a first-level index value range from the first-level retrieval request, and establishes at least one first-level index node, where the at least one first-level index node is a child node of the root node, and the at least one first-level index node includes the first-level index value range in the at least one first range set.

[0081] That is, the at least one first-level index node corresponds one-to-one to at least one first-level index value range, and each first-level index node includes a first-level index value range in a first range set.

[0082] 4043: For each first range set in the at least one first range set, the first range set includes a level 1 index value range and at least one level 2 index value range corresponding to the level 1 index value range, and for the level 1 index node including the level 1 index value range, the first storage node establishes at least one level 2 index node for the level 1 index node.

[0083] The at least one second-level index node is a child node of the first-level index node, and the at least one second-level index node includes at least one second-level index value range in the first range set. That is, the at least one second-level index node corresponds to at least one second-level index value range one by one, and each second-level index node includes a second-level index value range in the first range set.

[0084] 4044: The first storage node obtains at least one first range set including a t-th level index value range from the first-level retrieval request, and for each first range set in the at least one first range set, the first range set includes a t-th level index value range and at least one t+1-th level index value range corresponding to the t-th level index value range, and for the t-th level index node including the t-th level index value range, the first storage node establishes at least one t+1-th level index node for the t-th level index node.

[0085] Here, the initial value of t is equal to 2.

[0086] Among them, the at least one t+1th level index node is a child node of the tth level index node, and the at least one t+1th level index node includes at least one t+1th level index value range in the first range set. That is to say, the at least one t+1th level index node corresponds to at least one t+1th level index value range, and each t+1th level index node includes a t+1th level index value range in the first range set. Repeat the above process until t=Q-1, and the first storage node establishes the Q-level index node of the index tree.

[0087] 4045: The first storage node determines whether t is equal to Q-1. If t is less than Q-1, t=t+1 is set and the process returns to 4044. If t is equal to Q-1, the process ends.

[0088] After the first storage node completes establishing the Q-level index node of the index tree, it indicates that the first storage node has completed generating the multi-level index and sends a success response to the client.

[0089] After receiving the successful response, the client sends a first-level retrieval request to the second storage node. The first-level retrieval request includes the index operator. The second storage node is a storage node other than the first storage node among the M storage nodes. After receiving the first-level retrieval request, the second storage node generates a multi-level index through the above 4041-4045 according to the index operator included in the first-level retrieval request, that is, generates multiple index nodes included in the index tree through the above 4041-4045.

[0090] For example, the first storage node receives a first-level retrieval request, which includes a first range set, the first range set including a first-level index value range "age:9-11", and a second-level index value range "name:af" and "name:xz" corresponding to the first-level index value range "age:9-11".

[0091] See also Figure 5 , the first storage node establishes a root node, which is a blank node. The first-level index value range "age:9-11" is obtained from the first-level search request, and a first-level index node 11 is established. The first-level index node 11 is a child node of the root node, and the first-level index node 11 includes the first-level index value range "age:9-11".

[0092] For the two second-level index value ranges, the first storage node establishes two child nodes for the first-level index node, and the two child nodes are the second-level index node 21 and the second-level index node 22. The second-level index node 21 includes the second-level index value range "name:af", and the second-level index node 22 includes the second-level index value range "name:xz".

[0093] After the M storage nodes have established the multi-level index, the client sends the index result to the cloud service, and the index result is used to indicate that the M storage nodes have successfully generated the multi-level index. After the cloud service receives the index result, the cloud service can obtain multiple first records to be stored, and send the multiple first records to the M storage nodes, so that the M storage nodes save the multiple first records in the leaf nodes of the index tree. The implementation process includes the following steps 405-406.

[0094] Step 405: The client receives multiple first records sent by the cloud service, each first record includes a multi-level index value and data content, and sends at least one first record to each storage node among the M storage nodes, and the multiple first records include the at least one first record.

[0095] The multiple first records are generated by cloud services. For each first record, the first record includes a first-level index value, a second-level index value, ..., a Qth-level index value, and data content.

[0096] In some embodiments, the process of a client sending at least one first record to a storage node is as follows: for each first record received, the client obtains a hash value of the first record, determines the virtual node to which the first record belongs based on the hash value of the first record and N, and sends the first record to the storage node where the virtual node is located.

[0097] In some embodiments, the storage node where the virtual node is located may have at least one backup storage node, and the client may also send the first record to the at least one backup storage node so that the at least one backup storage node stores the first record. The detailed storage process is the same as the storage process of the storage node, and will not be described in detail. Optionally, the M storage nodes include the at least one backup storage node.

[0098] Wherein, N is the number of virtual nodes included in the M storage nodes, and N is an integer greater than M. Optionally, the client obtains a consistent hash algorithm from the MDC, obtains a hash value of the first record through the consistent hash algorithm, modulo the hash value of the first record based on N to obtain a sequence number of the virtual node to which the record belongs, and sends the first record to the storage node where the virtual node is located based on the sequence number of the virtual node.

[0099] In some embodiments, when adding a storage node, at least one virtual node may be selected from the N virtual nodes included in the M storage nodes, and the records belonging to the at least one virtual node may be moved to the added storage node. In this way, there is no need to re-determine the storage node to which each record in the cloud storage system belongs, thereby improving the efficiency of expanding the storage node.

[0100] Step 406: The first storage node receives at least one first record, and adds one or more leaf nodes to the index tree, each leaf node includes part or all of the at least one first record, and for each leaf node, the index value range to which the index value in the first record included in the leaf node belongs is the index value range included in each level of index nodes on the path between the leaf node and the root node of the index tree.

[0101] In step 406, the first storage node adds one or more leaf nodes to the index tree through the following process 4061-4064.

[0102] 4061: For each first record, the first storage node obtains, based on the first-level index value included in the first record, a first-level index value range to which the first-level index value belongs from at least one first-level index node of the root node including at least one first-level index value range, and determines a first-level index node including the first-level index value range.

[0103] For example, the first storage node receives multiple first records, which include first record 1, first record 2, first record 3, first record 4, first record 5, first record 6, first record 7, etc.

[0104] The first record 1 includes a first-level index value "age=10", a second-level index value "name=a" and a data content "id=1". The first record 2 includes a first-level index value "age=11", a second-level index value "name=b" and a data content "id=3". The first record 3 includes a first-level index value "age=10", a second-level index value "name=e" and a data content "id=2". The first record 4 includes a first-level index value "age=11", a second-level index value "name=f" and a data content "id=5". The first record 5 includes a first-level index value "age=10", a second-level index value "name=x" and a data content "id=6". The first record 6 includes a first-level index value "age=11", a second-level index value "name=y" and a data content "id=4". The first record 7 includes a first-level index value "age=10", a second-level index value "name=z" and a data content "id=7".

[0105] See also Figure 5 For the first record 1, the first storage node obtains the first-level index value range "age:9-11" to which the first-level index value "age=10" belongs from the first-level index value range "age:9-11" included in the first-level index node 11 of the root node based on the first-level index value "age=10" included in the first record 1, and determines the first-level index node 11 including the first-level index value range "age:9-11".

[0106] 4062: The first storage node obtains, based on the k-th index value included in the first record, a k-th index value range to which the k-th index value belongs from at least one k-th index value range included in at least one k-th index node of the k-1-th index node, and determines a k-th index node including the k-th index value range.

[0107] Here, the initial value of k is equal to 2.

[0108] For example, see Figure 5 For the first record 1, the first storage node obtains the second-level index value range "name:af" to which the second-level index value "name=a" belongs from the two second-level index value ranges ("name:af" and "name:xz") included in the two second-level index nodes (second-level index node 21 and second-level index node 22) of the first-level index node 11 based on the second-level index value "name=a" included in the first record 1, and determines the second-level index node 21 that includes the second-level index value range "name:af".

[0109] 4063: The first storage node determines whether k is equal to Q. If k is less than Q, set k=k+1 and return to 4062. If k is equal to Q, execute 4064.

[0110] If k is equal to Q, it means that for the Q-th level index value included in the first record, the first storage node obtains the Q-th level index node including the Q-th level index value range to which the Q-th level index value belongs from the index tree.

[0111] 4064: The first storage node stores the first record in a leaf node of the Q-th level index node.

[0112] In 4064, if there is no leaf node with a data volume less than the data volume threshold among the leaf nodes of the Q-th level index node in the index tree, the first storage node adds a leaf node for the Q-th level index node, and stores the first record in the added leaf node. If there is a leaf node with a data volume less than the data volume threshold among the leaf nodes of the Q-th level index node in the index tree, the first record is stored in the leaf node.

[0113] For example, see Figure 5 , the first storage node adds a leaf node 31 to the second-level index node 21 , and stores the first record 1 in the added leaf node 31 .

[0114] As another example, for the first record 2, the first storage node obtains the first-level index value range "age:9-11" to which the first-level index value "age=11" belongs from the first-level index value range "age:9-11" included in the first-level index node 11 of the root node based on the first-level index value "age=11" included in the first record 2, and determines the first-level index node 11 that includes the first-level index value range "age:9-11". Based on the second-level index value "name=b" included in the first record 2, the second-level index value range "name:af" to which the second-level index value "name=b" belongs is obtained from two second-level index value ranges ("name:af" and "name:xz") included in two second-level index nodes (second-level index node 21 and second-level index node 22) of the first-level index node 11, and determines the second-level index node 21 that includes the second-level index value range "name:af". The data volume of the leaf node 31 of the second-level index node 21 is less than the data volume threshold, and the first record 2 is stored in the leaf node 31 .

[0115] For another example, for the first record 3, the first storage node obtains the first-level index value range "age:9-11" to which the first-level index value "age=10" belongs from the first-level index value range "age:9-11" included in the first-level index node 11 of the root node based on the first-level index value "age=10" included in the first record 3, and determines the first-level index node 11 including the first-level index value range "age:9-11". Based on the second-level index value "name=e" included in the first record 3, the second-level index value range "name:af" to which the second-level index value "name=e" belongs is obtained from two second-level index value ranges ("name:af" and "name:xz") included in two second-level index nodes (second-level index node 21 and second-level index node 22) of the first-level index node 11, and determines the second-level index node 21 including the second-level index value range "name:af". The data volume of the leaf node 31 of the second-level index node 21 is greater than the data volume threshold, and the first storage node adds a leaf node 32 to the second-level index node 21 and stores the first record 3 in the added leaf node 32 .

[0116] According to the above process, the first record 4, the first record 5, the first record 6 and the first record 7 are stored in the leaf nodes of the index tree.

[0117] In some embodiments, the cloud service may also increase the index nodes included in the index tree in the storage node, and may also generate multiple second records, and store the multiple second records in the index tree. The first storage node receives a primary retrieval request and multiple second records, the primary retrieval request includes at least one second range set, the second range set includes a j-th level index value range and at least one j+1-th level index value range corresponding to the j-th level index value range, j=1, 2, 3, ..., Q-1, and there are records in the multiple second records including a j-th level index value belonging to the j-th level index value range and a j+1-th level index value belonging to the j+1-th level index value range.

[0118] The first storage node adds multiple non-leaf nodes and at least one leaf node in the index tree, the multiple non-leaf nodes include the j-th level index node corresponding to the j-th level index value range and at least one j+1-th level index node corresponding to the at least one j+1-th level index value range, the j-th level index node includes the j-th level index value range, each j+1-th level index node respectively includes the j+1-th level index value range corresponding to each j+1-th level index node, the at least one j+1-th level index node is a child node of the j-th level index node, the j-th level index node is a child node of a j-1-th level index node in the index tree, the leaf node is a child node of a node in the multiple non-leaf nodes, the leaf node is used to store at least one second record, the at least one second record is part or all of the multiple second records, the index value range included in the second record stored in the leaf node belongs to the index value range included in each level of index nodes on the path between the leaf node and the root node.

[0119] The multiple non-leaf nodes are multiple index nodes. The first storage node can add the multiple index nodes to the index tree according to the following process (1)-(5).

[0120] (1): The first storage node obtains the j-th level index value range from the first-level retrieval request.

[0121] The initial value of j is 1.

[0122] (2): The first storage node determines whether there is a j-th level index node that includes the j-th level index value range from at least one j-th level index node of the j-1-th level node of the index tree. If there is a j-th level index node that includes the j-th level index value range, execute (3); if there is no j-th level index node that includes the j-th level index value range, execute (4).

[0123] The 0th level index node is the root node of the index tree. The at least one jth level index node is a child node of the j-1th level index node.

[0124] (3): The first storage node sets j=j+1 and returns to execute (1).

[0125] (4): The first storage node adds a j-th level index node to the j-1-th level index node, the j-th level index node includes the j-th level index value range, and the j-th level index node is a child node of the j-1-th level index node.

[0126] (5): The first storage node determines whether j is equal to Q. If j is less than Q, set j=j+1 and return to execute (1). If j is equal to Q, end.

[0127] For example, see Figure 2 , the first storage node receives a first-level retrieval request, which includes a first-level index value range "age:12-15" and a second-level index value range "name:gk". The first-level index value range "age:12-15" is obtained from the first-level retrieval request, and there is no first-level index node including the first-level index value range "age:12-15" in the first-level index node of the 0-level index node. The first storage node adds a first-level index node 12 to the 0-level index node, and the first-level index node 12 includes the first-level index value range "age:12-15". The second-level index value range "name:gk" is obtained from the first-level retrieval request, and there is no second-level index node including the second-level index value range "name:gk" in the second-level index node of the first-level index node 12. A second-level index node 23 is added to the first-level index node, and the second-level index node 23 includes the second-level index value range "name:gk".

[0128] If j=Q, it means that the 0th-level index nodes to the Qth-level index nodes have been added to the index tree in the first storage node. Next, the first storage node can receive the second record sent by the cloud service, add a leaf node to the index tree, and save the second record in the leaf node. For the detailed implementation process, refer to the above-mentioned process of the first storage node saving the first record to the index tree, which will not be described in detail here.

[0129] In an embodiment of the present application, a first storage node receives a first-level retrieval request sent by a client, and the first-level retrieval request includes at least one first range set. For each first range set, the first range set includes an i-th level index value range and at least one i+1-th level index value range, and there are records including an i-th level index value belonging to the i-th level index value range and an i+1-th level index value belonging to the i+1-th level index value range among the multiple first records. The first storage node establishes an index tree based on the i-th level index value range and the at least one i+1-th level index value range, and the index tree includes multiple non-leaf nodes, and the multiple non-leaf nodes include an i-th level index node and at least one i+1-th level index node, the i-th level index node includes the i-th level index value range, the at least one i+1-th level index node includes the at least one i+1-th level index value range, and the at least one i+1-th level index node is a child node of the i-th level index node. The first storage node receives the multiple first records, and adds at least one leaf node in the index tree, the leaf node is a child node of one of the multiple non-leaf nodes, the leaf node is used to store at least one first record, the at least one first record is part or all of the multiple first records, and the multi-level index value range included in the first record stored in the leaf node belongs to the index value range included in each level of index nodes on the path between the leaf node and the root node of the index tree. Since the first-level retrieval request includes the i-th level index value range and at least one i+1-th level index value range corresponding to the i-th level index value range, the first storage node generates a retrieval transaction based on the first-level retrieval request, and in the retrieval transaction, an index tree is established based on the i-th level index value range and the at least one i+1-th level index value range. Since the index tree is established through a retrieval transaction, the time required to establish the index tree is reduced, thereby improving the efficiency of storing data in the index tree.

[0130] See also Figure 6 The present application embodiment provides a data index construction device 600, the device 600 is applied to Figure 1 or Figure 3 In the cloud storage system 100 shown, the device 600 is deployed on a client included in the cloud storage system 100, and the device 600 is used to manage multiple storage nodes set on an infrastructure, at least one of the multiple storage nodes stores user data, and the user's cloud business is also running on the infrastructure. The device 600 includes:

[0131] Processing unit 601 is used to obtain an index operator input by a user, where the index operator includes field parameters of a multi-level index;

[0132] A sending unit 602 is used to send the index operator to a first storage node among the at least one storage node, where the first storage node is any storage node among the at least one storage node;

[0133] The processing unit 601 is further used to configure the first storage node so that the first storage node generates a multi-level index according to the index operator.

[0134] Optionally, the detailed implementation process of the processing unit 601 obtaining the index operator input by the user can be found in Figure 4 The relevant contents of step 401 of the method 400 are not described in detail here.

[0135] Optionally, the detailed implementation process of the sending unit 602 sending the index operator to the first storage node of the at least one storage node is shown in Figure 4 The relevant contents of step 402 of the method 400 are not described in detail here.

[0136] Optionally, the detailed implementation process of the processing unit 601 configuring the first storage node is shown in Figure 4 The relevant contents of steps 403 - 404 of the method 400 are not described in detail here.

[0137] Optionally, the sending unit 602 is further used to send a first-level retrieval request to the first storage node, where the first-level retrieval request is used to enable the first storage node to generate the multi-level index in one retrieval transaction.

[0138] Optionally, the detailed implementation process of the sending unit 602 sending the primary search request is shown in Figure 4 The relevant contents of step 402 of the method 400 are not described in detail here.

[0139] Optionally, the cloud service includes multiple first records, the multi-level index is an index tree, the field parameters of the multi-level index include at least one first range set, the first range set includes the i-th level index value range and at least one i+1-th level index value range, i=1, 2, 3,..., and there are records including the i-th level index value belonging to the i-th level index value range and the i+1-th level index value belonging to the i+1-th level index value range among the multiple first records, and the first-level retrieval request is used to enable the first storage node to establish an index tree based on the at least one first range set and the multiple first records.

[0140] Optionally, the number of index operators input by the user is multiple, and the sending unit 602 is used to send the multiple index operators to the first storage node, so that the first storage node generates multiple multi-level indexes according to the multiple index operators.

[0141] Optionally, the detailed implementation process of the sending unit 602 sending multiple index operators can be found in Figure 4 The relevant contents of step 402 of the method 400 are not described in detail here.

[0142] Optionally, the device 600 further includes a receiving unit 603;

[0143] A receiving unit 603 is used to receive a success response sent by the first storage node after generating the multi-level index;

[0144] The sending unit 602 is further used to send the index operator to a second storage node, where the second storage node is a storage node other than the first storage node of the at least one storage node;

[0145] The processing unit 601 is further used to configure the second storage node so that the second storage node generates a multi-level index according to the index operator.

[0146] Optionally, the detailed implementation process of receiving the successful response by the receiving unit 603 is shown in Figure 4 The relevant contents of step 404 of the method 400 are not described in detail here.

[0147] Optionally, the detailed implementation process of the sending unit 602 sending the index operator to the second storage node can be found in Figure 4 The relevant contents of step 404 of the method 400 are not described in detail here.

[0148] Optionally, the detailed implementation process of the processing unit 601 configuring the second storage node can be found in Figure 4 The relevant contents of step 404 of the method 400 are not described in detail here.

[0149] Optionally, the sending unit 602 is further used to complete the establishment of the multi-level index in at least one storage node, and send the index result to the cloud service, where the index result is used to indicate that the at least one storage node has successfully generated the multi-level index.

[0150] Optionally, the detailed implementation process of the sending unit 602 sending the index result to the cloud service is shown in Figure 4 The relevant contents of step 404 of the method 400 are not described in detail here.

[0151] In an embodiment of the present application, a processing unit obtains an index operator input by a user, and the index operator includes field parameters of a multi-level index. A sending unit sends the index operator to a first storage node among at least one storage node, and the first storage node is any storage node among at least one storage node. The processing unit configures the first storage node so that the first storage node generates a multi-level index according to the index operator. Since the index operator includes field parameters of a multi-level index, the sending unit sends the index operator to the first storage node, so that the processing unit can configure the first storage node so that the first storage node generates a multi-level index according to the index operator in one transaction, which can reduce the time required to generate the multi-level index, thereby storing data in the multi-level index and improving the efficiency of storing data.

[0152] See also Figure 7 , the present application embodiment provides a computing device 700. For example, the computing device 700 may be Figure 1 or Figure 3 The cloud storage system 100 shown includes devices.

[0153] like Figure 7 As shown, computing device 700 includes: bus 702, processor 704, memory 706 and communication interface 708. Processor 704, memory 706 and communication interface 708 communicate through bus 702. Computing device 700 can be a server or a terminal device. It should be understood that the present application does not limit the number of processors and memories in computing device 700.

[0154] The bus 702 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 7 The bus 702 may include a path for transmitting information between various components of the computing device 700 (eg, the memory 706, the processor 704, and the communication interface 708).

[0155] The processor 704 may include any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0156] The memory 706 may include a volatile memory, such as a random access memory (RAM). The processor 704 may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid state drive (SSD).

[0157] See also Figure 7 , the memory 706 stores executable program codes, and the processor 704 executes the executable program codes to respectively implement Figure 6 The functions of the processing unit 601, the sending unit 602 and the receiving unit 603 in the device 600 shown in the figure are implemented to realize the method provided by any of the above embodiments. That is, the memory 706 stores instructions for executing the method provided by any of the above embodiments. Or,

[0158] The communication interface 708 uses a transceiver module such as, but not limited to, a network interface card or a transceiver to implement communication between the computing device 700 and other devices or communication networks.

[0159] The embodiment of the present application also provides a cluster. The cluster includes at least one computing device. The computing device can be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device can also be a terminal device such as a desktop computer, a laptop computer, or a smart phone.

[0160] like Figure 8 As shown, the cluster includes at least one computing device 700. The memory 706 in one or more computing devices 700 in the cluster may store the same instructions for executing the method provided by any of the above embodiments.

[0161] In some possible implementations, the memory 706 of one or more computing devices 700 in the cluster may also store partial instructions for executing the above instance creation method. In other words, the combination of one or more computing devices 700 may jointly execute instructions for executing the instance creation method provided in any of the above embodiments.

[0162] In some possible implementations, one or more computing devices in the cluster may be connected via a network, which may be a wide area network or a local area network. Fig. 9 A possible implementation is shown. Fig. 9As shown, two computing devices 700A and 700B are connected via a network. Specifically, they are connected to the network via a communication interface in each computing device.

[0163] In this type of possible implementation, the memory 706 in the computing device 700A stores the execution Figure 6 Instructions for the functions of the processing unit 601 in the illustrated embodiment. Meanwhile, the memory 706 in the computing device 700B stores instructions for executing the functions of the processing unit 601 in the illustrated embodiment. Figure 6 Instructions for the functions of the sending unit 602 and the receiving unit 603 in the illustrated embodiment.

[0164] It should be understood that Fig. 9 The functions of the computing device 700A shown in FIG. 7 may also be completed by multiple computing devices 700. Similarly, the functions of the computing device 700B may also be completed by multiple computing devices 700.

[0165] The present application embodiment also provides another cluster. The connection relationship between the computing devices in the cluster can be similar to that of Fig. 9 The connection mode of the cluster of the acquisition function is different in that the memory 706 in one or more computing devices 700 in the cluster may store the same instructions for executing the method provided in any of the above embodiments.

[0166] In some possible implementations, the memory 706 of one or more computing devices 700 in the cluster may also store partial instructions for executing the method provided in any of the above embodiments. In other words, the combination of one or more computing devices 700 may jointly execute instructions for executing the method provided in any of the above embodiments.

[0167] The present application also provides a computer program product including instructions. The computer program product may be software or a program product including instructions that can be run on a computing device or stored in any available medium. When the computer program product is run on at least one computing device, the at least one computing device executes the method provided in any of the above embodiments.

[0168] The embodiment of the present application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium that can be stored by a computing device or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk). The computer-readable storage medium includes instructions that instruct the computing device to execute the method provided in any of the above embodiments.

[0169] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0170] The above description is only an optional embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the principles of the present application shall be included in the protection scope of the present application.

Claims

1. A data index construction method, characterized in that: The method is applied to a client, the client is used to manage multiple storage nodes set on an infrastructure, at least one storage node among the multiple storage nodes stores user data, and the infrastructure also runs a cloud service of the user, the method includes: Obtaining the index operator input by the user, wherein the index operator includes field parameters of a multi-level index; Sending the index operator to a first storage node among the at least one storage node, where the first storage node is any storage node among the at least one storage node; The first storage node is configured so that the first storage node generates a multi-level index according to the index operator.

2. The method according to claim 1, characterized in that The configuring the first storage node includes: A first-level retrieval request is sent to the first storage node, where the first-level retrieval request is used to enable the first storage node to generate the multi-level index in one retrieval transaction.

3. The method according to claim 2, characterized in that The cloud service includes multiple first records, the multi-level index is an index tree, the field parameters of the multi-level index include at least one first range set, the first range set includes an i-th level index value range and at least one i+1-th level index value range, i=1, 2, 3,..., among the multiple first records, there are records including an i-th level index value belonging to the i-th level index value range and an i+1-th level index value belonging to the i+1-th level index value range, and the first-level retrieval request is used to enable the first storage node to establish the index tree based on the at least one first range set and the multiple first records.

4. The method according to any one of claims 1 to 3, characterized in that: The number of index operators input by the user is multiple, and sending the index operator to a first storage node among the at least one storage node includes: Sending multiple index operators to the first storage node so that the first storage node generates multiple multi-level indexes according to the multiple index operators.

5. The method according to any one of claims 1 to 4, characterized in that: The method further comprises: Receiving a success response sent by the first storage node after generating the multi-level index; Sending the index operator to a second storage node, where the second storage node is a storage node of the at least one storage node other than the first storage node; The second storage node is configured so that the second storage node generates a multi-level index according to the index operator.

6. The method according to claim 5, characterized in that The method further comprises: After the multi-level index is established in the at least one storage node, an index result is sent to the cloud service, where the index result is used to indicate that the at least one storage node successfully generated the multi-level index.

7. A data index construction device, characterized in that: The device is used to manage multiple storage nodes set on an infrastructure, at least one storage node among the multiple storage nodes stores user data, and the user's cloud service is also running on the infrastructure, and the device includes: A processing unit, configured to obtain an index operator input by the user, wherein the index operator includes field parameters of a multi-level index; A sending unit, configured to send the index operator to a first storage node among the at least one storage node, where the first storage node is any storage node among the at least one storage node; The processing unit is further used to configure the first storage node so that the first storage node generates a multi-level index according to the index operator.

8. The device according to claim 7, characterized in that The sending unit is further used to send a first-level retrieval request to the first storage node, where the first-level retrieval request is used to enable the first storage node to generate the multi-level index in one retrieval transaction.

9. The device according to claim 8, characterized in that The cloud service includes multiple first records, the multi-level index is an index tree, the field parameters of the multi-level index include at least one first range set, the first range set includes an i-th level index value range and at least one i+1-th level index value range, i=1, 2, 3,..., among the multiple first records, there are records including an i-th level index value belonging to the i-th level index value range and an i+1-th level index value belonging to the i+1-th level index value range, and the first-level retrieval request is used to enable the first storage node to establish the index tree based on the at least one first range set and the multiple first records.

10. The device according to any one of claims 7 to 9, characterized in that: The number of index operators input by the user is multiple, and the sending unit is used to send the multiple index operators to the first storage node, so that the first storage node generates multiple multi-level indexes according to the multiple index operators.

11. The device according to any one of claims 7 to 10, characterized in that: The device also includes a receiving unit; The receiving unit is configured to receive a success response sent by the first storage node after generating the multi-level index; The sending unit is further used to send the index operator to a second storage node, where the second storage node is a storage node of the at least one storage node other than the first storage node; The processing unit is further used to configure the second storage node so that the second storage node generates a multi-level index according to the index operator.

12. The device according to claim 11, characterized in that The sending unit is further used to complete establishing the multi-level index in the at least one storage node and send an index result to the cloud service, wherein the index result is used to indicate that the at least one storage node has successfully generated the multi-level index.

13. A computing device cluster, characterized in that: The cluster includes at least one computing device, each of the at least one computing device includes at least one processor and at least one memory, and the at least one memory stores computer-readable instructions; the at least one processor executes the computer-readable instructions so that the cluster executes the method according to any one of claims 1 to 6.

14. A computer storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

15. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images