Data processing method and device, storage medium and electronic equipment

Abstract

The invention discloses a data processing method and device, a storage medium and electronic equipment. Relates to the technical field of data processing. According to the target logic address information, a target self-adaptive radix tree is determined from multiple self-adaptive radix trees, the self-adaptive radix trees are composed of key value pairs corresponding to multiple pieces of metadata information, and the key value pairs are composed of logic addresses in the metadata information and physical addresses in the metadata information; and querying according to the target logic address information and the target adaptive cardinal tree to obtain target metadata information, and obtaining data information stored in a physical address in the target metadata information through the target metadata information. Through the data volume processing method and device, the problem that the processing efficiency of the metadata information is low due to the fact that the time complexity of operations such as searching and deleting of the red-black tree is high when the red-black tree serves as a memory structure of the metadata information of the data volume in the related technology is solved.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a data processing method and device, a storage medium, and an electronic device. Background Art

[0002] A distributed block storage system is a storage system that stores data in a distributed manner on multiple storage servers, and these distributed storage resources form a virtual storage device, providing block (read and write by location) semantics to the outside world. The metadata information of the block needs to be queried during each read and write io (Input / Output, data input / output operation) process, and updated during the write io process. In the prior art, ordinary red-black trees are used as the memory structure of metadata information, but the time complexity of the search, insertion and deletion operations of the red-black tree is high and cannot provide efficient operations. In addition, the keys of the metadata information of multiple nodes may contain the same prefix, but a copy is stored in each node, resulting in additional memory overhead.

[0003] Regarding the problem that the red-black tree is used as the memory structure of the metadata information of the data volume in the related technology, the time complexity of the red-black tree search, deletion and other operations is high, resulting in low efficiency in processing the metadata information. No effective solution has been proposed yet. Summary of the invention

[0004] The main purpose of the present application is to provide a data processing method and device, a storage medium and an electronic device to solve the problem that in the related art, a red-black tree is used as the memory structure of the metadata information of a data volume, and the time complexity of operations such as searching and deleting the red-black tree is high, resulting in low processing efficiency of the metadata information.

[0005] In order to achieve the above-mentioned purpose, according to one aspect of the present application, a data processing method is provided. The method comprises: obtaining a query request, wherein the query request at least includes the target logical address information in the target metadata information to be queried; determining a target adaptive radix tree from multiple adaptive radix trees according to the target logical address information, wherein the adaptive radix tree is composed of key-value pairs corresponding to multiple metadata information, and the key-value pairs are composed of the logical address in the metadata information and the physical address in the metadata information; querying according to the target logical address information and the target adaptive radix tree to obtain the target metadata information, wherein the data information stored in the physical address in the target metadata information is obtained through the target metadata information.

[0006] Furthermore, before determining the target adaptive radix tree from a plurality of adaptive radix trees according to the target logical address information, the method further includes: determining a plurality of data volumes currently existing in the distributed block storage system; sharding each data volume to obtain a plurality of data shards; and constructing an adaptive radix tree corresponding to each data shard according to the first metadata information corresponding to each data shard.

[0007] Furthermore, slicing each data volume to obtain multiple data slicings includes: obtaining the storage capacity and preset slicing rules corresponding to each data volume; slicing each data volume according to the storage capacity and the preset slicing rules to obtain the multiple data slicings.

[0008] Furthermore, based on the first metadata information corresponding to each data shard, constructing an adaptive radix tree corresponding to each data shard includes: for each data shard, obtaining multiple first metadata information corresponding to the data shard; based on the logical address in each first metadata information, constructing the nodes of the adaptive radix tree; based on the physical address in each first metadata information, determining the leaves of the adaptive radix tree; and based on the nodes of the adaptive radix tree and the leaves of the adaptive radix tree, constructing the adaptive radix tree corresponding to the data shard.

[0009] Furthermore, determining the leaves of the adaptive radix tree based on the physical address in each first metadata information includes: obtaining a preset array; encoding and compressing the physical address based on the array subscript of the preset array to obtain the encoded physical address; and determining the leaves of the adaptive radix tree based on the encoded physical address.

[0010] Furthermore, after querying according to the target logical address information and the target adaptive radix tree to obtain the target metadata information, the method also includes: obtaining a new request, wherein the new request includes at least second metadata information to be added to the target adaptive radix tree; obtaining a first quantity of the second metadata information and a second quantity of current leaf nodes of the target adaptive radix tree; judging whether it is necessary to modify the node type corresponding to the leaf nodes in the target adaptive radix tree based on the first quantity and the second quantity; if it is not necessary to modify the node type corresponding to the leaf nodes in the target adaptive radix tree, updating the target adaptive radix tree based on the second metadata information to obtain an updated target adaptive radix tree.

[0011] Further, judging whether it is necessary to modify the node type corresponding to the leaf node in the target adaptive radix tree based on the first quantity and the second quantity includes: performing a sum calculation based on the first quantity and the second quantity to obtain a target quantity; obtaining the current first node type of the leaf node in the target adaptive radix tree, and obtaining a node quantity threshold corresponding to the first node type; judging whether to modify the node type corresponding to the leaf node in the target adaptive radix tree based on the target quantity and the node quantity threshold.

[0012] Furthermore, after determining whether it is necessary to modify the node type corresponding to the leaf nodes in the target adaptive radix tree based on the first quantity and the second quantity, the method further includes: if it is necessary to modify the node type corresponding to the leaf nodes in the target adaptive radix tree, modifying the leaf nodes in the target adaptive radix tree to the second node type to obtain a modified target adaptive radix tree, wherein the node quantity threshold corresponding to the second node type is different from the node quantity threshold corresponding to the first node type; and updating the modified target adaptive radix tree based on the second metadata information to obtain an updated target adaptive radix tree.

[0013] Furthermore, after querying based on the target logical address information and the target adaptive radix tree to obtain the target metadata information, the method also includes: obtaining a deletion request, wherein the deletion request at least includes: a logical address range in the third metadata information to be deleted; determining leaf nodes to be pruned from the target adaptive radix tree based on the logical address range in the third metadata information to be deleted; and pruning the leaf nodes to be pruned to obtain a pruned target adaptive radix tree.

[0014] In order to achieve the above-mentioned purpose, according to another aspect of the present application, a data processing device is provided. The device includes: a first acquisition unit, used to acquire a query request, wherein the query request at least includes the target logical address information in the target metadata information to be queried; a first determination unit, used to determine a target adaptive radix tree from multiple adaptive radix trees based on the target logical address information, wherein the adaptive radix tree is composed of key-value pairs corresponding to multiple metadata information, and the key-value pairs are composed of the logical address in the metadata information and the physical address in the metadata information; a query unit, used to query based on the target logical address information and the target adaptive radix tree to obtain the target metadata information, wherein the data information stored in the physical address in the target metadata information is obtained through the target metadata information.

[0015] Furthermore, the device also includes: a second determination unit, used to determine multiple data volumes currently existing in the distributed block storage system before determining the target adaptive radix tree from multiple adaptive radix trees based on the target logical address information; a processing unit, used to perform sharding processing on each data volume to obtain multiple data shards; and a construction unit, used to construct an adaptive radix tree corresponding to each data shard based on the first metadata information corresponding to each data shard.

[0016] Furthermore, the processing unit includes: a first acquisition subunit, used to obtain the storage capacity and preset sharding rules corresponding to each data volume; a sharding subunit, used to shard each data volume according to the storage capacity and the preset sharding rules to obtain the multiple data shards.

[0017] Furthermore, the construction unit includes: a second acquisition subunit, used to acquire multiple first metadata information corresponding to each data slice; a first construction subunit, used to construct the nodes of the adaptive radix tree according to the logical address in each first metadata information; a determination subunit, used to determine the leaves of the adaptive radix tree according to the physical address in each first metadata information; and a second construction subunit, used to construct the adaptive radix tree corresponding to the data slice based on the nodes of the adaptive radix tree and the leaves of the adaptive radix tree.

[0018] Furthermore, the determination subunit includes: an acquisition module for acquiring a preset array; an encoding module for encoding and compressing the physical address according to the array subscript of the preset array to obtain the encoded physical address; and a determination module for determining the leaves of the adaptive radix tree based on the encoded physical address.

[0019] Furthermore, the device also includes: a first acquisition unit, used to obtain a new request after querying according to the target logical address information and the target adaptive radix tree to obtain the target metadata information, wherein the new request at least includes second metadata information to be added to the target adaptive radix tree; a second acquisition unit, used to obtain a first quantity of the second metadata information and a second quantity of current leaf nodes of the target adaptive radix tree; a judgment unit, used to judge whether it is necessary to modify the node type corresponding to the leaf node in the target adaptive radix tree according to the first quantity and the second quantity; a first update unit, used to update the target adaptive radix tree according to the second metadata information if it is not necessary to modify the node type corresponding to the leaf node in the target adaptive radix tree, so as to obtain an updated target adaptive radix tree.

[0020] Furthermore, the judgment unit includes: a calculation subunit, used to perform a sum calculation based on the first quantity and the second quantity to obtain a target quantity; a third acquisition subunit, used to obtain the current first node type of the leaf node in the target adaptive radix tree, and obtain a node quantity threshold corresponding to the first node type; a judgment subunit, used to judge whether to modify the node type corresponding to the leaf node in the target adaptive radix tree based on the target quantity and the node quantity threshold.

[0021] Furthermore, the device also includes: a modification unit, which is used to determine whether it is necessary to modify the node type corresponding to the leaf node in the target adaptive radix tree based on the first number and the second number, and if it is necessary to modify the node type corresponding to the leaf node in the target adaptive radix tree, modify the leaf node in the target adaptive radix tree to the second node type to obtain a modified target adaptive radix tree, wherein the node number threshold corresponding to the second node type is different from the node number threshold corresponding to the first node type; a second updating unit, which is used to update the modified target adaptive radix tree based on the second metadata information to obtain an updated target adaptive radix tree.

[0022] Furthermore, the device also includes: a third acquisition unit, used to obtain a deletion request after querying according to the target logical address information and the target adaptive radix tree to obtain the target metadata information, wherein the deletion request at least includes: a logical address range in the third metadata information to be deleted; a third determination unit, used to determine the leaf nodes to be pruned from the target adaptive radix tree according to the logical address range in the third metadata information to be deleted; and a pruning unit, used to prune the leaf nodes to be pruned to obtain the pruned target adaptive radix tree.

[0023] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is provided, wherein the storage medium stores a program, wherein when the program is executed, the device where the storage medium is located is controlled to execute any one of the above-mentioned data processing methods.

[0024] According to another aspect of an embodiment of the present invention, there is further provided an electronic device, comprising a memory storing an executable program; and a processor for running the program, wherein the program executes any one of the above-mentioned data processing methods when running.

[0025] Through this application, the following steps are adopted: obtaining a query request, wherein the query request at least includes the target logical address information in the target metadata information to be queried; determining the target adaptive radix tree from multiple adaptive radix trees according to the target logical address information, wherein the adaptive radix tree is composed of key-value pairs corresponding to multiple metadata information, and the key-value pair is composed of the logical address in the metadata information and the physical address in the metadata information; querying according to the target logical address information and the target adaptive radix tree to obtain the target metadata information, wherein the data information stored in the physical address in the target metadata information is obtained through the target metadata information, which solves the problem that the memory structure of the metadata information of the data volume is used as the red-black tree in the related technology, and the time complexity of the operations such as searching and deleting the red-black tree is high, resulting in low processing efficiency of the metadata information. In this scheme, an adaptive radix tree is used as the memory structure of the metadata information in the distributed block storage system. When a query request is received, the final target metadata information is obtained by querying in the adaptive radix tree according to the target logical address information in the query request. Each node of the adaptive radix tree can store multiple children, thereby making the tree structure flatter and reducing the number of vertical queries, thereby achieving the effect of improving the processing efficiency of the metadata information. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The drawings constituting a part of the present application are used to provide a further understanding of the present application. The illustrative embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0027] Figure 1 is a flow chart of a data processing method provided according to an embodiment of the present application;

[0028] Figure 2 is a schematic diagram of an adaptive radix tree provided according to an embodiment of the present application;

[0029] Figure 3 is a schematic diagram of data slicing provided according to an embodiment of the present application;

[0030] Figure 4 is a schematic diagram of array encoding provided according to an embodiment of the present application;

[0031] Figure 5 is a schematic diagram of a node type provided according to an embodiment of the present application;

[0032] Figure 6 is a schematic diagram of a data processing device provided according to an embodiment of the present application;

[0033] Figure 7 is a schematic diagram of an electronic device provided according to an embodiment of the present application. DETAILED DESCRIPTION

[0034] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0035] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0036] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present application described here. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0037] For the convenience of description, some nouns or terms involved in the embodiments of the present application are explained below:

[0038] Adaptive radix tree: Compressed prefix tree is a more space-saving prefix tree and multi-branch search tree. The middle nodes of the tree do not save leaves but point to child nodes, and leaves are only saved in the leaf nodes of the last layer. Each node includes a fixed number of slots, which is 2 to the power of n, called the base of the adaptive radix tree. The main feature is to store a set of strings in the order of characters and save space by sharing common prefixes. Its structure is a multi-branch tree, each node represents a character, starting from the root node, each node has several child nodes, each child node represents the next character, the adaptive radix tree used in this application has an n of 8 and a base of 2^8, which is 256.

[0039] Metadata information: the mapping between the logical location of the data volume and the physical location of the underlying storage server.

[0040] It should be noted that the relevant information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for display, data for analysis, etc.) involved in this disclosure are all information and data authorized by the user or fully authorized by all parties. For example, an interface is set up between this system and the relevant users or organizations. Before obtaining relevant information, it is necessary to send an acquisition request to the aforementioned user or organization through the interface, and obtain relevant information after receiving the consent information fed back by the aforementioned user or organization.

[0041] The present invention is described below in conjunction with preferred implementation steps. Figure 1 is a flow chart of a data processing method provided according to an embodiment of the present application, such as Figure 1 As shown, the method comprises the following steps:

[0042] Step S101: obtaining a query request, wherein the query request at least includes target logical address information in target metadata information to be queried.

[0043] Optionally, in order to achieve efficient index query, an adaptive radix tree is used as the memory structure of the metadata information, and the adaptive radix tree is composed of the logical address in the metadata information and the physical address in the metadata information. In an optional embodiment, the above query request can be received through the front-end interface. The query request needs to include the target logical address information in the target metadata information to be queried. It should be noted that the query request also includes the length of the data to be queried.

[0044] Step S102, determining a target adaptive radix tree from a plurality of adaptive radix trees according to the target logical address information, wherein the adaptive radix tree is composed of key-value pairs corresponding to a plurality of metadata information, and the key-value pairs are composed of a logical address in the metadata information and a physical address in the metadata information.

[0045] Optionally, a target adaptive radix tree storing the target logical address is determined from multiple adaptive radix trees according to the target logical address information in the query request. An adaptive radix tree (Radix Tree) is a Trie (prefix tree). Trie splits the key (i.e., the logical address in the metadata information above) into subsequences in order from root to leaf. The order from root to leaf is called a path. The complete path constitutes the key of the leaf. If multiple keys have the same prefix, the prefix part is only stored once. Figure 2AND, ANT and ANY have the same prefix AN, and AN is only stored once. ARE and ART have the same prefix AR, and AR is only stored once. The special feature of the adaptive radix tree is that each node stores multiple children, which makes the tree structure flatter, reduces the number of vertical queries and improves query efficiency. It can be compared with some commonly used data representations, such as the hexadecimal base is 16 and the ascii code base is 256. If the key in the uint32 range is stored in an adaptive radix tree with base 16 (analogous to the hexadecimal representation), then each node represents a 4-bit key, and it needs to store the address of the next 4-bit child node. The possible range of child values ​​is [0-15], with a maximum of 16 address pointers.

[0046] Step S103 , querying according to the target logical address information and the target adaptive radix tree to obtain target metadata information, wherein data information stored in the physical address in the target metadata information is obtained through the target metadata information.

[0047] Optionally, after determining the target adaptive radix tree, the target adaptive radix tree is queried through the target logical address information, the corresponding leaf node is found and the target metadata information is obtained, and the data information stored in the physical address in the target metadata information is obtained through the target metadata information.

[0048] In summary, an adaptive radix tree is used as the memory structure of metadata information in a distributed block storage system. When a query request is received, the adaptive radix tree is queried according to the target logical address information in the query request to obtain the final target metadata information. Each node of the adaptive radix tree can store multiple children, making the tree structure flatter and reducing the number of vertical queries, thereby achieving the effect of improving the processing efficiency of metadata information.

[0049] Optionally, in the data processing method provided in the embodiment of the present application, before determining the target adaptive radix tree from multiple adaptive radix trees based on the target logical address information, the method also includes: determining multiple data volumes currently existing in the distributed block storage system; sharding each data volume to obtain multiple data shards; and constructing an adaptive radix tree corresponding to each data shard based on the first metadata information corresponding to each data shard.

[0050] In an optional embodiment, in order to further reduce the scale of the adaptive radix tree and improve the efficiency of subsequent querying metadata information or updating metadata information, the following steps are also included: introducing the concept of shards to split the blocks (i.e., the above-mentioned multiple data volumes) displayed by the distributed block storage system into smaller units within the system, which are called shards here, that is, sharding each data volume, and finally constructing an adaptive radix tree corresponding to each data shard. It should be noted that one data shard corresponds to one adaptive radix tree. The logical offset at the volume level needs to be represented by 64 bits, but the offset within the shard is limited by the length of the shard (for example, the length of the shard is 32GiB). If one bit represents 4KiB of data, only 23 bits are needed to represent it.

[0051] By taking advantage of the fixed number of vertical queries in the adaptive radix tree, which depends on the height of the adaptive radix tree and the height of the tree depends on the length of the key, an ART is built for each shard. Since each shard only needs 23 bits to represent the offset within the shard, each query only needs to search three times from the root to find the node where the target leaf is located.

[0052] Since the logical address range of each shard representation block is of limited length, it is possible to convert longer logical offsets at the volume level into smaller logical offsets, thereby effectively reducing the size of the adaptive radix tree.

[0053] Optionally, in the data processing method provided in the embodiment of the present application, sharding each data volume to obtain multiple data shards includes: obtaining the storage capacity and preset sharding rules corresponding to each data volume; sharding each data volume according to the storage capacity and the preset sharding rules to obtain multiple data shards.

[0054] In an optional embodiment, when performing sharding, the storage capacity and preset sharding rules corresponding to each data volume are clarified, and then each data volume is sharded according to the storage capacity and the preset sharding rules to obtain multiple data shards. It should be noted that the preset sharding rule here is to split different data volumes into one data shard. For example, Figure 3 The data sharding process shown in the figure has a shard length of 32 GiB. According to the sharding rule of splitting different data volumes into one data shard, a 256 GiB volume is mapped into 8 shards. Figure 3 0-7 in the byte represents 8 data shards.

[0055] Through the above data sharding, different shards can be scheduled to different physical threads for processing, so as to make full use of the CPU resources in the system and improve the overall performance of the system, reduce the latency of a single IO, and improve the processing efficiency of metadata information.

[0056] Optionally, in the data processing method provided in the embodiment of the present application, constructing an adaptive radix tree corresponding to each data shard based on the first metadata information corresponding to each data shard includes: for each data shard, obtaining multiple first metadata information corresponding to the data shard; constructing a node of the adaptive radix tree based on the logical address in each first metadata information; determining a leaf of the adaptive radix tree based on the physical address in each first metadata information; and constructing an adaptive radix tree corresponding to the data shard based on the nodes of the adaptive radix tree and the leaves of the adaptive radix tree.

[0057] In an optional embodiment, the following steps are used to construct an adaptive radix tree corresponding to each data shard. For each data shard, multiple first metadata information corresponding to the data shard is obtained, that is, the corresponding logical address information and physical address information are obtained, and then the logical address in each first metadata information is split and stored in each subsequence in order from root to leaf. The order from root to leaf is called a path. The complete path constitutes the logical address information of the leaf. If multiple keys have the same prefix, the prefix part is only stored once. The last layer node where the leaf is located does not use a child pointer but directly embeds the value (that is, the physical address mentioned above) into the ART tree, thereby reducing memory usage. Finally, the adaptive radix tree corresponding to the data shard is constructed according to the node leaf.

[0058] Optionally, in the data processing method provided in the embodiment of the present application, determining the leaves of the adaptive radix tree based on the physical address in each first metadata information includes: obtaining a preset array; encoding and compressing the physical address based on the array subscript of the preset array to obtain the encoded physical address; and determining the leaves of the adaptive radix tree based on the encoded physical address.

[0059] In an optional embodiment, in order to minimize memory usage, a coding table is introduced to encode and compress part of the information in the value (i.e., the physical address mentioned above), and the information in the value includes: a 64-bit chunk id, a 32-bit chunk offset, and a 16-bit chunk write length. If no compression is performed, after memory alignment, the value occupies 128 bits (the memory usage must be an integer multiple of the maximum item). By compressing the chunk id to 14 bits, limiting the chunk offset to 16 bits (addressing range is 256 MiB), and limiting the chunk write length to 5 bits (maximum representation length is 128 KiB), only 40 bits (14+16+5=35 bits according to the result of byte alignment) are needed to store the value.

[0060] The following steps are used for compression: a pre-allocated array is used, one array item is 64 bits, and the number of array items is 2^14, so the total memory usage is At the same time, a bitmap can be introduced to record whether the array items have been allocated, thereby speeding up the query. Figure 4 As shown in the figure, the values ​​2065, 3089, and 1010 in the box represent the chunk ids before encoding, and the index number of the encoding array (numbering starts from 0) is used as the value after encoding; black represents the allocated array items, and white represents the unallocated array items. The array subscript (numbering starts from 0, i.e. 0, 1, 2, 3, 4, 5, 6) is used to represent the encoding result.

[0061] By encoding and compressing the physical address in the above manner, memory usage can be effectively reduced.

[0062] Optionally, in the data processing method provided in the embodiment of the present application, after querying based on the target logical address information and the target adaptive radix tree to obtain the target metadata information, the method also includes: obtaining a new request, wherein the new request includes at least second metadata information to be added to the target adaptive radix tree; obtaining a first quantity of the second metadata information and a second quantity of current leaf nodes of the target adaptive radix tree; judging whether it is necessary to modify the node type corresponding to the leaf nodes in the target adaptive radix tree based on the first quantity and the second quantity; if it is not necessary to modify the node type corresponding to the leaf nodes in the target adaptive radix tree, updating the target adaptive radix tree based on the second metadata information to obtain an updated target adaptive radix tree.

[0063] In an optional embodiment, after constructing the target adaptive radix tree, if second metadata information to be added to the target adaptive radix tree is obtained, it is necessary to determine whether to modify the node type corresponding to the leaf node in the target adaptive radix tree based on the first number of the second metadata information and the second number of the current leaf nodes of the target adaptive radix tree; if not necessary, the second metadata information is directly added to the target adaptive radix tree to obtain an updated target adaptive radix tree.

[0064] For example, ART has four built-in node types with a base of 256 (exactly one byte), such as Figure 5 As shown, there are 4 types of nodes: Node4 type stores up to 4 (0~3) children, for example, abcd, and the key and child pointer are stored separately; Node16 type stores up to 16 (0~15) children, and the key and child pointer are stored separately, and both are fixedly allocated to meet the maximum storage needs. In Node4 type and Node16 type, the key is used to store the key of the child node (for example, 0, 2, 3, 255), and the child pointer is used to store the pointer to the child node. The key and child pointer are one-to-one corresponding. Node48 type stores up to 48 children, and the pointer area is fixedly allocated 48 (0~47) slots, but the child index (child node index) is fixedly allocated with a base of 256 slots, thereby speeding up the query within the node. The above 3 types are indirect addressing, Node256 type stores up to 256 children, and 256 (0~255) slots are fixedly allocated to store the child pointer. The child index and child pointer are merged and do not need to be stored separately.

[0065] The memory usage can be adjusted dynamically based on the current number of leaf nodes and the amount of metadata information, thereby improving memory utilization.

[0066] Optionally, in the data processing method provided in the embodiment of the present application, judging whether it is necessary to modify the node type corresponding to the leaf node in the target adaptive radix tree based on the first quantity and the second quantity includes: performing a sum calculation based on the first quantity and the second quantity to obtain the target quantity; obtaining the current first node type of the leaf node in the target adaptive radix tree, and obtaining a node quantity threshold corresponding to the first node type; judging whether to modify the node type corresponding to the leaf node in the target adaptive radix tree based on the target quantity and the node quantity threshold.

[0067] In an optional embodiment, the following steps are used to determine whether it is necessary to modify the node type corresponding to the leaf node in the target adaptive radix tree: calculate the sum of the first number and the second number to obtain the above-mentioned target number, then determine the current first node type of the leaf node in the target adaptive radix tree and the threshold corresponding to the node type, and determine whether to modify the node type corresponding to the leaf node in the target adaptive radix tree based on the target number and the node number threshold. For example, if the target number is greater than the node number threshold, it is necessary to expand the node, that is, to modify the node type corresponding to the leaf node in the target adaptive radix tree. If the target number is less than or equal to the node number threshold, it is not necessary to modify the node type corresponding to the leaf node in the target adaptive radix tree.

[0068] Optionally, in the data processing method provided in the embodiment of the present application, after determining whether it is necessary to modify the node type corresponding to the leaf node in the target adaptive radix tree based on the first quantity and the second quantity, the method further includes: if it is necessary to modify the node type corresponding to the leaf node in the target adaptive radix tree, modifying the leaf node in the target adaptive radix tree to the second node type to obtain a modified target adaptive radix tree, wherein a node quantity threshold corresponding to the second node type is different from a node quantity threshold corresponding to the first node type; and updating the modified target adaptive radix tree based on the second metadata information to obtain an updated target adaptive radix tree.

[0069] In an optional embodiment, if the node type corresponding to the leaf node in the target adaptive radix tree needs to be modified, the leaf node in the target adaptive radix tree is modified to the second node type to obtain a modified target adaptive radix tree. For example, when the first leaf is inserted, a leaf node of type Node4 is assigned. When the number of leaves of a node exceeds 4, it is expanded to Node16, and then to Node48 and Node256 in sequence. Finally, the second metadata information is updated to the modified target adaptive radix tree.

[0070] Optionally, in the data processing method provided in the embodiment of the present application, after querying based on the target logical address information and the target adaptive radix tree to obtain the target metadata information, the method also includes: obtaining a deletion request, wherein the deletion request at least includes: the logical address range in the third metadata information to be deleted; determining the leaf nodes to be pruned from the target adaptive radix tree based on the logical address range in the third metadata information to be deleted; and pruning the leaf nodes to be pruned to obtain the pruned target adaptive radix tree.

[0071] In an optional embodiment, if a deletion request is received, the corresponding metadata information needs to be deleted from the target adaptive radix tree. It should be noted that the deletion request at least includes: the logical address range in the third metadata information to be deleted. For example, in the target adaptive radix tree, there is a leaf node corresponding to a value of 0, 1-15k, 0 is the compressed chunk id, 1-15k is the length written in the chunk, and the third metadata information to be deleted is 0, 1-15k, then the leaf node can be directly deleted. If the third metadata information to be deleted is 0, 6-8k, then the leaf node needs to be pruned into two leaf nodes of 0, 1-5k and 0, 9-15k.

[0072] In an optional embodiment, when leaves are deleted, if the number of leaves can be carried by the next level node type, the capacity is reduced to the node of the next level type. For example, when leaves in Node48 are deleted, if the number of leaves after deletion is less than or equal to 16, Node48 is reduced to Node16, but in order to avoid oscillation of expansion and reduction at the specification boundary, the reduction operation can be performed when the number of leaves is less than or equal to 3 / 4 of the maximum capacity of the next level type node. For example, when the number of leaves in Node48 is less than or equal to 12, Node48 is reduced to Node16.

[0073] In an optional embodiment, when a leaf is searched, the search path is recorded, allowing backtracking and then forward or backward searching for other sibling nodes. Based on the implementation of the search path, range search, update and deletion operations of multiple leaves can be quickly implemented.

[0074] In an optional embodiment, in an adaptive radix tree, the expansion or contraction of leaf nodes will not affect the number of intermediate nodes, that is, the number of intermediate nodes in the adaptive radix tree will not change. When the data shard is 32GiB, only three layers of ART are used in each shard, that is, only two layers of non-leaf nodes, and the number of nodes is very small. Therefore, when allocating non-leaf nodes, Node256 is directly used to avoid memory duplication caused by multiple expansions. At the same time, these non-leaf nodes are not allowed to be contracted, thereby accelerating the efficiency of ART search and update.

[0075] In an optional embodiment, based on the architecture of block shard logic, shards can be assigned to specific physical threads according to a certain strategy, so as to realize concurrent processing of IO operations on different shards. At the same time, when IO operations belonging to the same shard index the cache memory structure, they are processed in the same thread, so there is no need to lock mutual exclusion, thereby avoiding the complex logic and additional performance overhead introduced by locking.

[0076] The data processing method provided by the embodiment of the present application obtains a query request, wherein the query request at least includes the target logical address information in the target metadata information to be queried; according to the target logical address information, the target adaptive radix tree is determined from multiple adaptive radix trees, wherein the adaptive radix tree is composed of key-value pairs corresponding to multiple metadata information, and the key-value pair is composed of the logical address in the metadata information and the physical address in the metadata information; query is performed according to the target logical address information and the target adaptive radix tree to obtain the target metadata information, wherein the data information stored in the physical address in the target metadata information is obtained through the target metadata information, which solves the problem that the memory structure of the metadata information of the data volume is used as the red-black tree in the related technology, and the time complexity of the operations such as searching and deleting the red-black tree is high, resulting in low processing efficiency of the metadata information. In this scheme, an adaptive radix tree is used as the memory structure of the metadata information in the distributed block storage system. When a query request is received, the final target metadata information is obtained by querying in the adaptive radix tree according to the target logical address information in the query request. Each node of the adaptive radix tree can store multiple children, thereby making the tree structure flatter and reducing the number of vertical queries, thereby achieving the effect of improving the processing efficiency of the metadata information.

[0077] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0078] The embodiment of the present application also provides a data processing device. It should be noted that the data processing device of the embodiment of the present application can be used to execute the data processing method provided by the embodiment of the present application. The data processing device provided by the embodiment of the present application is introduced below.

[0079] Figure 6 is a schematic diagram of a data processing device according to an embodiment of the present application. Figure 6 As shown, the device includes: a first acquisition unit 601, a first determination unit 602 and a query unit 603.

[0080] A first acquisition unit 601 is used to acquire a query request, wherein the query request at least includes target logical address information in target metadata information to be queried;

[0081] A first determining unit 602 is used to determine a target adaptive radix tree from a plurality of adaptive radix trees according to target logical address information, wherein the adaptive radix tree is composed of key-value pairs corresponding to a plurality of metadata information, and the key-value pairs are composed of a logical address in the metadata information and a physical address in the metadata information;

[0082] The query unit 603 is used to query according to the target logical address information and the target adaptive radix tree to obtain the target metadata information, wherein the data information stored in the physical address in the target metadata information is obtained through the target metadata information.

[0083] The data processing device provided by the embodiment of the present application obtains a query request through a first acquisition unit 601, wherein the query request at least includes target logical address information in the target metadata information to be queried; a first determination unit 602 determines a target adaptive radix tree from multiple adaptive radix trees based on the target logical address information, wherein the adaptive radix tree is composed of key-value pairs corresponding to multiple metadata information, and the key-value pairs are composed of a logical address in the metadata information and a physical address in the metadata information; a query unit 603 performs a query based on the target logical address information and the target adaptive radix tree to obtain target metadata information, wherein data information stored in a physical address in the target metadata information is obtained through the target metadata information, thereby solving the problem in the related art of using a red-black tree as the memory structure of metadata information of a data volume, and low processing efficiency of metadata information due to high time complexity of operations such as searching and deleting of the red-black tree. In this solution, an adaptive radix tree is used as the memory structure of metadata information in a distributed block storage system. When a query request is received, the adaptive radix tree is queried according to the target logical address information in the query request to obtain the final target metadata information. Each node of the adaptive radix tree can store multiple children, making the tree structure flatter and reducing the number of vertical queries, thereby achieving the effect of improving the processing efficiency of metadata information.

[0084] Optionally, in the data processing device provided in the embodiment of the present application, the device also includes: a second determination unit, used to determine multiple data volumes currently existing in the distributed block storage system before determining the target adaptive radix tree from multiple adaptive radix trees based on the target logical address information; a processing unit, used to perform sharding processing on each data volume to obtain multiple data shards; and a construction unit, used to construct an adaptive radix tree corresponding to each data shard based on the first metadata information corresponding to each data shard.

[0085] Optionally, in the data processing device provided in the embodiment of the present application, the processing unit includes: a first acquisition sub-unit, used to obtain the storage capacity and preset sharding rules corresponding to each data volume; a sharding sub-unit, used to shard each data volume according to the storage capacity and the preset sharding rules to obtain multiple data shards.

[0086] Optionally, in the data processing device provided in the embodiment of the present application, the construction unit includes: a second acquisition sub-unit, used to obtain multiple first metadata information corresponding to each data shard; a first construction sub-unit, used to construct a node of an adaptive radix tree based on a logical address in each first metadata information; a determination sub-unit, used to determine a leaf of the adaptive radix tree based on a physical address in each first metadata information; and a second construction sub-unit, used to construct an adaptive radix tree corresponding to the data shard based on the nodes of the adaptive radix tree and the leaves of the adaptive radix tree.

[0087] Optionally, in the data processing device provided in the embodiment of the present application, the determination subunit includes: an acquisition module, used to acquire a preset array; an encoding module, used to encode and compress the physical address according to the array subscript of the preset array to obtain the encoded physical address; and a determination module, used to determine the leaves of the adaptive radix tree based on the encoded physical address.

[0088] Optionally, in the data processing device provided in the embodiment of the present application, the device also includes: a first acquisition unit, used to obtain a new request after querying according to the target logical address information and the target adaptive radix tree to obtain the target metadata information, wherein the new request at least includes the second metadata information to be added to the target adaptive radix tree; a second acquisition unit, used to obtain a first quantity of the second metadata information and a second quantity of the current leaf nodes of the target adaptive radix tree; a judgment unit, used to judge whether it is necessary to modify the node type corresponding to the leaf node in the target adaptive radix tree based on the first quantity and the second quantity; a first update unit, used to update the target adaptive radix tree based on the second metadata information if it is not necessary to modify the node type corresponding to the leaf node in the target adaptive radix tree, so as to obtain an updated target adaptive radix tree.

[0089] Optionally, in the data processing device provided in the embodiment of the present application, the judgment unit includes: a calculation subunit, used to perform a sum calculation based on the first quantity and the second quantity to obtain a target quantity; a third acquisition subunit, used to obtain the current first node type of the leaf node in the target adaptive radix tree, and obtain a node quantity threshold corresponding to the first node type; a judgment subunit, used to judge whether to modify the node type corresponding to the leaf node in the target adaptive radix tree based on the target quantity and the node quantity threshold.

[0090] Optionally, in the data processing device provided in the embodiment of the present application, the device also includes: a modification unit, which is used to determine whether it is necessary to modify the node type corresponding to the leaf node in the target adaptive radix tree based on the first quantity and the second quantity. If it is necessary to modify the node type corresponding to the leaf node in the target adaptive radix tree, the leaf node in the target adaptive radix tree is modified to the second node type to obtain a modified target adaptive radix tree, wherein the node quantity threshold corresponding to the second node type is different from the node quantity threshold corresponding to the first node type; a second updating unit, which is used to update the modified target adaptive radix tree based on the second metadata information to obtain an updated target adaptive radix tree.

[0091] Optionally, in the data processing device provided in the embodiment of the present application, the device also includes: a third acquisition unit, used to obtain a deletion request after querying based on the target logical address information and the target adaptive radix tree to obtain the target metadata information, wherein the deletion request at least includes: the logical address range in the third metadata information to be deleted; a third determination unit, used to determine the leaf nodes to be pruned from the target adaptive radix tree based on the logical address range in the third metadata information to be deleted; and a pruning unit, used to prune the leaf nodes to be pruned to obtain the pruned target adaptive radix tree.

[0092] The data processing device includes a processor and a memory. The above-mentioned first acquisition unit 601, first determination unit 602 and query unit 603 are all stored in the memory as program units, and the processor executes the above-mentioned program units stored in the memory to realize corresponding functions.

[0093] The processor includes a kernel, which calls the corresponding program unit from the memory. One or more kernels can be set, and efficient query of metadata information can be achieved by adjusting kernel parameters.

[0094] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0095] An embodiment of the present invention provides a computer-readable storage medium on which a program is stored. When the program is executed by a processor, a data processing method is implemented.

[0096] An embodiment of the present invention provides a processor, which is used to run a program, wherein the program executes a data processing method when it is run.

[0097] like Figure 7As shown, an embodiment of the present invention provides an electronic device, the device includes a processor, a memory, and a program stored in the memory and executable on the processor, and the processor implements the following steps when executing the program: obtaining a query request, wherein the query request includes at least target logical address information in the target metadata information to be queried; determining a target adaptive radix tree from multiple adaptive radix trees based on the target logical address information, wherein the adaptive radix tree is composed of key-value pairs corresponding to multiple metadata information, and the key-value pairs are composed of a logical address in the metadata information and a physical address in the metadata information; querying based on the target logical address information and the target adaptive radix tree to obtain the target metadata information, wherein the data information stored in the physical address in the target metadata information is obtained through the target metadata information.

[0098] Optionally, before determining the target adaptive radix tree from a plurality of adaptive radix trees according to the target logical address information, the method further includes: determining a plurality of data volumes currently existing in the distributed block storage system; sharding each data volume to obtain a plurality of data shards; and constructing an adaptive radix tree corresponding to each data shard according to the first metadata information corresponding to each data shard.

[0099] Optionally, slicing each data volume to obtain multiple data slicings includes: obtaining the storage capacity and preset slicing rules corresponding to each data volume; slicing each data volume according to the storage capacity and the preset slicing rules to obtain multiple data slicings.

[0100] Optionally, based on the first metadata information corresponding to each data shard, constructing an adaptive radix tree corresponding to each data shard includes: for each data shard, obtaining multiple first metadata information corresponding to the data shard; based on the logical address in each first metadata information, constructing a node of the adaptive radix tree; based on the physical address in each first metadata information, determining a leaf of the adaptive radix tree; and based on the nodes of the adaptive radix tree and the leaves of the adaptive radix tree, constructing an adaptive radix tree corresponding to the data shard.

[0101] Optionally, determining the leaves of the adaptive radix tree based on the physical address in each first metadata information includes: obtaining a preset array; encoding and compressing the physical address based on the array subscript of the preset array to obtain the encoded physical address; and determining the leaves of the adaptive radix tree based on the encoded physical address.

[0102] Optionally, after querying based on the target logical address information and the target adaptive radix tree to obtain the target metadata information, the method also includes: obtaining a new request, wherein the new request includes at least second metadata information to be added to the target adaptive radix tree; obtaining a first quantity of the second metadata information and a second quantity of current leaf nodes of the target adaptive radix tree; judging whether it is necessary to modify the node type corresponding to the leaf nodes in the target adaptive radix tree based on the first quantity and the second quantity; if it is not necessary to modify the node type corresponding to the leaf nodes in the target adaptive radix tree, updating the target adaptive radix tree based on the second metadata information to obtain an updated target adaptive radix tree.

[0103] Optionally, judging whether it is necessary to modify the node type corresponding to the leaf node in the target adaptive radix tree based on the first quantity and the second quantity includes: performing a sum calculation based on the first quantity and the second quantity to obtain the target quantity; obtaining the current first node type of the leaf node in the target adaptive radix tree, and obtaining a node quantity threshold corresponding to the first node type; judging whether to modify the node type corresponding to the leaf node in the target adaptive radix tree based on the target quantity and the node quantity threshold.

[0104] Optionally, after determining whether it is necessary to modify the node type corresponding to the leaf nodes in the target adaptive radix tree based on the first quantity and the second quantity, the method further includes: if it is necessary to modify the node type corresponding to the leaf nodes in the target adaptive radix tree, modifying the leaf nodes in the target adaptive radix tree to the second node type to obtain a modified target adaptive radix tree, wherein a node quantity threshold corresponding to the second node type is different from a node quantity threshold corresponding to the first node type; and updating the modified target adaptive radix tree based on the second metadata information to obtain an updated target adaptive radix tree.

[0105] Optionally, after querying based on the target logical address information and the target adaptive radix tree to obtain the target metadata information, the method further includes: obtaining a deletion request, wherein the deletion request at least includes: the logical address range in the third metadata information to be deleted; determining the leaf nodes to be pruned from the target adaptive radix tree based on the logical address range in the third metadata information to be deleted; and pruning the leaf nodes to be pruned to obtain the pruned target adaptive radix tree.

[0106] The devices in this article can be servers, PCs, PADs, mobile phones, etc.

[0107] The present application also provides a computer program product, which, when executed on a data processing device, is suitable for executing a program that is initialized with the following method steps: obtaining a query request, wherein the query request at least includes target logical address information in the target metadata information to be queried; determining a target adaptive radix tree from multiple adaptive radix trees based on the target logical address information, wherein the adaptive radix tree is composed of key-value pairs corresponding to multiple metadata information, and the key-value pairs are composed of a logical address in the metadata information and a physical address in the metadata information; querying based on the target logical address information and the target adaptive radix tree to obtain target metadata information, wherein data information stored in the physical address in the target metadata information is obtained through the target metadata information.

[0108] Optionally, before determining the target adaptive radix tree from a plurality of adaptive radix trees according to the target logical address information, the method further includes: determining a plurality of data volumes currently existing in the distributed block storage system; sharding each data volume to obtain a plurality of data shards; and constructing an adaptive radix tree corresponding to each data shard according to the first metadata information corresponding to each data shard.

[0109] Optionally, slicing each data volume to obtain multiple data slicings includes: obtaining the storage capacity and preset slicing rules corresponding to each data volume; slicing each data volume according to the storage capacity and the preset slicing rules to obtain multiple data slicings.

[0110] Optionally, based on the first metadata information corresponding to each data shard, constructing an adaptive radix tree corresponding to each data shard includes: for each data shard, obtaining multiple first metadata information corresponding to the data shard; based on the logical address in each first metadata information, constructing a node of the adaptive radix tree; based on the physical address in each first metadata information, determining a leaf of the adaptive radix tree; and based on the nodes of the adaptive radix tree and the leaves of the adaptive radix tree, constructing an adaptive radix tree corresponding to the data shard.

[0111] Optionally, determining the leaves of the adaptive radix tree based on the physical address in each first metadata information includes: obtaining a preset array; encoding and compressing the physical address based on the array subscript of the preset array to obtain the encoded physical address; and determining the leaves of the adaptive radix tree based on the encoded physical address.

[0112] Optionally, after querying based on the target logical address information and the target adaptive radix tree to obtain the target metadata information, the method also includes: obtaining a new request, wherein the new request includes at least second metadata information to be added to the target adaptive radix tree; obtaining a first quantity of the second metadata information and a second quantity of current leaf nodes of the target adaptive radix tree; judging whether it is necessary to modify the node type corresponding to the leaf nodes in the target adaptive radix tree based on the first quantity and the second quantity; if it is not necessary to modify the node type corresponding to the leaf nodes in the target adaptive radix tree, updating the target adaptive radix tree based on the second metadata information to obtain an updated target adaptive radix tree.

[0113] Optionally, judging whether it is necessary to modify the node type corresponding to the leaf node in the target adaptive radix tree based on the first quantity and the second quantity includes: performing a sum calculation based on the first quantity and the second quantity to obtain the target quantity; obtaining the current first node type of the leaf node in the target adaptive radix tree, and obtaining a node quantity threshold corresponding to the first node type; judging whether to modify the node type corresponding to the leaf node in the target adaptive radix tree based on the target quantity and the node quantity threshold.

[0114] Optionally, before determining whether it is necessary to modify the node type corresponding to the leaf nodes in the target adaptive radix tree based on the first quantity and the second quantity, the method also includes: if it is necessary to modify the node type corresponding to the leaf nodes in the target adaptive radix tree, modifying the leaf nodes in the target adaptive radix tree to the second node type to obtain a modified target adaptive radix tree, wherein a node quantity threshold corresponding to the second node type is different from a node quantity threshold corresponding to the first node type; and updating the modified target adaptive radix tree based on the second metadata information to obtain an updated target adaptive radix tree.

[0115] Optionally, after querying based on the target logical address information and the target adaptive radix tree to obtain the target metadata information, the method further includes: obtaining a deletion request, wherein the deletion request at least includes: the logical address range in the third metadata information to be deleted; determining the leaf nodes to be pruned from the target adaptive radix tree based on the logical address range in the third metadata information to be deleted; and pruning the leaf nodes to be pruned to obtain the pruned target adaptive radix tree.

[0116] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0117] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0118] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0119] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0120] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0121] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0122] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0123] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0124] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0125] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.

Claims

1. A data processing method, characterized in that: include: Obtaining a query request, wherein the query request at least includes target logical address information in the target metadata information to be queried; Determining a target adaptive radix tree from a plurality of adaptive radix trees according to the target logical address information, wherein the adaptive radix tree is composed of key-value pairs corresponding to a plurality of metadata information, and the key-value pairs are composed of a logical address in the metadata information and a physical address in the metadata information; The target metadata information is obtained by querying according to the target logical address information and the target adaptive radix tree, wherein the data information stored in the physical address in the target metadata information is obtained through the target metadata information.

2. The method according to claim 1, characterized in that Before determining a target adaptive radix tree from a plurality of adaptive radix trees according to the target logical address information, the method further includes: Determine multiple data volumes currently existing in the distributed block storage system; Each data volume is sliced ​​to obtain multiple data slices; According to the first metadata information corresponding to each data shard, an adaptive radix tree corresponding to each data shard is constructed.

3. The method according to claim 2, characterized in that Each data volume is sliced ​​to obtain multiple data slices including: Get the storage capacity and preset sharding rules corresponding to each data volume; Each data volume is sliced ​​according to the storage capacity and the preset slicing rule to obtain the multiple data slices.

4. The method according to claim 2, characterized in that: According to the first metadata information corresponding to each data shard, constructing an adaptive radix tree corresponding to each data shard includes: For each data shard, obtain multiple first metadata information corresponding to the data shard; Constructing a node of the adaptive radix tree according to the logical address in each first metadata information; Determining leaves of the adaptive radix tree according to the physical addresses in each first metadata information; An adaptive radix tree corresponding to the data fragment is constructed according to the nodes of the adaptive radix tree and the leaves of the adaptive radix tree.

5. The method according to claim 4, characterized in that Determining the leaves of the adaptive radix tree according to the physical addresses in each first metadata information includes: Get the preset array; Encoding and compressing the physical address according to the array subscript of the preset array to obtain an encoded physical address; The leaves of the adaptive radix tree are determined according to the encoded physical address.

6. The method according to claim 1, characterized in that After querying according to the target logical address information and the target adaptive radix tree to obtain the target metadata information, the method further includes: Obtaining a new addition request, wherein the new addition request at least includes second metadata information to be added to the target adaptive radix tree; Acquire a first quantity of the second metadata information and a second quantity of current leaf nodes of the target adaptive radix tree; Determining whether it is necessary to modify the node type corresponding to the leaf node in the target adaptive radix tree according to the first number and the second number; If there is no need to modify the node type corresponding to the leaf node in the target adaptive radix tree, the target adaptive radix tree is updated according to the second metadata information to obtain an updated target adaptive radix tree.

7. The method according to claim 6, characterized in that Judging whether it is necessary to modify the node type corresponding to the leaf node in the target adaptive radix tree according to the first number and the second number includes: Performing a sum calculation based on the first quantity and the second quantity to obtain a target quantity; Obtaining a current first node type of a leaf node in the target adaptive radix tree, and obtaining a node quantity threshold corresponding to the first node type; According to the target number and the node number threshold, it is determined whether to modify the node type corresponding to the leaf node in the target adaptive radix tree.

8. The method according to claim 6, characterized in that After determining whether it is necessary to modify the node type corresponding to the leaf node in the target adaptive radix tree according to the first number and the second number, the method further includes: If it is necessary to modify the node type corresponding to the leaf node in the target adaptive radix tree, modify the leaf node in the target adaptive radix tree to a second node type to obtain a modified target adaptive radix tree, wherein the node quantity threshold corresponding to the second node type is different from the node quantity threshold corresponding to the first node type; The modified target adaptive radix tree is updated according to the second metadata information to obtain an updated target adaptive radix tree.

9. The method according to claim 1, characterized in that: After querying according to the target logical address information and the target adaptive radix tree to obtain the target metadata information, the method further includes: Obtaining a deletion request, wherein the deletion request at least includes: a logical address range in the third metadata information to be deleted; Determining a leaf node to be pruned from the target adaptive radix tree according to a logical address range in the third metadata information to be deleted; The leaf nodes to be pruned are pruned to obtain a pruned target adaptive radix tree.

10. A data processing device, characterized in that: include: A first acquisition unit is used to acquire a query request, wherein the query request at least includes target logical address information in target metadata information to be queried; A first determining unit is used to determine a target adaptive radix tree from a plurality of adaptive radix trees according to the target logical address information, wherein the adaptive radix tree is composed of key-value pairs corresponding to a plurality of metadata information, and the key-value pairs are composed of a logical address in the metadata information and a physical address in the metadata information; A query unit is used to query according to the target logical address information and the target adaptive radix tree to obtain the target metadata information, wherein the data information stored in the physical address in the target metadata information is obtained through the target metadata information.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the storage medium is located is controlled to execute the data processing method according to any one of claims 1 to 9.

12. An electronic device, characterized in that: include: A memory storing an executable program; A processor, used to run the program, wherein the program executes the data processing method according to any one of claims 1 to 9 when running.

Images