Request processing method and apparatus, storage device, and storage medium

Abstract

The application relates to the technical field of storage, and provides a request processing method and device, a storage device and a storage medium, the method comprises the following steps: acquiring a base number tree, the base number tree comprises a plurality of leaf nodes, each leaf node comprises a preset number of index values arranged in sequence; if a target leaf node exists in the base number tree, then performing to-be-written data merging according to effective index values in the target leaf node to generate a writing request, the target leaf node is a leaf node with adjacent effective index values, and the effective index values are index values corresponding to to-be-written data; and the writing request is sent to a hard disk, so that the hard disk writes the merged to-be-written data into the hard disk according to the writing request. The application reduces the IO interaction times of the storage device and the hard disk, thereby improving the IO request processing capacity of the storage device.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of storage, in particular to a request processing method and device, a storage device and a storage medium. BACKGROUND

[0002] In the technical field of storage, the IO performance of a hard disk has always been a bottleneck restricting system performance. For example, after receiving an IO request, a storage device sends the IO request to a hard disk to store data in the hard disk or read data from the hard disk. The hard disk responds to each IO request issued by the storage device. However, due to the limited data read / write performance of the hard disk, the read / write processing capability of the entire storage device is greatly affected. SUMMARY

[0003] The present application provides a request processing method and device, a storage device and a storage medium, which can use a radix tree to first merge to-be-written data and then write the merged data together, thereby reducing the number of IO interactions between the storage device and the hard disk and improving the IO request processing capability of the storage device.

[0004] Embodiments of the present application can be implemented as follows:

[0005] In a first aspect, the present application provides a request processing method applied to a storage device, wherein the storage device includes a hard disk, and the method includes the following steps:

[0006] obtaining a radix tree, wherein the radix tree includes a plurality of leaf nodes, each leaf node includes a preset number of index values arranged in sequence, and the index values correspond to to-be-written data;

[0007] if a target leaf node exists in the radix tree, merging to-be-written data according to effective index values in the target leaf node to generate a writing request, wherein the target leaf node is a leaf node having adjacent effective index values, and the effective index values are index values corresponding to to-be-written data;

[0008] issuing the writing request to the hard disk to enable the hard disk to write the merged to-be-written data into the hard disk according to the writing request.

[0009] In an optional embodiment, the step of merging to-be-written data according to effective index values in the target leaf node to generate a writing request includes the following steps:

[0010] obtaining the minimum value and the maximum value of the continuous effective index values in the target leaf node;

[0011] taking the starting position of the to-be-written data corresponding to the minimum value as a writing position;

[0012] a sum of lengths of to-be-written data corresponding to effective index values between the minimum value and the maximum value as a to-be-written length;

[0013] generating the writing request according to the to-be-written length and the to-be-written key.

[0014] In an optional implementation, the storage device is in communication connection with a client, and the method further comprises:

[0015] receiving a writing request sent by the client, the writing request comprising a to-be-written block identifier, a to-be-written block internal offset, and to-be-written data;

[0016] determining a to-be-written key according to the to-be-written block identifier and the to-be-written block internal offset;

[0017] updating the to-be-written data of the writing request to a leaf node in the radix tree according to the to-be-written key.

[0018] In an optional implementation, the radix tree comprises nodes of a preset level, a node at a minimum level is a root node, a node at a maximum level is a leaf node, and nodes at remaining levels are intermediate nodes, each of the nodes comprises a preset number of index values arranged in sequence, the to-be-written key is divided into bit segments in ascending order according to bit positions from high to low, a number of the bit segments is the preset level, a bit segment with a smallest serial number is a current bit segment, and the root node is a parent node.

[0019] The step of updating the to-be-written data of the writing request to the leaf node in the radix tree according to the to-be-written key comprises:

[0020] determining a current index value corresponding to the current bit segment from the preset number of index values of the parent node;

[0021] if the parent node is not an intermediate node at the maximum level, determining a current intermediate node corresponding to the current index value from an intermediate node at a next level associated with the parent node;

[0022] taking the current intermediate node as the parent node and taking a next bit segment adjacent to the current bit segment in serial number as the current bit segment, and repeating the above steps until the parent node is an intermediate node at the maximum level;

[0023] updating the to-be-written data of the writing request to a leaf node corresponding to the current index value.

[0024] In an optional implementation, the step of updating the to-be-written data of the writing request to the leaf node corresponding to the current index value comprises:

[0025] if the current data exists in the leaf node corresponding to the current index value, obtaining a version number of the current data;

[0026] if the version number of the to-be-written data of the write request is greater than the version number of the current data, covering the current data with the to-be-written data of the write request.

[0027] In an optional implementation, the method further comprises:

[0028] receiving a read request sent by the client, the read request comprising a to-be-read block identifier and a to-be-read block internal offset;

[0029] judging, according to the to-be-read block identifier and the to-be-read block internal offset, whether to-be-read data desired to be read by the read request exists in the radix tree;

[0030] if the to-be-read data exists in the radix tree, reading the to-be-read data from the radix tree;

[0031] if the to-be-read data does not exist in the radix tree, reading the to-be-read data from the hard disk.

[0032] In an optional implementation, the step of judging, according to the to-be-read block identifier and the to-be-read block internal offset, whether to-be-read data desired to be read by the read request exists in the radix tree comprises:

[0033] determining a to-be-read key according to the to-be-read block identifier and the to-be-read block internal offset;

[0034] if a leaf node corresponding to the to-be-read key exists in the radix tree, determining that the to-be-read data exists in the radix tree;

[0035] otherwise, determining that the to-be-read data does not exist in the radix tree.

[0036] In a second aspect, the application provides a request processing device applied to a storage device, the storage device comprising a hard disk, and the device comprising:

[0037] an obtaining module, configured to obtain a radix tree, the radix tree comprising a plurality of leaf nodes, each leaf node comprising a preset number of index values arranged in sequence;

[0038] a processing module, configured to, if a target leaf node exists in the radix tree, perform to-be-written disk data merging according to effective index values in the target leaf node to generate a to-be-written disk request, the target leaf node being a leaf node having adjacent effective index values, and the effective index value being an index value corresponding to to-be-written disk data.

[0039] The sending module is configured to send the disk-down request to the hard disk, so that the hard disk writes the merged to-be-downloaded data into the hard disk according to the disk-down request.

[0040] In a third aspect, the present application provides a storage device, comprising a processor and a memory, wherein the memory is configured to store a program, and the processor is configured to implement the request processing method according to any one of the preceding embodiments when executing the program.

[0041] In a fourth aspect, the present application provides a computer readable storage medium, which stores a computer program, and the computer program is configured to implement the request processing method according to any one of the preceding embodiments when executed by a processor.

[0042] Compared with the prior art, the embodiment first acquires a radix tree, the radix tree comprises a plurality of leaf nodes, each leaf node comprises a preset number of index values sorted in order, if a target leaf node exists in the radix tree, the to-be-downloaded data is merged according to the effective index values in the target leaf node, a disk-down request is generated, the target leaf node is a leaf node having adjacent effective index values, and the effective index value is an index value corresponding to the to-be-downloaded data, and finally the disk-down request is sent to the hard disk, so that the hard disk writes the merged to-be-downloaded data into the hard disk according to the disk-down request. Since the to-be-downloaded data is merged by using the radix tree in the embodiment, and then the merged data is downloaded together, on the one hand, the radix tree can improve the merging efficiency, and on the other hand, the data is downloaded after being merged, thereby reducing the number of IO interactions between the storage device and the hard disk, and improving the IO request processing capability of the storage device. BRIEF DESCRIPTION OF DRAWINGS

[0043] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments. It should be understood that the following drawings only show some embodiments of the present application, and therefore should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can also be obtained without creative labor on the basis of these drawings.

[0044] Figure 1 An example diagram of the system architecture provided for the embodiment.

[0045] Figure 2 A block diagram of the storage device provided for the embodiment.

[0046] Figure 3 An example diagram of the hard disk space division provided for the embodiment.

[0047] Figure 4 An example diagram of the structure of the radix tree provided for the embodiment.

[0048] Figure 5A flowchart example of the request processing method provided for the embodiment Figure 1 .

[0049] Figure 6 A process example of data insertion into a radix tree provided for the embodiment Figure 1 .

[0050] Figure 7 A process example of data insertion into a radix tree provided for the embodiment Figure 2 .

[0051] Figure 8 A flowchart example of the request processing method provided for the embodiment Figure 2 .

[0052] Figure 9 An example diagram of the data merging process based on a radix tree provided for the embodiment.

[0053] Figure 10 A flowchart example of the request processing method provided for the embodiment Figure 3 .

[0054] Figure 11 An example diagram of the data query process based on a radix tree provided for the embodiment.

[0055] Figure 12 A block schematic diagram of the request processing apparatus provided for the embodiment of the application.

[0056] Icon: 10 - storage device; 11 - processor; 12 - memory; 13 - bus; 14 - communication interface; 20 - client; 100 - data request processing apparatus; 110 - acquisition module; 120 - processing module; 130 - sending module. DETAILED DESCRIPTION

[0057] In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some but not all of the embodiments of the present application. The components of the embodiments of the present application described and shown in the drawings herein can be arranged and designed in various different configurations.

[0058] Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the claimed application, but only represents selected embodiments of the application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present application without creative labor are within the scope of protection of the present application.

[0059] It should be noted that like reference numerals and characters refer to like elements throughout the following description with like reference numerals and characters referring to like elements throughout the following description and across different drawings indicated to be reference numerals and characters in the drawings and like suffixes refer to like elements. It is to be expressly understood that, wherein an element is described as having a feature, such a feature is not required for the element.

[0060] In the description of the present application, it should be noted that the terms "upper", "lower", "inner", "outer" and the like indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, or the orientation or positional relationship in which the product of the application is usually placed, which is only for the convenience of describing the application and simplifying the description, and does not indicate or imply that the device or element referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as a limitation on the application.

[0061] In addition, the terms "first", "second" and the like appear only to distinguish the description and cannot be understood as indicating or implying relative importance.

[0062] It should be noted that the features in the embodiments of the present application can be combined with each other without conflict.

[0063] Please refer to Figure 1 , Figure 1 The example diagram of the system architecture provided by the present embodiment is as follows, Figure 1 In the example diagram, the client 20 and the storage device 10 are in communication connection, the client 20 is an access visitor of the storage device 10, and the client 20 needs to store the to-be-written data to the storage device 10 or read the stored data from the storage device 10. The storage device 10 includes a PMEM (Persistent Memory) as a cache and n hard disks, each hard disk corresponds to a radix tree, and the radix tree is used to manage the data to be stored to the corresponding hard disk and currently temporarily stored in the PMEM.

[0064] When the client 20 writes the to-be-written data to the storage device 10, the storage device 10 writes the to-be-written data to the PMEM and returns the writing success to the client 20 to ensure timely response to the client 20. Each hard disk corresponds to a merging thread, and the merging thread inserts the data to be stored in the corresponding hard disk and currently temporarily stored in the PMEM into the radix tree corresponding to the hard disk, and then performs the merging before the IO to the corresponding hard disk based on the corresponding radix tree. In this way, the number of IOs to the hard disk can be reduced, and the write IO request processing capability of the storage device can be improved.

[0065] When the client 20 reads the to-be-read data from the storage device 10, the client 20 issues a read request to the storage device 10, and the storage device 10 judges whether the to-be-read data is in the corresponding radix tree based on the received read request. If yes, the to-be-read data is directly read from the radix tree, otherwise the to-be-read data is read from the hard disk. Since the PMEM has the function of caching data, for the data cached in the PMEM, the to-be-read data can be quickly found based on the radix tree, so that a relatively ideal read IO request processing capability can be obtained.

[0066] The storage device 10 can be a storage server, or a storage server cluster or a storage array composed of multiple storage servers.

[0067] The client 20 can be a host, a host cluster, a server, a server cluster, a tablet computer, a notebook computer, a mobile phone, etc.

[0068] Based on the example diagram of the system architecture, the embodiment further provides a block schematic diagram of the storage device 10 in the system architecture. Figure 1 , Figure 1 , Figure 2 , Figure 2 The block schematic diagram of the storage device 10 provided by the embodiment, the storage device 10 can execute the above request processing method. The storage device 10 includes a processor 11, a memory 12, a bus 13 and a communication interface 14. The processor 11 and the memory 12 are connected through the bus 13.

[0069] The processor 11 can be an integrated circuit chip with signal processing capability. In the implementation process, each step of the above method can be completed by the integrated logic circuit of hardware or the instruction in the form of software in the processor 11. The above processor 11 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; also can be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a ready programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component.

[0070] The memory 12 is used for storing programs, for example, the request processing device in the embodiment of the application. The request processing device includes at least one software function module stored in the memory 12 in the form of software or firmware. After receiving an execution instruction, the processor 11 executes the program to implement the request processing method in the embodiment of the application.

[0071] The memory 12 can include a high-speed random access memory (RAM) and can also include a non-volatile memory. Optionally, the memory 12 can be a storage device built into the processor 11 or a storage device independent of the processor 11.

[0072] The bus 13 can be an ISA bus, a PCI bus, an EISA bus, or the like. Figure 2 Only one bidirectional arrow is shown, but this does not mean that there is only one bus or only one type of bus.

[0073] The storage device 10 communicates with the client 20 through the communication interface 14.

[0074] To facilitate understanding, before describing the technical solution of the request processing provided by the embodiment of the present application, the partitioning manner of the hard disk in the embodiment will be introduced first. Please refer to Figure 3 , Figure 3 The example diagram of the partitioning of the hard disk space provided by the embodiment is shown in Figure 3 , a hard disk is divided into a plurality of blocks (also referred to as Block), each block includes a block metadata area (also referred to as Blockmate area) and a data area, the Block Mate area stores the version number of the data in the data area, and the write order of the data can be distinguished according to the version number, for example, the smaller the version number, the earlier the write time, and vice versa. The size of the data corresponding to each version number can be set in advance, for example, set to 4 KB, that is, the data length of each data request is 4 KB. Taking the size of the data area as 64 MB as an example, the data area of 64 MB can store up to 16K pieces of 4 KB data, and if each version number is represented by 8 bytes, then the size of the block metadata area in each block is 128 KB, and the number of blocks on each hard disk is: hard disk capacity / (64 MB+128 KB).

[0075] It should be noted that the above is only one implementation manner of the partitioning of the hard disk, and in fact, the size of the data corresponding to each version number can also be set according to actual needs, for example, set to 8 KB, and the size of the data area can also be set, for example, set to 138 MB, and the implementation manner is similar to that of the above, which will not be described herein again. Figure 3

[0076] Based on the partitioning manner of the hard disk, Figure 3 the embodiment also provides an example diagram of the structure of a radix tree, which is also referred to as a radix tree. Please refer to Figure 4 , Figure 4 The example diagram of the structure of the radix tree provided by the embodiment is shown in Figure 4 ​In this radix tree, there are 6 levels. Level 1 contains a root node, and levels 2 through 5 each contain 64 intermediate nodes. 2 ~64 5 The diagram only shows two intermediate nodes for each layer as an example. The 6th layer consists of leaf nodes. The root node, intermediate nodes, and leaf nodes all have the same structure, including 64 index values ​​from 0 to 63. The difference in the leaf nodes is that each index value corresponds to a 4KB data unit for the leaf node. Figure 4 A radix tree in can represent 64 6 *4KB = 256TB, meaning hard drives with a capacity less than 256TB can be used. Figure 4 The radix tree representation is used. For any 4KB data on the hard drive, a key is calculated based on its storage location. Starting from the root node, the query is performed level by level based on the key. If a corresponding leaf node exists, the data corresponding to the leaf node is the data to be queried; otherwise, it means that the data is not in the radix tree.

[0077] To clearly illustrate how any data on the hard drive is managed using a radix tree, this embodiment also provides an application... Figure 1 and Figure 2 The request handling method for storage device 10, which is used to handle write requests, is described in the following reference. Figure 5 , Figure 5 A flowchart example of the request processing method provided in this embodiment. Figure 1 The method includes the following steps:

[0078] Step S101: Receive a write request sent by the client. The write request includes the identifier of the block to be written, the offset within the block to be written, and the data to be written.

[0079] In this embodiment, the hard disk is divided into multiple blocks, each of which has a block identifier. The block identifier to be written in the write request is the identifier of the block on the hard disk where the data to be written should be written. The offset within the block to be written is the offset address within the block represented by the block identifier to be written. For example, if the block identifier to be written is 10 and the offset address within the block is 128KB, it means that the starting address for writing the data to be written is the address of the 128KB block on the hard disk.

[0080] Step S102: Determine the key to be written based on the identifier of the block to be written and the offset within the block to be written.

[0081] In this embodiment, with Figure 3 Taking the disk space partitioning method as an example, the formula for determining the key to be written can be: Key to be written = (Identifier of block to be written << 14) | (Offset within block to be written / 4K).

[0082] S103, updating the to-be-written data of the write request to the leaf node in the radix tree according to the to-be-written keyword.

[0083] It should be noted that, in order to reduce the space occupied by the radix tree, the to-be-written data can not be inserted into the leaf node, and the to-be-written data can be written into the PMEM, and the address of the to-be-written data in the PMEM is stored in the leaf node. Specifically, a pointer is added in the leaf node, which points to the address of the to-be-written data in the PMEM.

[0084] The above method provided by the embodiment organizes the to-be-written data by using the radix tree, so that adjacent index values in the same leaf node in the radix tree correspond to continuous data on the hard disk, and then the to-be-written data can be efficiently merged when being written to the disk, the number of disk IO is reduced, the disk efficiency is improved, and the processing capacity of the storage device 10 is improved.

[0085] In the embodiment, the to-be-written keyword can be split into several parts which are the same as the preset levels of the radix tree, and the index value in the node of the corresponding level is determined by each part. One implementation manner is as follows:

[0086] The to-be-written keyword is divided into bit segments in ascending order from small to large according to the bit from high to low, and the number of bit segments is the preset level. The bit segment with the smallest serial number is taken as the current bit segment, and the root node is taken as the parent node. For example, the preset level is 6 layers, and the to-be-written keyword is 36 bits. The 36 bits are divided into 6 bit segments with 6 bits as one bit segment. For example, the binary representation of the to-be-written keyword 16385 is: 000000 000000 000000 000100 000000 000001, which includes 6 bit segments, and each bit segment includes 6 bits.

[0087] The implementation manner of updating the to-be-written data of the write request to the leaf node in the radix tree according to the to-be-written keyword can be as follows:

[0088] Firstly, the current index value corresponding to the current bit segment is determined from the preset number of index values of the parent node.

[0089] In the embodiment, as one implementation manner, the index value which is the same as the decimal number corresponding to the current bit segment in the 64 index values in the parent node can be taken as the current index value. For example, the current bit segment is: 000100, and the corresponding decimal number is 4, so the current index value is 4.

[0090] Secondly, if the parent node is not the middle node of the maximum level, the current middle node corresponding to the current index value is determined from the middle node of the next level associated with the parent node.

[0091] Thirdly, the current intermediate node is taken as the parent node, and the next bit segment adjacent to the serial number of the current bit segment is taken as the current bit segment, and the above steps are repeated until the parent node is the intermediate node of the maximum level.

[0092] Fourthly, the to-be-written data of the write request is updated to the leaf node corresponding to the current index value.

[0093] In the present embodiment, in order to more specifically illustrate how the data is inserted into the radix tree, please refer to the following description. Figure 6 , Figure 6 The following is a process example diagram provided for the present embodiment for inserting data into the radix tree, Figure 6 In the present embodiment, the to-be-written block identifier and the to-be-written block internal offset in the write request are 1 and 4 KB respectively, and the to-be-written keyword calculated according to the formula to-be-written keyword = (to-be-written block identifier << 14) | (to-be-written block internal offset / 4K) is: ((1 << 14) | (4K / 4K)) = 16385, and its binary representation is: 000000 000000 000000000100000000 000001, from left to right, it is the 1st-6th bit segment, the 1st bit segment is taken as the current bit segment, and the node of the 1st layer is taken as the parent node, the decimal value of the 1st bit segment is 0, and the index value 0 of the parent node is the current index value, since the parent node is not the intermediate node of the maximum level, it is determined that the intermediate node 20 of the 2nd layer corresponding to the index value 0 of the parent node is the current intermediate node, the intermediate node 20 is taken as the parent node, and the 2nd bit segment is taken as the current bit segment, and its decimal value is 0, it is determined that the index value 0 of the intermediate node 20 is the current index value, since the intermediate node 20 is not the intermediate node of the maximum level, the intermediate node 30 of the 3rd layer corresponding to the intermediate node 20 is taken as the current intermediate node in the above manner, until the current intermediate node is the leaf node 61 of the 6th layer, and the current index value is 1, the to-be-written data in the write request is written into the leaf node 61. As an implementation manner, the leaf node can further include a pointer pointing to the address of the to-be-written data in the PMEM, so that the corresponding to-be-written data can be directly found according to the pointer. As another implementation manner, the leaf node can also directly save the offset position of the to-be-written address in the PMEM, so as to directly find the corresponding to-be-written data according to the pre-stored first address and offset position of the PMEM.

[0094] It should be further noted that the radix tree can be dynamically generated, that is, when the intermediate node of the next layer corresponding to the intermediate node of the current layer does not exist, the intermediate node can be generated first, and then initialized, so that the newly generated intermediate node includes a preset number of index values arranged in sequence, so as to determine the current index value from the index values of the newly generated intermediate node.

[0095] In the embodiment, in the scenario that the order of data cannot be guaranteed, for example, the scenario of network communication, in order to correctly store the latest data, the version number is also used in the embodiment. The version number can be an accumulated counter. The version number of the data to be written is greater than the version number of the data written first. When the data to be written is inserted into the radix tree, if the data already exists in the radix tree, in order to correctly insert the data to be written into the radix tree, the embodiment also provides an implementation manner of updating the data to be written in the write request to the leaf node corresponding to the current index value:

[0096] If the current data exists in the leaf node corresponding to the current index value, the version number of the current data is obtained.

[0097] If the version number of the data to be written in the write request is greater than the version number of the current data, the data to be written in the write request is used to overwrite the current data.

[0098] In the embodiment, the version number of the current data can be obtained from the block metadata area of the block according to the block identifier of the current data in the hard disk. The version number of the data to be written in the write request can be directly obtained from the write request. The embodiment provides an example diagram of data inserted into the radix tree and needing to overwrite the current data. Please refer to Figure 7 , Figure 7 The process example of data inserted into the radix tree provided in the embodiment Figure 2 After the storage device receives the write request, the key is determined according to the foregoing manner, and the key is searched in the radix tree layer by layer to finally determine that the data to be written should be inserted into the 4K data area corresponding to the index value 2 of the 6th layer. However, the 4K data area corresponding to the index value 2 of the 6th layer already stores the current data, the version number of the current data is 1, and the version number of the data to be written in the write request is 2. Therefore, the data to be written in the write request is the latest data, and the data to be written in the write request is used to overwrite the current data.

[0099] In the embodiment, the data inserted into the radix tree according to the foregoing manner, if the index values in the same leaf node are adjacent, and the adjacent index values both correspond to the data to be written, the data to be written corresponding to the adjacent index values are also adjacent on the hard disk. At this time, the data to be written can be merged first and then written, thereby reducing the number of IOs of writing. Therefore, the embodiment also provides an implementation manner of processing the write request. Please refer to Figure 8 , Figure 8 The flow example of the request processing method provided in the embodiment Figure 2 The method comprises the following steps:

[0100] In step S201, a radix tree is obtained. The radix tree comprises a plurality of leaf nodes. Each leaf node comprises a preset number of index values arranged in sequence.

[0101] In step S202, if the target leaf node exists in the radix tree, the to-be-written data is merged according to the valid index values in the target leaf node, and a writing request is generated, the target leaf node is a leaf node in which adjacent valid index values exist, and the valid index value is an index value corresponding to the to-be-written data.

[0102] In this embodiment, only the to-be-written data in the same leaf node, the index value of which is a valid index value, and the valid index values are adjacent, can be merged, otherwise, the to-be-written data cannot be merged. The adjacent valid index values can be two or more than two. When the valid index values are more than two, the valid index values are adjacent in turn, for example, the valid index values are 2, 3 and 4, wherein 2 and 3 are adjacent, and 3 and 4 are adjacent.

[0103] In step S203, the writing request is sent to the hard disk, so that the hard disk writes the merged to-be-written data into the hard disk according to the writing request.

[0104] In this embodiment, the storage device 10 can include one hard disk or multiple hard disks. If the storage device 10 includes one hard disk, the processing of steps S101-S103 is the processing flow of the hard disk in the storage device 10. If the storage device 10 includes multiple hard disks, the processing of steps S201-S203 is performed for the data to be written into each hard disk.

[0105] The above method provided in this embodiment first merges the to-be-written data by using the radix tree, and then writes the merged data together, thereby reducing the number of IO interactions between the storage device and the hard disk, and improving the IO request processing capability of the storage device.

[0106] In order to improve the merging efficiency and the writing efficiency, and ultimately improve the IO request processing capability of the storage device, this embodiment further provides an implementation manner of merging to-be-written data according to the valid index values in the target leaf node, and generating a writing request:

[0107] Obtain the minimum value and the maximum value in the continuous valid index values in the target leaf node.

[0108] Take the start position of the to-be-written data corresponding to the minimum value as the writing position.

[0109] Take the sum of the lengths of the to-be-written data corresponding to the valid index values between the minimum value and the maximum value as the to-be-written length.

[0110] Generate the writing request according to the writing position and the to-be-written length.

[0111] In the embodiment, the continuous valid index values in the target leaf node can be two or more, wherein the start position of the to-be-written data corresponding to the minimum value of the valid index values is the writing position, and the to-be-written length is the sum of the lengths of the to-be-written data corresponding to the continuous valid index values. For example, the continuous valid index values are 1, 2 and 3, the start position of the to-be-written data corresponding to the index value 1 is address 1, and the length of the to-be-written data corresponding to each index value is 4 KB. The sum of the lengths of 1, 2 and 3 is 6 KB. The writing request is to write 6 KB of to-be-written data starting from address 1.

[0112] To more clearly illustrate the merging process, refer to Figure 9 , Figure 9 An example diagram of the data merging process based on the radix tree provided in the embodiment is shown in Figure 9 , the data to be written by the request 1 (1, 4 KB, 4 KB) is inserted into the position with the index value 1 in the 6th layer of the radix tree, and the data to be written by the request 2 (1, 8 KB, 4 KB) is inserted into the position with the index value 2 in the 6th layer of the radix tree. Since the request 1 and the request 2 belong to the same leaf node and the index values are adjacent, the request 1 and the request 2 can be merged into one writing request (1, 4 KB, 8 KB). The block identifier of the writing request is the same as that of the request 1 and the request 2. The start address of the writing request is the start address of the request 1. The writing length of the writing request is the sum of the lengths of the request 1 and the request 2, which is 8 KB.

[0113] As an implementation manner: all the data in the radix tree is traversed. The data continuously existing in the leaf nodes of the radix tree is saved into an array. Finally, when there is no data in the current leaf node of the radix tree or the current block identifier is not equal to the previous block identifier, a writing logic is triggered. The continuous data is written into the disk as one IO, thereby realizing the function of IO merging.

[0114] It should be noted that the longer the data exists in the radix tree, the greater the probability of data merging, and the greater the consumed PMEM space. In order to balance the two, a cache threshold can be set. When the cache threshold reaches n% of the total cache (n can be flexibly configured according to the system environment), the above writing process is triggered.

[0115] It should be further noted that when too much data is merged, the instantaneous IO occupancy rate can be too high to affect other processes executed on the storage device. In order to avoid the influence of too much merged data on other processes, a configuration item can be configured to flush the disk regularly according to the business requirements. The specific disk flushing manner can be: all the data in the radix tree is written into the hard disk. After the writing is successful, the space in the PMEM occupied by the data is released, and the indexes in the radix tree are cleared.

[0116] It should be noted that for IO requests that do not meet the merging condition, a certain degree of IO acceleration can also be achieved by using the radix tree, for example, there are two IO requests a and b in sequence, a is initiated first, and the offset A needs to be written, b is initiated later, and the offset B needs to be written, when offset A>offset B, taking a hard disk using magnetic recording technology to store data as an example, according to the characteristics of the disk, when the IO request a is completed, the head needs to rotate for one more revolution before the IO request b can be executed (the head rotates for 2 revolutions). The radix tree has the function of sorting the offset of the IO, so the IO request b is executed first, and the IO request a is executed later (the head only rotates for 1 revolution), thereby reducing the distance of the disk rotation, thereby improving the IO down-disk performance.

[0117] In the present embodiment, since the data in the radix tree is located in the PMEM, and the access capability of the PMEM is generally greater than that of the hard disk, when reading data, if the data to be read is in the PMEM, it can be quickly queried in the radix tree, thereby improving the read request processing capability. The present embodiment also provides a processing method for a read request, please refer to Figure 10 , Figure 10 A flow example of the request processing method provided by the present embodiment Figure 3 The method comprises the following steps:

[0118] Step S301, receiving a read request sent by a client, the read request comprising a to-be-read block identifier and a to-be-read block offset.

[0119] In the present embodiment, the to-be-read block identifier and the to-be-read block offset are similar to the to-be-written block identifier and the to-be-written block offset, which will not be described here.

[0120] Step S302, judging whether the to-be-read data to be read by the read request exists in the radix tree according to the to-be-read block identifier and the to-be-read block offset.

[0121] In the present embodiment, whether the to-be-read data exists in the radix tree can be determined by querying the radix tree, and one implementation manner can be:

[0122] Determining a to-be-read key according to the to-be-read block identifier and the to-be-read block offset.

[0123] If the radix tree exists a leaf node corresponding to the to-be-read key, it is determined that the to-be-read data exists in the radix tree.

[0124] Otherwise, it is determined that the to-be-read data does not exist in the radix tree.

[0125] In the present embodiment, the determination manner of the to-be-read key is the same as that of the to-be-written key, which will not be described here.

[0126] Step S303, if the to-be-read data exists in the radix tree, reading the to-be-read data from the radix tree.

[0127] Step S304, if the to-be-read data does not exist in the radix tree, reading the to-be-read data from the hard disk.

[0128] In order to more clearly illustrate the process of reading the to-be-read data, please refer to Figure 11 , Figure 11 the example diagram of the data query process based on the radix tree provided by the present embodiment, Figure 11 In the example diagram, the values of the decimal numbers corresponding to the 6-bit segments of the to-be-read keyword are: 2, 62, 0, 3, 0, and 1. According to the values of the decimal numbers of the 6-bit segments, the 2 of the first layer, the 62 of the second layer, the 0 of the third layer, the 3 of the fourth layer, the 0 of the fifth layer, and the 1 of the sixth layer are sequentially queried, and finally the data corresponding to the 1 of the sixth layer is found to be the to-be-read data.

[0129] The above method provided by the present embodiment can quickly determine whether the to-be-read data exists in the radix tree by querying the radix tree. Since the radix tree has a fast query speed, the processing efficiency of the read request is improved.

[0130] In order to perform the corresponding steps in the above embodiments and various possible implementation manners, an implementation manner of a request processing apparatus is given below. Please refer to Figure 12 , Figure 12 the block schematic diagram of the request processing apparatus 100 provided by the present embodiment. It should be noted that the basic principle and the technical effects generated by the request processing apparatus 100 provided by the present embodiment are the same as those of the above embodiments. For brief description, part of the present embodiment is not mentioned.

[0131] The present application provides a request processing apparatus applied to a storage device, the storage device comprising a hard disk, and the apparatus comprising:

[0132] The obtaining module 110 is configured to obtain a radix tree, the radix tree comprising a plurality of leaf nodes, each leaf node comprising a preset number of index values arranged in sequence;

[0133] The processing module 120 is configured to, if a target leaf node exists in the radix tree, perform to-be-disk data merging according to effective index values in the target leaf node to generate a to-be-disk request, the target leaf node being a leaf node having adjacent effective index values, and the effective index value being an index value corresponding to to-be-disk data.

[0134] The sending module 130 is configured to send the to-be-disk request to the hard disk, so that the hard disk writes the merged to-be-disk data into the hard disk according to the to-be-disk request.

[0135] In an optional implementation, the sending module 130 is specifically configured to: obtain a minimum value and a maximum value in the continuous valid index values in the target leaf node; take a starting position of the to-be-disked data corresponding to the minimum value as a disk position; take a sum of lengths of the to-be-disked data corresponding to the valid index values between the minimum value and the maximum value as a to-be-disked length; and generate a disk request according to the disk position and the to-be-disked length.

[0136] In an optional implementation, the storage device is in communication connection with the client, and the processing module 120 is further configured to: receive a write request sent by the client, the write request including a to-be-written block identifier, a to-be-written block internal offset, and to-be-written data; determine a to-be-written key according to the to-be-written block identifier and the to-be-written block internal offset; and update the to-be-written data of the write request to a leaf node in the radix tree according to the to-be-written key.

[0137] In an optional implementation, the radix tree includes nodes of preset levels, the node at a minimum level is a root node, the node at a maximum level is a leaf node, and the nodes at the remaining levels are intermediate nodes, each of the nodes includes a preset number of index values arranged in sequence, the to-be-written key is divided into bit segments in a descending order of bits according to an ascending order of serial numbers, the number of the bit segments is a preset level, a bit segment with a smallest serial number is taken as a current bit segment, and the root node is taken as a parent node; the processing module 120 is specifically configured to: determine a current index value corresponding to the current bit segment from the preset number of index values of the parent node; if the parent node is not an intermediate node at the maximum level, determine a current intermediate node corresponding to the current index value from an intermediate node at a next level associated with the parent node; take the current intermediate node as the parent node and take a next bit segment adjacent to the current bit segment in the order of serial numbers as the current bit segment, and repeat the above steps until the parent node is an intermediate node at the maximum level; and update the to-be-written data of the write request to a leaf node corresponding to the current index value.

[0138] In an optional implementation, when updating the to-be-written data of the write request to the leaf node corresponding to the current index value, the processing module 120 is specifically configured to: if there is current data in the leaf node corresponding to the current index value, obtain a version number of the current data; and if a version number of the to-be-written data of the write request is greater than the version number of the current data, overwrite the current data with the to-be-written data of the write request.

[0139] In an optional implementation, the processing module 120 is further configured to: receive a read request sent by the client, the read request including a to-be-read block identifier and a to-be-read block internal offset; determine whether to-be-read data desired to be read by the read request exists in the radix tree according to the to-be-read block identifier and the to-be-read block internal offset; read the to-be-read data from the radix tree if the to-be-read data exists in the radix tree; and read the to-be-read data from the hard disk if the to-be-read data does not exist in the radix tree.

[0140] In an optional implementation, the processing module 120 is specifically configured to: determine the to-be-read keyword according to the to-be-read block identifier and the offset within the to-be-read block; if there is a leaf node corresponding to the to-be-read keyword in the radix tree, determine that the to-be-read data exists in the radix tree; otherwise, determine that the to-be-read data does not exist in the radix tree.

[0141] The embodiment of the present application further provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the request processing method according to any one of the foregoing embodiments.

[0142] To sum up, the embodiment of the present application provides a request processing method, device, storage equipment and storage medium, which is applied to the storage equipment including a hard disk, and the method comprises: acquiring a radix tree, the radix tree comprising a plurality of leaf nodes, each leaf node comprising a preset number of index values arranged in sequence; if there is a target leaf node in the radix tree; then, according to the effective index values in the target leaf node, merging to-be-written data to generate a writing request, the target leaf node being a leaf node having adjacent effective index values, and the effective index value being an index value corresponding to the to-be-written data; and issuing the writing request to the hard disk, so that the hard disk writes the merged to-be-written data into the hard disk according to the writing request. Compared with the prior art, the embodiment of the present application firstly merges the to-be-written data by using the radix tree, and then writes the merged data together, thereby reducing the number of IO interactions between the storage equipment and the hard disk, and improving the IO request processing capability of the storage equipment.

[0143] The above merely provides a specific implementation of the present application, but the protection scope of the present application is not limited thereto, any person skilled in the art can easily think of changes or replacements within the technical range disclosed by the present application, which should be covered in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A request processing method, characterized in that, Applied to a storage device, the storage device including a hard disk, the storage device being communicatively connected to a client, the method includes: Obtain a radix tree, which includes nodes at a preset level, with the node at the lowest level being the root node, the node at the highest level being the leaf node, and the nodes at the remaining levels being intermediate nodes. Each node includes a preset number of index values ​​arranged in order. If a target leaf node exists in the radix tree, the data to be removed is merged according to the valid index value in the target leaf node to generate a removal request. The target leaf node is a leaf node with adjacent valid index values, and the valid index value is the index value of the corresponding data to be removed. The disk partitioning request is sent to the hard disk so that the hard disk writes the merged data to be partitioned to the hard disk according to the disk partitioning request. The method further includes: Receive a write request sent by the client, the write request including the identifier of the block to be written, the offset within the block to be written, and the data to be written; Based on the identifier of the block to be written and the offset within the block to be written, the key to be written is determined. The key to be written is divided into bit segments with sequentially increasing numbers from high to low bits. The number of bit segments is the preset level. The bit segment with the smallest sequence number is taken as the current bit segment, and the root node is taken as the parent node. Determine the current index value corresponding to the current bit segment from the preset number of index values ​​of the parent node; If the parent node is not a leaf node, then determine the current intermediate node corresponding to the current index value from the intermediate nodes of the next level associated with the parent node; Take the current intermediate node as the parent node, and take the next bit segment adjacent to the current bit segment as the current bit segment. Repeat the above steps until the parent node is a leaf node. The data to be written in the write request is updated to the leaf node based on the current index value.

2. The request processing method as described in claim 1, characterized in that, The step of merging the data to be dismantled based on the valid index values ​​in the target leaf nodes and generating a dismantling request includes: Obtain the minimum and maximum values ​​among consecutive valid index values ​​in the target leaf node; The starting position of the data to be dropped corresponding to the minimum value is taken as the dropping position; The sum of the lengths of the data to be disassembled corresponding to the valid index values ​​between the minimum and the maximum values ​​is taken as the length of the data to be disassembled. The plate-down request is generated based on the plate-down position and the plate-down length to be downed.

3. The request processing method as described in claim 1, characterized in that, The step of updating the data to be written in the write request to the leaf node corresponding to the current index value includes: If the current data exists in the leaf node corresponding to the current index value, then obtain the version number of the current data; If the version number of the data to be written in the write request is greater than the version number of the current data, then the data to be written in the write request will overwrite the current data.

4. The request processing method as described in claim 1, characterized in that, The method further includes: Receive a read request sent by the client, the read request including the identifier of the block to be read and the offset within the block to be read; Based on the identifier of the block to be read and the offset within the block to be read, it is determined whether the data to be read by the read request exists in the radix tree; If the data to be read exists in the radix tree, then the data to be read is read from the radix tree; If the data to be read does not exist in the radix tree, then the data to be read is read from the hard disk.

5. The request processing method as described in claim 4, characterized in that, The step of determining whether the data to be read by the read request exists in the radix tree based on the identifier of the block to be read and the offset within the block to be read includes: The keyword to be read is determined based on the identifier of the block to be read and the offset within the block to be read; If a leaf node corresponding to the keyword to be read exists in the radix tree, then it is determined that the data to be read exists in the radix tree; Otherwise, it is determined that the data to be read does not exist in the radix tree.

6. A request processing apparatus, characterized in that, Applied to a storage device, the storage device including a hard disk, the storage device being communicatively connected to a client, the device comprising: The acquisition module is used to acquire a radix tree, which includes nodes of a preset level, with the node at the lowest level being the root node, the node at the highest level being the leaf node, and the nodes at the other levels being intermediate nodes. Each node includes a preset number of index values ​​arranged in order. The processing module is used to merge the data to be removed from the disk according to the valid index value in the target leaf node if there is a target leaf node in the radix tree, and generate a removal request. The target leaf node is a leaf node with adjacent valid index values, and the valid index value is the index value of the corresponding data to be removed from the disk. The sending module is used to send the disk partitioning request to the hard disk, so that the hard disk writes the merged data to be partitioned to the hard disk according to the disk partitioning request; The processing module is further configured to: receive a write request sent by the client, the write request including a block identifier to be written, an offset within the block to be written, and data to be written; determine a key to be written based on the block identifier and the offset within the block to be written, the key to be written being divided into bit segments with sequentially increasing sequence numbers according to the bits from high to low, the number of bit segments being the preset level, taking the bit segment with the smallest sequence number as the current bit segment, and taking the root node as the parent node; determine the current index value corresponding to the current bit segment from the index values ​​of the preset number of parent nodes; if the parent node is not a leaf node, determine the current intermediate node corresponding to the current index value from the intermediate nodes of the next level associated with the parent node; take the current intermediate node as the parent node, take the next bit segment adjacent to the sequence number of the current bit segment as the current bit segment, and repeat the above processing until the parent node is a leaf node; update the data to be written in the write request to the leaf node according to the current index value.

7. A storage device, characterized in that, It includes a processor and a memory, the memory being used to store a program, and the processor being used to implement the request processing method as described in any one of claims 1-5 when executing the program.

8. A computer-readable storage medium, characterized in that, It stores a computer program that, when executed by a processor, implements the request processing method as described in any one of claims 1-5.

Images