A hybrid data update method and storage system for memory key-value erasure codes

Abstract

The present invention provides a hybrid data update method and storage system for memory key-value erasure code, which belongs to the field of reliable data storage of computers. The method includes: constructing a hybrid update storage system; obtaining the stripe ID, sequence number, offset and length of the object according to the object index metadata; reading the historical value of the data block corresponding to the object and the XOR value of the stripe, and obtaining the coding coefficient of each check block; subtracting the historical value of the data block from the updated value to obtain the difference of the data block; using the difference of the data block and the coding coefficient of the check block to calculate the difference of each check block; XORing the XOR block and the difference of the XOR block to update the XOR block; the data block corresponding to the object and the updated XOR block are written into the memory node in an in-place update mode; the check block difference with the offset and length is written into the log node in a check log update mode. The present invention can balance the relationship between storage cost, data update performance and single fault repair performance.

Description

Technical Field

[0001] The present invention belongs to the field of reliable data storage of computers, and more specifically, relates to a hybrid data update method and storage system for memory key-value erasure codes. Background Art

[0002] Parallel file systems on modern HPC clusters often use in-memory key-value storage for efficient data analysis. To ensure storage reliability, many distributed key-value stores store copies of data on different nodes. This approach greatly increases the redundancy of data in the system while achieving fault tolerance. The latest research uses erasure coding technology to ensure the reliability of in-memory key-value storage, which significantly reduces storage costs compared to replication technology. Erasure coding encodes the original data block into a check block, and the original data block and the check block together form a stripe; when data is damaged or lost, the remaining data blocks on the stripe can be used to recover the lost data.

[0003] For big data analysis workloads, data updates are cumbersome and inevitable. In erasure code storage, the update overhead is very high, and the update of any data block will result in the update of all check blocks. In the case of large stripes, each stripe has more data blocks, which may be scattered among dozens or even hundreds of nodes, and the update overhead becomes even greater.

[0004] There are three mainstream methods for updating the data target of erasure codes: 1) in-place update, where the original data blocks and check blocks are replaced in place by the updated data blocks and check blocks; 2) full-stripe update, where the updated data blocks are organized together and directly encoded to generate new stripes; 3) parity logging update, where the update of the check blocks is placed in the log node's disk as an appended update. Among the above three update methods, the in-place update method will occupy higher network transmission resources, the full-stripe update method will have a high storage overhead, and the parity log update eliminates the overhead of reading the original check blocks during the process and the storage overhead is currently the lowest.

[0005] However, today's memory key-value storage systems that deploy erasure codes still only use in-place updates or full-stripe updates. Although the verification log update can effectively reduce the network transmission overhead and system storage overhead during erasure code updates, the gap between the disk performance of the verification log node and the memory key-value storage system performance is huge and difficult to handle. As a result, no system has considered using verification log updates. Specifically, even if the network transmission overhead is reduced, the log node will still reduce the update performance. In addition, when a single failure occurs, since the verification log is stored on the disk, the process of reading and recovering to obtain the latest verification block will also drag down the performance of the degraded read. Summary of the invention

[0006] In view of the defects of the prior art, the present invention provides a hybrid data update method and device for memory key-value erasure codes, which aims to reduce the storage overhead in the memory and improve the update performance when data changes by using a hybrid update storage architecture for data blocks and check blocks and using check log update technology on the memory key-value system based on the hybrid update storage architecture.

[0007] To achieve the above-mentioned object, in a first aspect, the present invention provides a hybrid update storage system for a hybrid memory key-value erasure code system, comprising: a plurality of clients, a proxy server, k+1 memory nodes and r-1 log nodes; k is the number of the data blocks; r is the number of the check blocks;

[0008] The proxy server and the client, the memory node and the log node are all connected in a two-way communication;

[0009] The memory node is used to store and update the XOR blocks in the data block and the check block by in-place update; the log node is used to store and update the remaining check blocks by checking the log update;

[0010] The proxy server serves as the front-end interface of the client, and is used to store stripe metadata, implement access requests transmitted by the client, and repair node failures;

[0011] The client is used to provide external memory access requests.

[0012] More specifically, the queue in the proxy server serves as an encoding buffer for reading the client's write request; the proxy server provides k data blocks of a fixed size for the object and encodes the k data blocks into r check blocks of the same size; the proxy server saves stripe metadata to complete client requests, downgraded reads, and repair operations.

[0013] Preferably, the metadata includes a stripe ID, an object index, and a stripe index;

[0014] The stripe ID is used to distinguish each stripe; the object index is used to store the mapping relationship between the object and the stripe ID, and the mapping relationship between the object and the specific data block metadata; the stripe index is used to store the k data blocks and r check blocks corresponding to each stripe.

[0015] Preferably, the log node includes a buffer and a disk; the buffer is located between the proxy server and the disk;

[0016] The buffer is used to store the differences of multiple check blocks;

[0017] The disk is used to write all the check block differences to the disk in the order of their storage time in the buffer if the buffer is full, and then regularly merge the check block differences of the same stripe, and store the merged differences in the reserved space corresponding to each check block.

[0018] Preferably, if a memory node fails, a check block is read from the log node and k-1 data blocks are read from the memory node to repair the failed memory node.

[0019] In a second aspect, the present invention provides a hybrid data update method for memory key-value erasure codes, comprising the following steps:

[0020] Construct a hybrid update storage system; wherein the storage system includes multiple clients, a proxy server, k+1 memory nodes and r-1 log nodes; the XOR blocks in the data blocks and the check blocks are stored on the memory nodes; the remaining check blocks are stored in the log nodes; k is the number of data blocks; r is the number of check blocks;

[0021] Get the object's stripe ID, sequence number, offset, and length based on the object index metadata;

[0022] Based on the stripe ID of the object, the proxy server reads the historical value of the data block corresponding to the object and the XOR block of the stripe; and obtains the coding coefficient of each check block based on the serial number of the object;

[0023] Subtract the historical value of the data block from the updated value to obtain the difference value of the data block;

[0024] Calculate the difference of each check block using the difference of the data block and the coding matrix coefficient corresponding to the check block;

[0025] XOR-merge the XOR block and the difference between the XOR blocks, and update the XOR block;

[0026] The data block corresponding to the object and the updated XOR block are written into the memory node using an in-place update method;

[0027] The check block difference with offset and length is written into the log node using the check log update method.

[0028] Preferably, the log node includes a buffer and a disk; the buffer is located between the proxy server and the disk;

[0029] The verification log is updated in the following way: the differences of multiple verification blocks are stored in the buffer;

[0030] If the buffer is full, all check block differences are first written to the continuous disk space in the order of their storage time in the buffer;

[0031] The check block differences of the same stripe are merged periodically, and the merged differences are stored in the reserved space corresponding to each check block.

[0032] Preferably, if a memory node fails, a check block is read from the log node and k-1 data blocks are read from the memory node to repair the failed memory node.

[0033] In general, the above technical solution conceived by the present invention has the following beneficial effects compared with the prior art:

[0034] The hybrid data update method provided by the present invention constructs a hybrid update storage system, wherein the XOR blocks of data blocks and check blocks are stored on memory nodes, and the remaining check blocks are stored on log nodes; wherein the data blocks and XOR blocks on the memory nodes are updated by using an in-place update method, while the remaining check blocks on the log nodes are updated by using a check log update method. The combination of the in-place update method and the check log update method can greatly balance the relationship between storage cost and data update performance and single fault repair performance.

[0035] There is a huge gap between disk performance and memory performance in the prior art. If all check blocks are on the log node, when a single data is damaged and needs to be downgraded, the check block difference and the check block must be read from the disk and merged to obtain the latest check block; finally, the latest check block is used to decode with the remaining data block to obtain the damaged data block. In this process, the disk reading is relatively slow. Based on the above reasons, the hybrid data update method for memory key-value erasure code provided by the present invention reads the XOR block from the memory node for encoding when the XOR block exists in the memory node; the data block and the XOR block are updated in situ, which makes it superior to the current mainstream solution in terms of repair performance and memory storage overhead; the remaining check blocks are updated by verification log to ensure efficient update performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] FIG. 1( a ) is a schematic diagram of an update process of an in-place update provided by an embodiment of the present invention;

[0037] FIG1( b ) is a schematic diagram of an update process of a full stripe update when the update frequency is high according to an embodiment of the present invention;

[0038] FIG1( c ) is a schematic diagram of an update process of a full stripe update when the update frequency is low according to an embodiment of the present invention;

[0039] FIG. 1( d ) is a schematic diagram of an update process of a verification log update according to an embodiment of the present invention;

[0040] Figure 2 is a storage architecture diagram of a hybrid update when the parameters provided by an embodiment of the present invention are (6, 3);

[0041] Figure 3 It is a schematic diagram of an updating process of updating D1 to D1' when the parameters provided by an embodiment of the present invention are (6, 3);

[0042] Figure 4 It is a schematic diagram of a merge-based cache log of the check block a+2b provided in an embodiment of the present invention;

[0043] Figure 5 It is a schematic diagram of the PLM implementation process when the parameters provided by the embodiment of the present invention are (2, 2). DETAILED DESCRIPTION

[0044] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0045] On the one hand, the present invention provides a hybrid data update method for a memory key-value erasure code system, comprising the following steps:

[0046] (1) Hybrid Update Storage System

[0047] A hybrid update storage system is adopted for the storage of data blocks and check blocks. For the erasure code deployment with parameters (k, r), the architecture includes multiple clients, a proxy server, k+1 memory nodes and r-1 log nodes; the data blocks are stored in the memory nodes and updated using the in-place update method, and the XOR block in the check block (the first check block) is also placed in the memory node and updated using the in-place update method; the remaining check blocks are stored in the log node and updated using the check log update method; the log node consists of a buffer and a disk, and the buffer is located between the proxy server and the disk. During the update, the difference of the check blocks is written to the buffer of the log node, and then written to the disk of the log node in batches in one step; k is the number of data blocks; r is the number of check blocks;

[0048] (2) Steps to update data

[0049] An update operation means that the object value stored in an existing data block changes to a new value, and the check block in the corresponding stripe needs to be updated at the same time. In order to update the object value stored in the data block, the proxy server first obtains the stripe ID, serial number, offset and length of the object based on the object index metadata. Then the proxy server reads the value of the old data object and the XOR block of the stripe (they are all obtained based on the stripe ID). At this time, the proxy server first calculates the difference of the data block based on the values ​​stored in the new and old data blocks. Then, the difference of the XOR block is calculated using the difference of the data block and the coding matrix coefficient corresponding to the check block (determined by the serial number). The XOR block and the difference of the XOR block are XOR-merged to calculate a new XOR block. The proxy server also sends the difference of the data block to each log node, along with the corresponding coding coefficient of the log node. The log node calculates the difference of other check blocks in a similar way. Finally, the new data object and XOR block are written to the memory node, and the difference of the check block with offset and length is written to the log node.

[0050] (3) Merge-based cache log

[0051] In the log node buffer, there are usually multiple check block differences for one check block; multiple check block differences will be merged into one block and then stored; this method can reduce disk IO during log node update;

[0052] (4) Verification log technology based on merging

[0053] For log nodes, the writing process of log records uses the lazy merged check log technology (PLM); first, the check block differences are written to an additional continuous disk space in chronological order, and then they are read back together, the check block difference data of the same stripe is merged, and then the merged difference is stored;

[0054] The method used to save the check blocks and their differences on the log nodes is to record the check logs in the reserved space. The specific operation is: pre-allocate the corresponding reserved storage space of the check blocks on the log nodes, and when an update occurs, keep the check block differences merged by PLM in the continuous reserved storage space after the old check blocks. This method can reduce the random disk read work when calculating the latest check blocks, and improve the performance of data degradation read repair;

[0055] (5) Node repair optimization technology based on log node assistance

[0056] When a memory node fails, the traditional repair method is to use all remaining memory nodes to perform downgraded read decoding to obtain the data on the failed node, while the existing memory nodes also need to provide continuous services through proxy servers, and transmitting data for repair at the same time is likely to cause network congestion of the node; the node repair based on log node assistance of the present invention uses the relatively idle log node and memory node to perform repair together; taking the erasure code deployed with (k, r) parameters as an example, to restore a block of a stripe on a failed node, this method needs to read a check block from the log node and read k-1 blocks from the memory node, which reduces the network transmission overhead of the memory node compared with the traditional repair method.

[0057] On the other hand, the present invention provides a hybrid update storage system for a hybrid memory key-value erasure code system, comprising: a plurality of clients, a proxy server, k+1 memory nodes and r-1 log nodes; k is the number of the data blocks; r is the number of the check blocks;

[0058] The proxy server and the client, the memory node and the log node are all connected in a two-way communication;

[0059] The memory node is used to store and update the XOR blocks in the data block and the check block by in-place update; the log node is used to store and update the remaining check blocks by checking the log update;

[0060] The proxy server serves as the front-end interface of the client, and is used to store stripe metadata, implement access requests transmitted by the client, and repair node failures;

[0061] The client is used to provide external memory access requests.

[0062] More specifically, the queue in the proxy server serves as an encoding buffer for reading the client's write request; the proxy server provides k data blocks of a fixed size for the object and encodes the k data blocks into r check blocks of the same size; the proxy server saves stripe metadata to complete client requests, downgraded reads, and repair operations.

[0063] Preferably, the metadata includes a stripe ID, an object index, and a stripe index;

[0064] The stripe ID is used to distinguish each stripe; the object index is used to store the mapping relationship between the object and the stripe ID, and the mapping relationship between the object and the specific data block metadata; the stripe index is used to store the k data blocks and r check blocks corresponding to each stripe.

[0065] Preferably, the log node includes a buffer and a disk; the buffer is located between the proxy server and the disk;

[0066] The buffer is used to store the differences of multiple check blocks;

[0067] The disk is used to write all the check block differences to the disk in the order of their storage time in the buffer if the buffer is full, and then regularly merge the check block differences of the same stripe, and store the merged differences in the reserved space corresponding to each check block.

[0068] Preferably, if a memory node fails, a check block is read from the log node and k-1 data blocks are read from the memory node to repair the failed memory node.

[0069] Example

[0070] Before introducing the embodiments of the present invention, three update methods of erasure codes, namely, in-place update, full stripe update, and check log update, are briefly introduced;

[0071] As shown in Figure 1(a) to (d), taking the erasure code configuration with parameters (6,3) as an example, a stripe consists of 6 data blocks D1 to D6 and 3 check blocks P1 to P3; Figure 1(a) provides the process of in-place update, D1 is updated to D1', the node calculates ΔD1 = D1-D1', and transmits it to the nodes where the three check blocks are located. The check block node reads the original check block and combines it with ΔD1 to calculate the updated P1' to P3'; in this process, the check block read operation is performed 3 times, and a total of 9 blocks are stored after the update is completed; Figure 1(b) and Figure 1(c) respectively show the update process of full stripe update under two different update frequencies; Figure 1(b) is the case of frequent updates, and all data blocks D1 to D6 are updated to D1' to D6'; the updated data blocks are re-encoded New data blocks P4~P6; old data blocks and old check blocks are cleaned up by the subsequent garbage collection process; there is no check block read operation in this update process, and a total of 18 blocks are stored after the update is completed; Figure 1(c) is when the update is not frequent, only D1 is updated to D1'; at this time, D1' is encoded again with D2~D6 to obtain check blocks P7~P9; in this update process, the check blocks are recalculated 3 times, and a total of 13 blocks are stored after the update is completed; Figure 1(d) provides the process of updating the check log, D1 is updated to D1', the node calculates ΔD1=D1-D1', and transmits it to the nodes where the three check logs are located, calculates the check block increments ΔP1~ΔP3, and saves the increments in the log. There is no check block read operation in this update process, and a total of 12 blocks are stored after the update is completed;

[0072] The hybrid data update method for a memory key-value erasure code system according to an embodiment of the present invention is further described below in conjunction with a specific implementation method, including the following steps:

[0073] (1) Hybrid Update Storage Architecture

[0074] Figure 2 It is a storage architecture for mixed updates when the parameters are (6,3); the architecture includes multiple clients, a proxy server, 7 memory nodes and 2 log nodes; taking a stripe as an example, data blocks D1-D6 are stored in 6 memory nodes and updated using in-place updates; XOR check block P1 is also stored in the memory node and updated using in-place updates; the remaining 2 check blocks P2 and P3 are stored in the log node and updated using the check log update method; the log node consists of a buffer and a disk, and the buffer is located between the proxy server and the disk. When updating, the difference of the check blocks is written to the buffer of the log node, and then asynchronously written to the disk of the log node in batches;

[0075] (2) Steps to update data

[0076] like Figure 3 As shown in the figure, the workflow in the case of erasure code deployment with parameters (6,3) updates the data block D1 to D1';

[0077] (2.1) The proxy server obtains the stripe ID, sequence number, offset and length of the updated object by updating the object index metadata;

[0078] (2.2) The node where D1 and P1 are located transmits D1 and P1 to the proxy server;

[0079] (2.3) The proxy server calculates ΔD1 and P1', updates D1 to D1', and P1 to P1'. The proxy server sends ΔD1 to the nodes where P2 and P3 are located.

[0080] (2.4) After the update is completed, the nodes where D1 and P1 are located send confirmation information to the proxy server. The nodes where P2 and P3 are located calculate ΔP2 and ΔP3 respectively and save them in the buffer of their respective nodes;

[0081] (2.5) The nodes where P2 and P3 are located send a confirmation message to the proxy server;

[0082] (2.6) After the proxy server receives the confirmation information from the four nodes D1, P1, P2, and P3, it confirms that the update operation has been completed;

[0083] (3) Merge-based cache log

[0084] Figure 4It is the cache status of the node where the check block with content a+2b is located; suppose the incoming data stream is: a, b, a', b', a''; where a and b are the data of the original data block; a', a'' are updated by a, and b' is updated by b; the cache area stores the check block increments a'-a, 2(b'-b), and a''-a' calculated according to the data block increment, a total of 3 check block increments, these three check block increments are merged into a check block increment (a''-a)+2(b'-b), and then stored;

[0085] (4) Verification log technology based on merging

[0086] like Figure 5 As shown in the figure, it is a process diagram of a log node PLM in an erasure code architecture with parameters (2,2) deployed; assuming that a and b, c and d are data blocks from two stripes respectively, and the second check blocks of these two stripes are stored in the log node shown in the figure; the data input and update process in the system is: a, b, c, d, a', c', c", b', a", b"; the corresponding data block stream in the log node cache is: a+2b, c+2d, a'-a, c'-c, c"-c', 2(b'-b), a"-a', 2(b"-b'); the above data blocks will be directly stored in a continuous disk space according to PLM; then they are read out to calculate the merged result, and after the merge calculation is completed, blocks a"+2b" and c"+2d are obtained and stored in the pre-allocated log file on the disk, and the area reserved on the right side of the check block is used to store the check block difference that may appear in the future;

[0087] (5) Log-assisted node repair

[0088] Taking the erasure code with (6,3) parameter deployment as an example, to recover a block of a stripe on a failed node under log-assisted node repair, it is necessary to read 1 check block from the log node and 5 blocks from the memory node. Compared with the traditional repair method, the number of blocks read from the memory node is reduced. Since not all memory nodes are used, the corresponding method can be used to achieve parallel implementation of data repair on the node.

[0089] In summary, compared with the prior art, the present invention has the following advantages:

[0090] The hybrid data update method provided by the present invention constructs a hybrid update storage system, wherein the XOR blocks of data blocks and check blocks are stored on memory nodes, and the remaining check blocks are stored on log nodes; wherein the data blocks and XOR blocks on the memory nodes are updated by using an in-place update method, while the remaining check blocks on the log nodes are updated by using a check log update method. The combination of the in-place update method and the check log update method can greatly balance the relationship between storage cost and data update performance and single fault repair performance.

[0091] There is a huge gap between disk performance and memory performance in the prior art. If all check blocks are on the log node, when a single data is damaged and needs to be downgraded, the check block difference and the check block must be read from the disk and merged to obtain the latest check block; finally, the latest check block is used to decode with the remaining data block to obtain the damaged data block. In this process, the disk reading is relatively slow. Based on the above reasons, the hybrid data update method for memory key-value erasure code provided by the present invention reads the XOR block from the memory node for encoding when the XOR block exists in the memory node; the data block and the XOR block are updated in situ, which makes it superior to the current mainstream solution in terms of repair performance and memory storage overhead; the remaining check blocks are updated by verification log to ensure efficient update performance.

[0092] It will be easily understood by those skilled in the art that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A hybrid data update storage system for memory key-value erasure codes, characterized in that: include: Several clients, proxy servers, memory nodes, and log nodes; The proxy server is bidirectionally connected to the client, the memory node, and the log node; The number of memory nodes is k+1, which are used to store and update the XOR blocks of data blocks and check blocks in an in-place update manner, where k is the number of data blocks; the number of log nodes is r-1, which are used to store and update the remaining check blocks in a check log update manner, where r is the number of check blocks; The proxy server serves as a front-end interface of the client, and is used to store stripe metadata, implement memory access requests transmitted by the client, and repair node failures; The client is used to provide external memory access requests; Wherein, the log node includes a buffer and a disk; the buffer is located between the proxy server and the disk; The buffer is used to store multiple check block differences; the disk is used to write all check block differences in the order of time storage in the buffer if the buffer is full, merge the check block differences of the same stripe, and store the merged differences in the reserved space corresponding to each check block.

2. The hybrid data update storage system according to claim 1, characterized in that: The metadata includes a stripe ID, an object index, and a stripe index; The stripe ID is used to distinguish each stripe; the object index is used to store the mapping relationship between the object and the stripe ID, and the mapping relationship between the object and the specific data block metadata; the stripe index is used to store the k data blocks and r check blocks corresponding to each stripe.

3. The hybrid data update storage system according to claim 1 or 2, characterized in that: If a memory node fails, a check block is read from the log node and k-1 data blocks are read from the memory node to repair the failed memory node.

4. A hybrid data update method for memory key-value erasure codes, characterized in that: The following steps are involved: Construct a hybrid update storage system; wherein the storage system includes multiple clients, a proxy server, k+1 memory nodes and r-1 log nodes; the XOR blocks in the data blocks and the check blocks are stored in the memory nodes; the remaining check blocks are stored in the log nodes; k is the number of the data blocks; r is the number of the check blocks; Get the object's stripe ID, sequence number, offset, and length based on the object index metadata; Based on the stripe ID of the object, the proxy server reads the historical value of the data block corresponding to the object and the XOR value of the stripe, and obtains the coding coefficient of each check block based on the serial number of the object; Subtract the historical value of the data block from the updated value to obtain the difference value of the data block; Calculating the difference of each check block using the difference of the data block and the coding coefficient of the check block; XOR-merge the XOR block and the difference between the XOR blocks, and update the XOR block; The data block corresponding to the object and the updated XOR block are written into the memory node using an in-place update method; The check block difference with offset and length is written into the log node by using the check log update method; The log node includes a buffer and a disk; the buffer is located between the proxy server and the disk; The verification log is updated in the following way: Store multiple check block differences in the buffer; if the buffer is full, write all check block differences to continuous disk space in the order of their storage time in the buffer; regularly merge the check block differences of the same stripe, and store the merged differences in the reserved space corresponding to each check block.

5. The hybrid data updating method according to claim 4, characterized in that: If a memory node fails, a check block is read from the log node, and a total of k-1 data blocks and XOR blocks are read from the memory node to repair the failed memory node.

Images