An erasure code storage system and data update method

Abstract

This invention relates to the field of clustered storage technology, and more particularly to an erasure coding storage system and a data update method. The method includes: multiple data centers, multiple racks located in each data center, and multiple computer nodes located in each rack; the data centers are interconnected via a network, and computer nodes located in the same rack and in different racks are interconnected via network devices; the multiple computer nodes include several client nodes, several management nodes, and several storage nodes; the multiple storage nodes corresponding to each data object include one primary storage node and multiple secondary storage nodes. The primary storage node is used to determine the optimal update mechanism based on the data update location, update data volume, and rack relationship of the storage nodes corresponding to the update interval of the data object, thereby minimizing cross-rack data transfer during the update process and improving update efficiency.

Description

Technical Field

[0001] This invention relates to the field of clustered storage technology, and in particular to an erasure coding storage system and a data update method. Background Technology

[0002] Traditional data centers or storage clusters typically employ a multi-replica data redundancy strategy, which offers high access performance but suffers from high storage redundancy and costs. To address these issues, an increasing number of commercial storage systems are introducing erasure coding (including traditional erasure coding, storage regeneration coding / network coding) data protection strategies, achieving equivalent reliability with less data redundancy. Erasure coding algorithms encode multiple data blocks to generate multiple parity blocks, where data blocks and parity blocks are collectively referred to as coded blocks. When a coded block becomes unavailable, it can be recovered by reading the surviving data blocks and parity blocks. To maintain consistency between data blocks and parity blocks, erasure coding requires updating all parity blocks simultaneously when updating data blocks, increasing write latency, leading to write amplification, and reducing storage system update performance. Therefore, erasure coding storage systems suffer from low update efficiency. Summary of the Invention

[0003] This application provides an erasure coding storage system and a data update method, which can at least solve the above-mentioned technical problems.

[0004] In a first aspect, embodiments of the present invention provide an erasure coding storage system, comprising: multiple data centers, multiple racks located in each data center, and multiple computer nodes located in each rack; the data centers are interconnected via a network, and computer nodes located in the same rack and computer nodes located in different racks are interconnected via network devices;

[0005] The plurality of computer nodes include several client nodes, several management nodes, and several storage nodes; the plurality of storage nodes corresponding to each data object include one primary storage node and several secondary storage nodes, wherein the primary storage node is a storage node of the data object that meets the set requirements, and the secondary storage nodes are the storage nodes of the data object other than the primary storage node;

[0006] The primary storage node is used to obtain the updated data of the object access request sent by the client node and the encoding storage mapping table of the data object corresponding to the updated data sent by the management node.

[0007] Based on the storage locations of the n encoded blocks of the data object in the encoded storage mapping table, the storage architecture of the data object is determined, wherein the n encoded blocks include k historical data blocks and m historical check blocks, where n, k, and m are all integers not less than 1, and n = k + m;

[0008] If the storage architecture is a planar architecture, then according to the updated data, a planar architecture update process is performed on the data object, wherein the planar architecture is an architecture in which the storage locations of the n coded blocks are located in different racks;

[0009] If the storage architecture is a hierarchical architecture, then a hierarchical architecture update process is performed on the data object according to the updated data, wherein the hierarchical architecture is an architecture in which at least two coded blocks are stored in the same rack.

[0010] Preferably, each client node is used for:

[0011] Receive data access requests from remote or local applications, including access requests sent by file systems, object storage, and / or block storage devices;

[0012] The data access request is converted into an object access request, which is an access request for a fixed-length data object / data segment.

[0013] The system queries the main management node for the primary storage node of the data object corresponding to the updated data of the object access request, and then sends the updated data to the primary storage node of the corresponding data object.

[0014] Preferably, the management node is used for:

[0015] The management nodes maintain the consistency of cluster information of the system through a consensus algorithm and elect the master management node for each data object;

[0016] Maintain the status information, location information, and storage device information of each storage device of each storage node;

[0017] Maintain each data object and the encoding storage mapping table for each data object, wherein the encoding storage mapping table is a list of mapping relationships between each encoding block of the data object and the storage device where each encoding block is stored;

[0018] Respond to the client node's request to obtain the primary storage node for each data object;

[0019] Respond to the request from the primary storage node of each data object to obtain the encoded storage mapping table of each data object.

[0020] Preferably, the multiple auxiliary nodes of each data object include: multiple data nodes and multiple verification nodes, wherein the data nodes are storage nodes for storing data blocks of the data object, and the verification nodes are storage nodes for storing verification blocks of the data object;

[0021] The rack where the primary storage node is located is the main rack, and the rack where the secondary storage node is located is the secondary rack;

[0022] Each storage node includes a processor, memory, a network interface card (NIC) connected to the processor, and several storage devices.

[0023] Based on the same inventive concept, in a second aspect, the present invention also provides a data update method for an erasure coding storage system, applied to the main storage node of the erasure coding storage system as described above, the method comprising:

[0024] Obtain the updated data of the object access request sent by the client node and the encoding storage mapping table of the data object corresponding to the updated data sent by the management node;

[0025] Based on the storage locations of the n encoded blocks of the data object in the encoded storage mapping table, the storage architecture of the data object is determined, wherein the n encoded blocks include k historical data blocks and m historical check blocks, where n, k, and m are all integers not less than 1, and n = k + m;

[0026] If the storage architecture is a planar architecture, then according to the updated data, a planar architecture update process is performed on the data object, wherein the planar architecture is an architecture in which the storage locations of the n coded blocks are located in different racks;

[0027] If the storage architecture is a hierarchical architecture, then a hierarchical architecture update process is performed on the data object according to the updated data, wherein the hierarchical architecture is an architecture in which at least two coded blocks are stored in the same rack.

[0028] Preferably, the step of performing a planar architecture update process on the data object based on the updated data includes:

[0029] Obtain the data block to be updated located in the auxiliary rack corresponding to the updated data and the data block not updated located in the auxiliary rack of the data object;

[0030] If the number of data blocks to be updated located in the auxiliary rack is less than the number of data blocks not yet updated located in the auxiliary rack, then the incremental update mode (Delta Based Approach, or DBA mode for short) is entered.

[0031] Otherwise, it enters Reconstruct Writes (RCW mode).

[0032] Preferably, the step of performing a hierarchical architecture update process on the data object based on the updated data includes:

[0033] Obtain the data block to be updated located in the auxiliary rack corresponding to the updated data and the data block not updated located in the auxiliary rack of the data object;

[0034] If the number of data blocks to be updated located in the auxiliary rack is less than the number of data blocks not yet updated located in the auxiliary rack, then enter DBA mode;

[0035] If the number of data blocks to be updated located in the auxiliary rack is greater than the number of data blocks not yet updated located in the auxiliary rack, then enter RCW mode.

[0036] Preferably, after obtaining the data block to be updated located in the auxiliary rack corresponding to the updated data and the unupdated data block of the data object located in the auxiliary rack, the method further includes:

[0037] If the number of data blocks to be updated located in the auxiliary rack is equal to the number of data blocks not updated located in the auxiliary rack, then obtain the data blocks to be updated and the data blocks not updated located in the auxiliary storage node in the main rack.

[0038] In the main rack, if the number of data blocks to be updated on the secondary storage node is less than the number of data blocks not yet updated on the secondary storage node, the system enters the DBA mode; if the number of data blocks to be updated on the secondary storage node is not less than the number of data blocks not yet updated on the secondary storage node, the system enters the RCW mode.

[0039] Preferably, the RCW mode further includes:

[0040] Read t new data blocks of the updated data and kt unupdated data blocks of the data object, where k≥t≥1;

[0041] The t new data blocks and the kt unupdated data blocks are reconstructed to obtain the reconstructed data object;

[0042] The reconstructed data object is encoded to generate m new check blocks, and the m new check blocks are respectively sent to the check nodes where the historical check blocks of the data object are stored, so that each check node replaces the historical check block with the new check block.

[0043] The t new data blocks are sent to the storage node where the historical data block corresponding to each new data block is stored, so that each storage node replaces the corresponding historical data block with the new data block.

[0044] Preferably, the DBA mode further includes:

[0045] Read t new data blocks of the updated data and the historical data blocks corresponding to each of the t new data blocks;

[0046] Each new data block is incrementally compared with its corresponding historical data block to generate an incremental data block, thus obtaining t incremental data blocks;

[0047] Based on the length of the data object, the t incremental data blocks and kt empty data blocks are combined to obtain the combined data object;

[0048] The combined data object is encoded to generate m incremental verification blocks. The m incremental verification blocks are then sent to the verification nodes where the historical verification blocks of the data object are stored, so that each verification node performs incremental calculations on the incremental verification blocks and the historical verification blocks to obtain the target verification block and replaces the historical verification block with the target verification block.

[0049] Each new data block is sent to the storage node where the corresponding historical data block is stored, so that each storage node replaces the corresponding historical data block with the new data block.

[0050] One or more technical solutions in the embodiments of the present invention have at least the following technical effects or advantages:

[0051] Compared to traditional data update methods, the data update method of this invention, based on the structure of the erasure coding storage system of this invention, solves the problems of traditional data update methods, namely, the write amplification problem and the low update efficiency of the erasure coding storage system caused by reduced storage system update performance. The data update method of this invention determines whether to use RCW or DBA mode to update data based on the update location of the data object, the length of the updated data, and the distribution of the storage nodes corresponding to the update interval in the rack. This minimizes cross-rack data transmission during updates (cross-rack bandwidth is a scarce resource, generally a fraction of the intra-rack bandwidth, and in extreme cases, a fraction of a fraction), thus improving update efficiency. When several racks are distributed across multiple data centers across a wide area network, this invention also reduces cross-data center data transmission bandwidth, thereby improving the data update performance of the erasure coding storage cluster deployed across a wide area network. Attached Figure Description

[0052] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0053] Figure 1A cluster diagram of the erasure coding storage system in an embodiment of the present invention is shown;

[0054] Figure 2 A schematic diagram illustrating the RS encoding process and data object writing process in an embodiment of the present invention is shown;

[0055] Figure 3 A schematic diagram illustrating the regenerated code encoding process and the data object writing process in an embodiment of the present invention is shown;

[0056] Figure 4 A flowchart illustrating the steps of a data update method for an erasure coding storage system according to an embodiment of the present invention is shown.

[0057] Figure 5 A schematic diagram illustrating the execution of RCW mode under a planar architecture in an embodiment of the present invention is shown;

[0058] Figure 6 This diagram illustrates the execution of the DBA mode under a planar architecture in an embodiment of the present invention.

[0059] Figure 7 A flowchart illustrating the hierarchical architecture update process in an embodiment of the present invention is shown.

[0060] Figure 8 An example diagram of the execution of the RCW mode under the hierarchical architecture in an embodiment of the present invention is shown;

[0061] Figure 9 This diagram illustrates another execution example of the RCW mode under the hierarchical architecture in an embodiment of the present invention.

[0062] Figure 10 An execution example diagram of the DBA mode under the hierarchical architecture in an embodiment of the present invention is shown;

[0063] Figure 11 This diagram illustrates another execution example of the DBA mode under the hierarchical architecture in an embodiment of the present invention. Detailed Implementation

[0064] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0065] Example 1

[0066] The first embodiment of the present invention provides an erasure coding storage system, such as... Figure 1As shown, this includes: multiple data centers, multiple racks in each data center, and multiple computer nodes in each rack. Data centers are interconnected via networks, meaning the racks within data centers are interconnected via networks, including core networks, backbone networks, or wide area networks (WANs). Computer nodes within the same rack and computer nodes in different racks are interconnected via network devices. These network devices can be configured according to actual needs; for example, computer nodes within the same rack may be interconnected via Top of Rack (ToR) switches, while computer nodes in different racks may be interconnected via aggregate switches and / or core switches (such as core routers).

[0067] The multiple computer nodes include several client nodes, several management nodes, and several storage nodes. In some embodiments, client nodes, management nodes, and storage nodes may be located in the same rack or in the same computer node. When client nodes and storage nodes are in the same computer node, the system is deployed in a converged deployment. Each storage node includes a processor, memory connected to the processor, a network interface card (NIC), and several storage devices. Storage devices are the final persistent storage targets for data, such as hard disks (HDDs), solid-state drives (SSDs), and persistent memory (PM).

[0068] like Figure 1 As shown, Figure 1 This is a schematic diagram of an erasure coding storage system. Figure 1 The erasure coding storage system comprises racks 100, 110, 120, 130, and 140. Each rack includes multiple computer nodes. Rack 100 includes multiple client nodes, namely client nodes 102, 104, and 106. Rack 110 includes management nodes 112, 114, and 116. Rack 120 includes storage nodes 122, 124, and 126; rack 130 includes storage nodes 132, 134, and 136; and rack 140 includes storage nodes 142, 144, and 146. The racks are interconnected via a core network 150. Computer nodes within each rack are interconnected via ToR switches (ToRSwitch). Computer nodes in rack 100 are interconnected via ToR switch 108, those in rack 110 via ToR switch 118, those in rack 120 via ToR switch 128, those in rack 130 via ToR switch 138, and those in rack 140 via ToR switch 148. Computer nodes in different racks are interconnected via core switches; for example, client node 102 in rack 100 and management node 112 in rack 110 are interconnected via a core switch.

[0069] Each data object corresponds to multiple storage nodes, including one primary storage node and multiple secondary storage nodes. The primary storage node is the storage node that meets the defined requirements of the data object, while the secondary storage nodes are the storage nodes for the data object other than the primary storage node.

[0070] Specifically, if a data object is an object, stripe, or segment, then a data object is a fixed-length segment of data protected by a data redundancy strategy (such as erasure coding algorithms). Objects and segments generally refer to original data blocks, while stripes contain parity data blocks. Data blocks can also be called data fragments, and parity data blocks can be called parity fragments. In some embodiments, the data content contained in a data object can be determined based on the context.

[0071] For each data object, after encoding, the object / strip / data segment generates n encoded blocks. Therefore, a data object comprises n encoded blocks, which include k data blocks and m parity blocks, where n, k, and m are all integers not less than 1, and n = k + m. Since the data object is already stored in this system, the data blocks are called historical data blocks, and the parity blocks are called historical parity blocks. Furthermore, each of the n encoded blocks of the data object is stored in a specific storage device on a specific storage node. Thus, the data object is stored across the corresponding N storage nodes, meaning it is stored on n storage devices across the corresponding N storage nodes, satisfying n ≥ N ≥ 1. The N storage nodes are divided into one primary storage node and N-1 secondary storage nodes. The primary storage node is a storage node that meets the set requirements for the data object; these requirements can be set according to actual needs. Therefore, the primary storage node can be the storage node where the first data block of a data object is stored, or the storage node where the first storage device in the data object's encoded storage mapping table is located, or a storage node where a randomly assigned data block of a data object is stored, or a storage node where a data block of a data object is stored according to actual needs. Secondary storage nodes are the storage nodes in a data object other than the primary storage node. Multiple secondary nodes for a data object include: multiple data nodes and multiple check nodes. Data nodes are the storage nodes where data blocks of the data object are stored, and check nodes are the storage nodes where check blocks of the data object are stored. It should also be noted that any storage node in this system can serve as the primary storage node for a data object.

[0072] Furthermore, for each data object, the rack where the primary storage node resides is called the main rack, and the rack where the secondary storage node resides is called the secondary rack. The N storage nodes corresponding to the data object are located in r racks. If r = N = n, it means that each rack in the N = n racks has one storage node for the data object, and it also means that the n encoded blocks of the data object are located in different racks (i.e., in the N = n racks), that is, each rack in the N = n racks stores one encoded block of the data object. Therefore, if r = N, the storage architecture of the data object is a flat architecture. If n > r ≥ 1, it means that there are at least two encoded blocks of the data object stored in the same rack in the r racks, and the storage architecture of the data object is a hierarchical architecture. In some embodiments, the n encoded blocks of the data object are evenly distributed in the r racks, assuming that n is divisible by r, that is, each rack stores 1 coded block, and satisfying In addition, any storage device in this system can store a certain encoded block of multiple data objects.

[0073] The client node, A1, is used to: receive data access requests from remote or local applications. Data access requests include those from file systems, object storage (such as Amazon S3), and / or block storage devices. If the data access request pertains to various requests for application data, it includes write requests, read requests, update requests, etc. The client node receives and processes all requests from the application. In this embodiment of the invention, the initial creation operation of a data object is called a write, and subsequent partial write operations on the data object are called updates.

[0074] A2. Convert data access requests into object access requests. Object access requests are access requests for fixed-length data objects / segments, i.e., read / write requests for fixed-length data objects / segments. The specific conversion process involves transforming a single file or object storage or block storage device request into multiple fixed-length data object or segment access requests.

[0075] A3. Query the primary management node for the primary storage node of the data object corresponding to the updated data of the object access request, and send the updated data to the primary storage node of the corresponding data object.

[0076] Specifically, for write / update requests of application data, a segmentation process is used to convert the write request into a fixed-length object request. For each data object update request, the client node obtains the primary storage node corresponding to the data object from the management node and sends the update request to the primary storage node where the primary storage device is located. For each data object read request, the client node sends the data object read request to the primary storage node of the data object. The primary storage node of the data object reads the data object from the primary storage device and / or multiple secondary storage devices and returns the relevant data of the data object to the client node. This embodiment of the invention focuses only on write and update requests from the client node. Data object read requests only focus on historical (i.e., old) data read requests issued by the primary storage node of the data object when the data object is updated.

[0077] Management nodes are used for: B1. Maintaining the consistency of cluster information in the system and electing the master management node for each data object through consensus algorithms among management nodes.

[0078] B2. During the initial write process of each data object, based on parameters such as the online status of storage nodes, storage device space utilization, and access performance, several storage devices are selected for the data object. The corresponding storage node for the data object is then determined based on the selected storage devices, thereby generating an encoded storage mapping table for the data object and identifying the primary storage node for the data object. The online status of a storage node in this system indicates its accessibility; if a storage node is offline, its storage devices are inaccessible. This partitioning based on storage device space utilization and performance aims to evenly distribute different data objects across all storage devices in the cluster, achieving storage load balancing.

[0079] B3. Maintain the status information, location information, and storage device information of each storage node and each storage device within each storage node. The status information of each storage node includes its online, offline, faulty, and fault-recovery status. The location information of each storage node includes the location of each storage node and each storage device within the cluster architecture. The storage device information of each storage device within each storage node includes, but is not limited to, the device ID and storage capacity of each storage device.

[0080] B4. Maintain each data object and its encoding-storage mapping table, where the encoding-storage mapping table is a list of mapping relationships between each encoded block of the data object and the storage device where each encoded block is stored. Specifically, using... Figure 1For example, suppose a data object consists of two encoded blocks. The encoding storage mapping table for this data object is a list of mapping relationships where the first encoded block is stored in a storage device on storage node 122 of rack 120, and the second encoded block is stored in a storage device on storage node 132 of rack 130. The management node can divide the stored data objects and their encoding storage mapping tables into different partitions based on the object identifiers of the data objects (e.g., evenly distributed in lexicographical order). Each partition stores the object identifiers and their encoding storage mapping table information within a certain range of object identifiers. Each partition corresponds to a different list of management nodes and a master management node. This allows query requests for the encoding storage mapping tables of different data objects to be sent to different management nodes, achieving load balancing.

[0081] It should be noted that each storage device can be bound to an IP address and a port. The IP address is the IP address of the storage node corresponding to the storage device, and the port is the access port of the storage device. Taking n storage devices on several storage nodes for a certain data object as an example, in the data object's encoded storage mapping table, each storage device's IP address corresponds to a storage node, and its port corresponds to a storage device.

[0082] B5. Respond to client nodes' requests to obtain the primary storage node for each data object. Specifically, client nodes send object access requests to any management node other than the primary management node of the data object whose updated data is requested. These arbitrary management nodes forward the object access requests to the primary management node corresponding to the corresponding partition of the data object. The primary management node of the corresponding data object queries the primary storage node of the corresponding data object and sends the relevant information of the primary storage node to the client nodes. The relevant information of the primary storage node includes the IP address and port of the primary storage node.

[0083] B6. Respond to the request from the primary storage node of each data object to obtain the encoded storage mapping table of each data object. Specifically, for each data object, after receiving the query request for the encoded storage mapping table of the data object sent by the primary storage node of the data object, send the encoded storage mapping table of the data object to the primary storage node of the data object.

[0084] The primary storage node is used for: C1. Encoding the data object during the data object storage phase, generating n encoded blocks, and storing each of the n encoded blocks in a specific storage device of a specific storage node.

[0085] C2. During the data object update phase, obtain the update data of the object access request sent by the client node and the encoding storage mapping table of the data object corresponding to the update data sent by the management node;

[0086] Based on the storage locations of the n encoded blocks of the data object in the encoded storage mapping table, the storage architecture of the data object is determined. The n encoded blocks include k historical data blocks and m historical check blocks, where n, k, and m are all integers not less than 1, and n = k + m.

[0087] If the storage architecture is a flat architecture, then the flat architecture update process is performed on the data object according to the updated data. Here, the flat architecture is an architecture in which the storage locations of n coded blocks are located in different racks.

[0088] If the storage architecture is a hierarchical architecture, then the hierarchical architecture update process is performed on the data object according to the updated data. The hierarchical architecture is an architecture in which at least two coded blocks are stored in the same rack.

[0089] by Figure 1 For example, following the data flow along the dotted line, suppose client node 102 receives a data access request. Client node 102 converts the data access request into an object access request and queries management node 112 for the primary storage node of the data object corresponding to the updated data in the object access request. Then, it obtains the primary storage node 122 of the data object from management node 112. Client node 102 sends the updated data to primary storage node 122. Primary storage node 122 also obtains the encoded storage mapping table of the data object from any of the management nodes. The secondary storage nodes of the data object obtained from the encoded storage mapping table are 124, 126, 132, 134, 136, 142, 144, and 146, respectively.

[0090] Based on its ability to execute layered services or processes located on different computer nodes (groups), and according to its service layering structure, this system comprises an access layer, an encoding layer, a distribution layer, and a storage layer. Each layer of this system is described in detail below:

[0091] The access layer, typically located on client nodes, translates data access requests into object access requests. In some cases, erasure coding storage systems can be viewed as object storage systems, with the storage devices called Object Storage Devices (OSDs). An OSD can correspond to a drive (such as an HDD or SSD) or a partition or directory on a drive within a storage node. A drive can be considered a hard disk, a type of storage device. Each data object has a unique identifier (Object ID, OID) within the cluster's global address space. The management node cluster maintains the mapping between data object identifiers (OIDs) and an object-device-map (ODM). Client nodes retrieve the ODM corresponding to the OID from any management node, obtain the primary storage device from the ODM, and send the updated data from the object access request to the storage node where the primary storage device resides—the primary storage node.

[0092] The encoding layer is typically located on the primary storage node. In some embodiments, the encoding layer and the access layer can both reside on the storage node (or both on the client node), which is referred to as a compute and storage converged deployment. The encoding layer selects a specified encoding algorithm (such as Reed Soloman, RS) and encoding parameters (n, k) according to the data redundancy strategy. For RS(n, k) encoding, the data object is divided into k equal-length data blocks, and m = nk parity blocks are generated through encoding, resulting in a total of n = k + m encoded blocks (data blocks and parity blocks are collectively referred to as encoded blocks).

[0093] The encoding algorithms supported by this invention include linearly operational erasure codes such as RS, Regenerating Codes (RC) / Network Coding (NC), and all involved encodings are (n, k)-MDS encodings, meaning that any k out of n encoded blocks can be decoded to reconstruct the original object. Furthermore, all encoding algorithms are systematic codes, meaning the original data exists in the encoded blocks as k data blocks. When the k data blocks are available, the original object can be directly restored by combination, without the need for linear decoding. For RS encoding, the data block is the basic encoding unit; for regenerating codes or network coding, the encoding and decoding process further splits each encoded block into multiple sub-blocks; this splitting operation is called sub-packetization. In regenerating code semantics, sub-blocks are also called symbols. To improve access efficiency, multiple sub-blocks in regenerating codes are generally stored adjacently, and the sub-blocks are the basic encoding units. To more clearly illustrate this invention, the data update process of the erasure coding storage system uses coded blocks (RS codes) or sub-blocks (regenerated codes) as the basic read / write units (i.e., indivisible basic units). For smaller data update patterns, those skilled in the art can design practical update mechanisms based on the clearly defined core ideas of this invention.

[0094] The distribution layer is typically located on the primary storage node and can be situated in the same location as the encoding layer. The distribution layer is responsible for sending the encoded blocks generated from the encoded data object to the storage devices (minimum failure domain, meaning each storage device stores at most one encoded block of the data object). n storage devices are distributed across r racks (n ≥ r). This invention primarily considers rack failure domains, ensuring the cluster can still provide normal service even if any rack fails. Under this reliability setting, each of the r racks distributed to each data object contains l (1 ≤ l ≤ m = nk) encoded blocks. Thus, if any rack fails, at most m encoded blocks will be lost, and the complete data object can still be reconstructed by downloading k encoded blocks from all surviving encoded blocks (≥ k) stored in r-1 racks, and then using the decoding algorithm and MDS attributes to rebuild the original object, thereby maintaining data availability.

[0095] As described in this invention, different data objects can correspond to different primary storage nodes. One of the n encoded blocks is stored in the primary storage device (i.e., the storage device of the primary storage node), and the remaining n-1 encoded blocks are stored in the secondary storage device (i.e., the storage device of the secondary storage node). Different data objects can be stored in different racks, and a rack can contain secondary storage nodes, while a primary storage node can contain secondary storage devices. For ease of explanation, in this embodiment, only the case where one storage node stores one encoded block of a data object is considered, i.e., one storage node contains one storage device storing the data object. For the case where one storage node stores multiple encoded blocks of multiple data objects, those skilled in the art can, based on the core concept of this invention, determine which update mode to adopt to minimize cross-rack traffic during data object updates and achieve optimal update efficiency.

[0096] The storage layer resides on each storage node and is responsible for receiving encoded blocks from the primary storage node (one encoded block is stored locally on the primary storage node) and saving the received encoded blocks to the local storage device, i.e., the object storage device (OSD). Besides basic encoded block read / write operations, the storage layer can also perform data operations, such as reading historical parity blocks (i.e., old parity blocks) and performing XOR operations with incremental parity blocks (when using a finite field GF(2)). w (When) the new check block overwrites the historical check block.

[0097] The data update mechanism of this invention primarily makes decisions at the encoding and distribution layers, and is applicable to two distribution strategies for n encoded blocks of a data object: Flat Architecture Displacement (FAD) and Hierarchical Architecture Displacement (HAD). When the distribution strategy is Flat Architecture Displacement, the n encoded blocks formed after encoding for each data object are stored in storage devices on different storage nodes of n different racks, i.e., one encoded block is stored per rack. When the distribution strategy is Hierarchical Architecture Displacement, the n encoded blocks formed after encoding for the data object are distributed and stored in r < n different racks, with each rack storing no more than m = nk encoded blocks.

[0098] The following section, based on the structure of this system, describes the encoding method and data update method for each data object in this system:

[0099] In the encoding method, the encoding layer in the main storage phase of the data object is responsible for the encoding process. The encoding algorithms used in distributed storage systems mainly include traditional erasure codes (such as RS coding) and regenerating codes. Regenerating codes are a type of network coding used in distributed storage. RS(n,k) coding divides the data object into k equal-length data blocks. If the length of the data object is less than the length of k data blocks, the last data block is padded with zeros. m = nk linear operations are performed on the k data blocks, including addition and scalar multiplication, to obtain m parity blocks. Therefore, any parity block is obtained by linear operations on the k data blocks. To maintain the consistency between the data blocks and the parity blocks (i.e., the linear operation relationship), each time any data block is updated, these m parity blocks must be modified simultaneously. Therefore, the data block is re-encoded to generate a new parity block.

[0100] like Figure 2 The example shown illustrates the execution process of RS(n=9, k=6) encoding, demonstrating the writing process of the data object corresponding to RS(n=9, k=6). The primary storage node 220 receives the data object 210 from the client node 202, divides it into six equal-length data blocks 210.0–210.5, and performs three linear combination operations on these blocks to obtain three parity blocks 210.6–210.8. This set of six data blocks and three parity blocks constitutes a stripe 230. To mitigate system failures, the distributed storage system divides the system into different failure domains, with the smallest failure domain typically being a single storage device (such as a disk). The nine encoded blocks in stripe 230 are stored in different storage devices 220–228, with storage device 220 being the primary storage device for this data object, and storage devices 221–228 being auxiliary storage devices. For a single stripe, the device storing the data blocks is the data device, and the device storing the parity blocks is the parity device. Accordingly, the storage node where the data device resides is called the data node, and the storage node where the verification device resides is called the verification node. In this invention, for ease of description, it is assumed that a single coded block is the basic unit for reading and writing. For cases involving several incomplete data blocks during updates, the basic principles of this invention can be applied, namely, selecting an update strategy method based on the principle of minimizing the amount of data transferred across racks. It should be noted that each data object can have different primary storage nodes, and each storage device can simultaneously store data blocks and verification blocks of different stripes to achieve load balancing for data access and storage.

[0101] RS(n,k) erasure coding has the highest storage efficiency, but when repairing a single coded block, k coded blocks need to be read and transmitted to the repair node. The amount of data transmitted during coded block repair is called the repair bandwidth. The total amount of data read during RS erasure coding repair is equal to the total length of the original data object, which is k times the length of a single faulty coded block. Figure 2 As shown, for RS(9,6) encoding, when any data block is lost, such as data block 210.0, six encoded blocks need to be downloaded from six different storage devices, such as encoded blocks 210.1 to 210.6, and then the lost data block 210.0 is generated through decoding operations. The amount of data to be read during repair is k times the amount of data to be repaired. To alleviate the above problem, technicians proposed regenerating codes, the principle of which is to further break down the encoded blocks in RS (called sub-packetization). Each storage node does not store a single block, but stores multiple subblocks, also called symbols. At this time, the basic unit of encoding changes from block to symbol. When data is lost in any storage node, the amount of data to be read for repair is much smaller than that of RS erasure codes, that is, the repair bandwidth is smaller than the length of the original object. The data update mechanism of this invention is not only applicable to RS erasure codes, but also to regenerating codes and other storage codes based on linear operations.

[0102] Unlike erasure coding, where each storage device stores a single encoded block of a data object, regenerating codes are implemented by storing several subblocks (Symbols or Subblocks) on each storage device. These subblocks are the basic units for reading, writing, and encoding. Each data object is encoded and distributed across n storage devices, with k data devices storing the raw data block and m = nk parity devices storing the parity block. This invention only describes systematic codes, where the unencoded raw data is stored across k data devices. This invention uses a single stripe as an example, where each stripe is encoded independently; the same mechanism can be applied to multiple stripes.

[0103] like Figure 3 As shown, Figure 3 This paper demonstrates an encoding process for Exact Minimum Storage Regenerating (EMSR) code, which is a regenerating code implementation. For EMSR(n,k,r) encoding, each stripe contains n·r sub-blocks of symbols, stored across n storage devices, meaning each storage device stores r symbols. EMSR(n,k,r) encoding is also (n,k)-MDS encoding and, like RS, is a linear encoding. Figure 3This is an implementation of EMSR(6, 3, 3). Client node 302 sends a data object / segment 312 to the storage device 320 of the primary storage node. The primary storage node divides the data into three data blocks 312.0–312.2, encodes these three data blocks to generate three parity blocks 312.3–321.5, and the data blocks and parity blocks form a stripe 332. During encoding, each data block is further divided into three Symbols. Each generated encoded block 312.x contains three Symbols 312.xy, where x = 0, 1, 2, ..., 5, and y = 0, 1, and 2, respectively stored in storage device 32z, where z = 0, 1, ..., 5. An update to any Symbol will cause changes to some or all of the parity Symbols. For example... Figure 3 As shown, Symbol(312.xy(x=0,1,2,...,5,y=0,1,2)) is used as the basic unit for data updates. Based on the position of the updated Symbol and the total number of Symbols, the following update mechanism selection algorithm is used to select the optimal update mechanism.

[0104] The cross-rack traffic-aware update mechanism proposed in this invention is applicable to storage codes based on linear operations, such as RS erasure codes and regenerable codes. Below, this invention uses RS coding as an example to explain the update mechanism selection method for various embodiments, but this method is also applicable to various regenerable codes and locally recoverable codes (LRCs) based on linear operations.

[0105] like Figure 4 As shown, in the data update method, in step S401, the primary storage node first obtains the update data of the object access request sent by the client node and the encoding storage mapping table of the data object corresponding to the update data sent by the management node. The update data includes the update position and update length. Specifically, the client node receives the data access request for the data object, i.e., the update request for the data object, converts the data access request into an object access request, queries the primary management node to find the primary storage node of the data object, and sends the object access request to the primary storage node. After receiving the object access request, the primary storage node obtains the encoding storage mapping table of the data object from the management node.

[0106] S402, the primary storage node then determines the storage architecture of the data object based on the storage locations of the n encoded blocks of the data object in the encoded storage mapping table. Here, the n encoded blocks include k historical data blocks and m historical check blocks, where n, k, and m are all integers not less than 1, and n = k + m. Specifically, the primary storage node can obtain the storage location of each encoded block of the data object from the encoded storage mapping table, i.e., the storage device to which each encoded block belongs, the storage node to which the storage device belongs, and the rack to which the storage node belongs. The storage architecture of the data object is determined based on the storage location of each encoded block of the data object.

[0107] S403, if the storage architecture is a flat architecture, then perform a flat architecture update process on the data object according to the updated data, where the flat architecture is an architecture in which the storage locations of n coded blocks are located in different racks.

[0108] S404 If the storage architecture is a hierarchical architecture, then perform a hierarchical architecture update process on the data object according to the updated data, wherein a hierarchical architecture is an architecture in which at least two coded blocks are stored in the same rack.

[0109] Next, the selection process of the data update method of the present invention will be described in detail:

[0110] The data update method in this embodiment of the invention adopts two modes: the Delta-Based Approach (DBA mode) and the Reconstruct Writes (RCW mode).

[0111] Reconstruct-Writes (RCW) mode: First, the written but not updated data (i.e., old data in the non-updated range) is read. The data to be updated (newly written data) is then merged with the old data to form a new data object. After re-encoding the new data object, the data block of the data to be updated and all check blocks are sent to the corresponding storage nodes. Each storage node then replaces the original data block and check block in the corresponding storage device with the data block of the data to be updated and all check blocks.

[0112] Delta-Based-Approach (DBA) update mode: First, read the old data corresponding to the location of the data to be updated, calculate the change between the new and old data blocks, which is the increment. In Galois GF(2w), this can be implemented using XOR operation. Then, encode the increment to generate the parity block increment (ΔP). Finally, send the new data block and the parity block increment to the corresponding storage node. Each storage node replaces the old data in the corresponding storage device with the data block to be updated, and performs an increment operation between the parity block increment and the old parity block (P) in the storage device (which can be implemented using XOR operation in Galois GF(2w), i.e., ...). ), and the result of the calculation ( That is, the new parity block replaces the original parity block (P) in the corresponding storage device.

[0113] Step S403: If the storage architecture is a flat architecture, then perform a flat architecture update process on the data object based on the updated data. Specifically:

[0114] Retrieve the data blocks to be updated located in the auxiliary rack corresponding to the updated data, and the unupdated data blocks of the data object located in the auxiliary rack. If the number of data blocks to be updated in the auxiliary rack is less than the number of unupdated data blocks in the auxiliary rack, then enter the Delta Based Approach (DBA) mode. Otherwise, enter the Reconstruct Writes (RCW) mode.

[0115] In this embodiment of the invention, it is assumed that each read / write basic unit resides in a single storage device, and the updated data range is an integer multiple of the read / write basic unit. For a planar architecture, each read / write basic unit resides in a different rack. For erasure coding, the read / write basic unit is equal to the coding basic unit, typically of fixed length, such as 4K, 8K, 16K, 64K, etc. For RS erasure coding, the coding basic unit is a data block; for regenerated coding, the coding basic unit is a sub-block (Symbol / Subblock / Subpacket).

[0116] When the total number of data blocks to be updated in the auxiliary rack equals the total number of data blocks not yet updated, the update mechanism selection strategy prioritizes RCW mode. This is because RCW mode avoids the local reading process of multiple old check blocks when updating the check node of a data object, and instead directly overwrites the old check blocks with the new check blocks.

[0117] Step S404: If the storage architecture is a hierarchical architecture, then perform a hierarchical architecture update process on the data object based on the updated data. Specifically:

[0118] Retrieve the data blocks to be updated located in the auxiliary rack corresponding to the updated data, and the unupdated data blocks of the data object located in the auxiliary rack. If the number of data blocks to be updated in the auxiliary rack is less than the number of unupdated data blocks in the auxiliary rack, enter DBA mode. If the number of data blocks to be updated in the auxiliary rack is greater than the number of unupdated data blocks in the auxiliary rack, enter RCW mode.

[0119] If the number of data blocks to be updated in the secondary rack is equal to the number of data blocks not yet updated in the secondary rack, then the data blocks to be updated and data blocks not yet updated in the secondary storage nodes of the primary rack are retrieved. Within the primary rack, if the number of data blocks to be updated in the secondary storage nodes is less than the number of data blocks not yet updated in the secondary storage nodes, then DBA mode is entered; if the number of data blocks to be updated in the secondary storage nodes is not less than the number of data blocks not yet updated in the secondary storage nodes, then RCW mode is entered.

[0120] When selecting an update strategy for a hierarchical architecture, the RCW method is preferred when the number of data blocks to be updated in the auxiliary rack is equal to the number of unupdated data blocks in the auxiliary rack, and when the number of data blocks to be updated in the auxiliary nodes of the main rack is equal to the number of unupdated data blocks in the auxiliary nodes of the main rack. This is because the reading and operation (such as XOR) process of the parity blocks is reduced in the auxiliary nodes of the auxiliary rack where the parity blocks to be updated are located, resulting in higher update efficiency.

[0121] In RCW mode, t new data blocks of updated data and kt unupdated data blocks of the data object are read, where k ≥ t ≥ 1. The t new data blocks and kt unupdated data blocks are reconstructed to obtain the reconstructed data object. The reconstructed data object is encoded to generate m new check blocks, and these m new check blocks are sent to the check nodes where each historical check block of the data object is stored, so that each check node replaces its historical check block with a new check block. The t new data blocks are then sent to the storage nodes where the historical data blocks corresponding to each new data block are stored, so that each storage node replaces its corresponding historical data block with a new data block. It should be noted that, in this embodiment of the invention, the new data block of the updated data is the data block to be updated.

[0122] In DBA mode, t new data blocks for updating data and the corresponding historical data blocks for each of the t new data blocks are read. Each new data block is incrementally processed with its corresponding historical data block to generate an incremental data block, resulting in t incremental data blocks. Based on the length of the data object, the t incremental data blocks and kt empty data blocks are combined to obtain a combined data object. The combined data object is encoded to generate m incremental check blocks. These m incremental check blocks are then sent to the check nodes where the historical check blocks of the data object are stored, respectively. Each check node performs an incremental operation on the incremental check blocks and the historical check blocks to obtain the target check block, which replaces the historical check block. Each new data block is sent to the corresponding storage node where the historical data blocks are stored, so that each storage node replaces its corresponding historical data block with the new data block.

[0123] Specifically, to tolerate rack-level failures, this RS erasure coding storage system can choose a flat architecture distribution strategy when distributing data. This means that each coded block of a data object is placed in a different rack. If any rack fails, the data stored in the failed rack can be repaired by accessing data in any k racks. It's important to note that the update mechanism selection in this case only depends on the cross-rack data volume of the updated data. In the flat architecture, the n coded blocks of each stripe are distributed across different storage devices on different storage nodes in n different racks, and communication between the primary storage node and the secondary storage nodes is cross-rack transmission.

[0124] Based on the principle of step S403, such as Figure 5 As shown, Figure 5 This is a schematic diagram illustrating the execution of RCW mode under a planar architecture. The data object includes 6 data blocks (i.e., historical data blocks) 510-515 and 3 check blocks (i.e., historical check blocks) 516-518. The primary storage node 530 of the main rack 501 receives 3 new data blocks 521-523 of the updated data sent by the client 500. Since the number of unupdated data blocks 514 and 515 in the auxiliary rack is less than the number of data blocks 521, 522, and 523 to be updated in the auxiliary rack, RCW mode is selected according to the update strategy selection algorithm of the planar architecture.

[0125] In RCW mode, the primary storage node 530 of main rack 501 first reads the unupdated data blocks 510, 514, and 515, and reconstructs and merges them with the new data blocks 521, 522, and 523 to form a reconstructed data object. The reconstructed data object is then re-encoded to generate new parity blocks 526, 527, and 528. The new data blocks 521, 522, and 523, along with the new parity blocks 526, 527, and 528, are then sent to the corresponding storage nodes. For example, new data block 521 is sent to storage node 531 of auxiliary rack 502, new parity block 526 is sent to storage node 536 of auxiliary rack 507, and so on. After receiving a new encoded block, each storage node replaces historical data blocks 511, 512, and 513 with new data blocks 521, 522, and 523, respectively. Similarly, each check node replaces historical check blocks 516, 517, and 518 with new check blocks 526, 527, and 528, respectively. For example, storage node 531 in auxiliary rack 502 replaces historical data block 511 with new data block 521, storage node 536 in auxiliary rack 507 replaces historical check block 516 with new check block 526, and so on.

[0126] Based on the principle of step S403, such as Figure 6 As shown, Figure 6 This is a schematic diagram illustrating the execution of DBA mode in a flat architecture. The data object includes 6 data blocks (i.e., historical data blocks) 610-615 and 3 check blocks (i.e., historical check blocks) 616-618. The primary storage node 630 of the main rack 601 receives two new data blocks 622-623 from the client 600, representing the updated data. Since the number of unupdated data blocks 611, 614, and 615 in the auxiliary rack is greater than the number of data blocks 622 and 623 to be updated in the auxiliary rack, DBA mode is selected.

[0127] In DBA mode, the primary storage node 630 of main rack 601 reads historical data blocks 612 and 613 corresponding to the data block to be updated from storage nodes 632 of auxiliary rack 603 and 633 of auxiliary rack 604, respectively. Then, an incremental operation (e.g., finite field GF(2)) is performed between each new data block and its corresponding historical data block. w In the original text, the XOR operation is used to obtain incremental data blocks. For example, performing an incremental operation on new data block 622 and historical data block 612 yields incremental data block 642, and then incremental data blocks 642 and 643. These two incremental data blocks are then encoded. During the encoding process, the increment of unupdated data blocks is considered 0, and the operation is a null operation. Figure 6As shown, the four 000 blocks are zero-filled blocks, i.e., four empty data blocks. Incremental data blocks 642 and 643 are combined with the four empty data blocks to obtain the combined data object. The combined data object is encoded to generate three incremental check blocks 646-648. These incremental check blocks are sent to check nodes 636-638, where the corresponding historical check blocks reside. For example, incremental check block 646 is sent to storage node 636 in auxiliary rack 607, and so on. Each check node reads historical check blocks 616-618 from the storage device and performs incremental operations (such as XOR) with the incremental check blocks 646-648 to generate target check blocks 626-628. The target check blocks 626-628 are then used to overwrite and replace the historical check blocks 616-618. For example, storage node 636 in auxiliary rack 607 performs incremental calculations on incremental parity block 646 and historical parity block 616 to obtain target parity block 626, and then replaces historical parity block 616 with target parity block 626, and so on. Furthermore, primary storage node 630 sends each new data block to the storage node where the corresponding historical data block is stored, and the storage node where the historical data block is stored replaces the corresponding historical data block with each new data block. For example, primary storage node 630 sends new data block 622 to storage node 632 where historical data block 612 is stored, and storage node 632 replaces historical data block 612 with new data block 622, and so on.

[0128] To reduce cross-rack communication, multiple encoded blocks of a data object are placed in the same rack, with no more than m = nk encoded blocks per rack. This ensures that if any single rack fails, the encoded blocks stored in the failed rack can be correctly repaired. This data distribution is called a hierarchical architecture distribution, where n encoded blocks located in a stripe of the hierarchical architecture are placed in storage devices with r < n racks. For example... Figure 7 As shown, Figure 7 The flowchart illustrates the algorithm selection process for the hierarchical architecture update process, specifically the cross-rack traffic-aware update mechanism. The specific execution process is as follows:

[0129] S4041, obtain the data block to be updated located in the auxiliary rack corresponding to the updated data and the unupdated data block of the data object located in the auxiliary rack.

[0130] S4042, If the number of data blocks to be updated located in the auxiliary rack is less than the number of data blocks not yet updated located in the auxiliary rack, then enter DBA mode.

[0131] S4043, If the number of data blocks to be updated located in the auxiliary rack is greater than the number of data blocks not yet updated located in the auxiliary rack, then enter RCW mode.

[0132] S4044: If the number of data blocks to be updated located in the auxiliary rack is equal to the number of data blocks not yet updated located in the auxiliary rack, then retrieve the data blocks to be updated and data blocks not yet updated located in the auxiliary storage nodes of the main rack. Within the main rack, if the number of data blocks to be updated located in the auxiliary storage nodes is less than the number of data blocks not yet updated located in the auxiliary storage nodes, then enter DBA mode; if the number of data blocks to be updated located in the auxiliary storage nodes is not less than the number of data blocks not yet updated located in the auxiliary storage nodes, then enter RCW mode.

[0133] Figure 7 The core idea behind the update process of the illustrated hierarchical architecture is to reduce cross-rack or (within-rack) cross-node network communication during updates. Update methods with lower cross-rack read volume are prioritized. When cross-rack read volume is equal, update methods with lower cross-node communication volume are prioritized. When cross-rack and intra-rack cross-node communication volume are equal, the RCW (Reference Check Wrapper) method is preferred because this mode eliminates the reading of old check data when updating check nodes; each check node directly overwrites the old check block with the new check block.

[0134] like Figure 8 and Figure 9 The diagram shown illustrates two RCW modes corresponding to the hierarchical architecture.

[0135] like Figure 8 As shown, Figure 8 This is an example diagram illustrating the execution of RCW mode under a hierarchical architecture. The data object includes six data blocks (i.e., historical data blocks) 820-825 and three check blocks (i.e., historical check blocks) 826-828. The primary storage node 810 of the main rack 804 receives three new data blocks 832-834 from the client node 802. The historical data blocks corresponding to the new data blocks 832-834 are 822-824, where the data block 822 to be updated is located in the main rack, and the data blocks 823 and 824 to be updated are located in the auxiliary rack. The number of data blocks 823 and 824 to be updated in the auxiliary rack is greater than the number of unupdated data blocks 825 in the auxiliary rack. Based on the aforementioned... Figure 7 The update process shown should select RCW mode, which reads fewer old data blocks across racks.

[0136] RCW mode execution process: The primary storage node 810 reads historical data blocks 820, 821, and 825 from the primary storage node 810 (i.e., its own node) on the main rack 804, the auxiliary node 811 on the main rack 804, and the auxiliary node 815 on the auxiliary rack 806. The new data blocks 832–834 are reconstructed with the historical data blocks 820, 821, and 825 to obtain the reconstructed data object 850. The reconstructed data object 850 is re-encoded to generate new parity blocks 836–838. The primary storage node 810 sends the new parity blocks 836–838 to the corresponding parity nodes 816–818, replacing the historical parity blocks 826–828 in each parity node. Simultaneously, the primary storage node 810 sends the new data blocks 832–834 to the corresponding storage nodes 812–814, replacing the historical data blocks 822–824 in each storage node. For example, primary storage node 810 sends a new data block 832 to storage node 812 in main rack 804, and storage node 812 replaces the historical data block 822 with the new data block 832. Primary storage node 810 then sends a new parity block 836 to storage node 816 in auxiliary rack 808, and storage node 816 replaces the historical data block 826 with the new parity block 836. And so on.

[0137] like Figure 9 As shown, Figure 9 This is another execution example diagram for the RCW mode under a hierarchical architecture. The data object includes 6 data blocks (i.e., historical data blocks) 920-925 and 3 check blocks (i.e., historical check blocks) 926-928. Client node 902 sends new data blocks 933 and 934 to the main storage node 910 of the main rack 904. The historical data blocks (i.e., data blocks to be updated) corresponding to the new data blocks 933 and 934 are 923 and 924. Both 923 and 924 are located in the auxiliary rack. It can be seen that the number of data blocks 923 and 924 to be updated in the auxiliary rack is greater than the number of data blocks 925 that have not been updated in the auxiliary rack. According to Figure 7 The update process shown uses RCW mode, which reads fewer old data blocks across racks. If the traditional update method is used (without considering cross-rack bandwidth), the number of updated data blocks is 2, less than half of the total number of data blocks (6). This is considered a small-scale data update, and in this case, DBA mode should be used. However, according to the data update mechanism of this invention, cross-rack bandwidth is a scarce resource, and using RCW mode offers higher update performance, i.e., lower update latency.

[0138] RCW mode execution process: The primary storage node 910 reads historical data blocks 920, 921, 922, and 925 from the primary storage node 910 (its own node) on the main rack 904, auxiliary nodes 911 and 912 on the main rack 904, and auxiliary node 915 on the auxiliary rack 906. The new data blocks 933-934 are reconstructed with the historical data blocks 920, 921, 922, and 925 to obtain the reconstructed data object 950. The reconstructed data object 950 is re-encoded to generate new parity blocks 936-938. The primary storage node 910 sends the new parity blocks 936-938 to the corresponding parity nodes 916-918, replacing the historical parity blocks 926-928 in each parity node. Simultaneously, the primary storage node 910 sends the new data blocks 933 and 934 to the corresponding storage nodes 913 and 914, replacing the historical data blocks 923 and 924 in the storage nodes. For example, primary storage node 910 sends a new data block 933 to storage node 913 in auxiliary rack 906, and storage node 913 replaces the historical data block 923 with the new data block 933. Primary storage node 910 then sends a new parity block 936 to storage node 916 in auxiliary rack 908, and storage node 916 replaces the historical data block 926 with the new parity block 936. And so on.

[0139] like Figure 10 and Figure 11 The diagram shown illustrates two DBA modes corresponding to a hierarchical architecture.

[0140] like Figure 10 As shown, Figure 10 This is an example diagram illustrating the execution of DBA mode in a hierarchical architecture. The data object includes 6 data blocks (i.e., historical data blocks) 1020-1025 and 3 check blocks (i.e., historical check blocks) 1026-1028. Client 1002 sends two new data blocks 1032 and 1033 to the primary storage node 1010 of main rack 1004. The corresponding coded blocks 1032 and 1033 to be updated for the new data blocks are located at secondary node 1012 of main rack 1004 and secondary storage node 1013 of secondary rack 1006, respectively. Figure 7 During the update process, the number of data blocks 1033 to be updated in auxiliary rack 1006 is less than the number of data blocks 1024 and 1025 that have not been updated in auxiliary rack 1006. Based on the mechanism of prioritizing the reduction of cross-rack read volume, DBA mode is selected.

[0141] DBA mode execution process: The primary storage node 1010 reads historical data blocks 1022 and 1023 from auxiliary nodes 1012 (main rack 1004) and 1013 (auxiliary rack 1006), respectively. Then, each new data block is incremented (e.g., using an XOR operation) with its corresponding historical data block to obtain incremental data blocks. For example, incrementing the new data block 1032 with the historical data block 1022 yields incremental data block 1042, which in turn yields incremental data blocks 1042 and 1043. These two incremental data blocks are combined with four empty data blocks 000 (filled with zeros, as there is no change before and after, the increment is 0) to obtain the combined data object 1050. The combined data object 1050 is re-encoded to generate three incremental check blocks 1046–1048, which are then sent to the corresponding check nodes (storage nodes) 1016–1018 of the auxiliary rack 1008. For example, incremental check block 1046 is sent to storage node 1016 of auxiliary rack 1008, and so on. Check nodes 1016-1018 receive incremental check blocks 1046-1048, perform incremental operations with historical check blocks 1026-1028 read locally from the check nodes, generate target check blocks 1036-1038, and overwrite the historical check blocks 1026-1028. For example, storage node 1016 of auxiliary rack 1008 performs incremental operations with incremental check block 1046 and historical check block 1026 to obtain target check block 1036, and then uses target check block 1036 to overwrite historical check block 1026, and so on. Meanwhile, primary storage node 1010 sends new data blocks 1032 and 1033 to auxiliary nodes 1012 of primary rack 1010 and auxiliary nodes 1013 of auxiliary rack 1006, respectively. Finally, auxiliary nodes 1012 and 1013 overwrite historical data blocks 1022 and 1023 with the new data blocks 1032 and 1033, respectively. For example, primary storage node 1010 sends new data block 1032 to storage node 1012 of primary rack 1004, and storage node 1012 overwrites and replaces historical data block 1022 with the new data block 1032, and so on.

[0142] like Figure 11 As shown, Figure 11 This is another execution example diagram for the DBA mode under a hierarchical architecture. The data object includes 6 data blocks (i.e., historical data blocks) 1120-1125 and 3 check blocks (i.e., historical check blocks) 1126-1128. Client 1102 sends 4 new data blocks 1130-1033 to the primary storage node 1110 of the host rack 1104. Intuitively, for traditional update strategies, when the amount of updated data is large, the RCW method is suitable. However, in this embodiment of the invention, the DBA method is more efficient. Figure 11As shown, the data blocks to be updated are 1130 to 1133, of which 1130 to 1132 are located in the main rack 1104, and only 1133 is located in the auxiliary rack 1106. At this time, in the auxiliary rack 1106, the number of data blocks to be updated 1123 is less than the number of unupdated data blocks 1124 and 1125. Although more than half of the data blocks have been updated (4 out of 6 data blocks are being updated), the DBA method is still chosen because cross-rack bandwidth is even more scarce, only a fraction or even less than one-tenth of the bandwidth between racks.

[0143] DBA mode execution process: The primary storage node 1110 reads historical new data 1123 from auxiliary node 1113 located in auxiliary rack 1106, historical data blocks 1121 and 1122 from auxiliary nodes 1111 and 1112 respectively from primary rack 1104, and historical data 1120 from the primary storage node 1110 (its own node). Each new data block is incremented with its corresponding historical data block (e.g., using an XOR operation) to obtain incremental data blocks. For example, incrementing new data block 1130 and historical data block 1120 yields incremental data block 1140, which in turn yields incremental data blocks 1040 to 1143. These four incremental data blocks are combined with two empty data blocks 000 (filled with zeros, as there is no change before and after, the increment is 0) to obtain the combined data object 1150. The combined data object 1150 is re-encoded to generate three incremental check blocks 1146-1148, which are then sent to the corresponding check nodes 1116-1118. Check nodes 1116-1118 receive the check increments 1146-1148 and perform incremental operations (such as XOR operations) with their local historical check blocks 1126-1128 to obtain the target check blocks 1136-1138, which then overwrite and replace the historical check blocks 1126-1128. Simultaneously, the primary storage node 1110 sends new data blocks 1130-1133 to storage nodes 1110-1113. Storage nodes 1110-1113 overwrite the historical data 1120-1123 with the new data blocks 1130-1133, completing the data block update for the storage nodes.

[0144] Traditional data update methods either use Restricted Read Written (RCW) or Data Absorption Module (DBA) approaches. RCW introduces numerous read operations, resulting in low update efficiency, and requires updating the entire parity block. When updating a small number of data blocks, it suffers from write amplification. DBA does not consider variations in data length and is only suitable for small-volume updates. When updating large amounts of data, it exhibits read amplification during writes, meaning reading large amounts of old data. Furthermore, neither RCW nor DBA considers a hierarchical storage cluster architecture. A hierarchical cluster can contain several racks, distributed across several data centers. Each rack can contain several storage nodes, where data transfer between racks and data centers is the primary performance bottleneck. It's important to note that a hierarchical architecture means the cluster is divided into multiple fault domains: data centers, racks, computer nodes, and storage devices. When data centers, racks, computer nodes, or storage devices fail, the relevant data of the data object remains accessible. For example, for erasure coding (n,k), if any k of the n storage devices are available, the data can be correctly recovered, allowing data object read requests to be executed normally.

[0145] Compared to traditional data update methods, the data update method of this invention, based on the structure of the erasure coding storage system, solves the problems of traditional data update methods, namely, the write amplification problem and the low update efficiency of the erasure coding storage system caused by reduced storage system update performance. The data update method of this invention determines whether to use RCW or DBA mode to update data based on the update location of the data object, the length of the updated data, and the distribution of the storage nodes corresponding to the update interval within the rack. This minimizes cross-rack data transmission during updates (cross-rack bandwidth is a scarce resource, generally a fraction of the intra-rack bandwidth, and in extreme cases, a fraction of a fraction), thus improving update efficiency. When several racks are distributed across multiple data centers across a wide area network (WAN), this invention also reduces cross-data center data transmission bandwidth, thereby improving the data update performance of the WAN-deployed erasure coding storage cluster. This invention can be applied to distributed file systems, object storage (such as Amazon S3 or OpenStack Swift), block storage, etc.

[0146] Example 2

[0147] Based on the same inventive concept, the second embodiment of the present invention also provides a data update method for an erasure coding storage system, such as... Figure 4 As shown, applied to the main storage node of the erasure coding storage system as described in Embodiment 1, the method includes:

[0148] S401, obtain the updated data of the object access request sent by the client node and the encoding storage mapping table of the data object corresponding to the updated data sent by the management node;

[0149] S402, determine the storage architecture of the data object based on the storage locations of the n encoded blocks of the data object in the encoded storage mapping table, wherein the n encoded blocks include k historical data blocks and m historical check blocks, and n, k and m are all integers not less than 1, n = k + m;

[0150] S403, If the storage architecture is a flat architecture, then according to the updated data, the flat architecture update process is performed on the data object, where the flat architecture is an architecture in which the storage locations of n coded blocks are located in different racks;

[0151] S404 If the storage architecture is a hierarchical architecture, then perform a hierarchical architecture update process on the data object according to the updated data, wherein a hierarchical architecture is an architecture in which at least two coded blocks are stored in the same rack.

[0152] As an optional embodiment, the process of performing a plane architecture update on the data object based on the updated data includes:

[0153] Obtain the data block to be updated located in the auxiliary rack corresponding to the updated data and the data block not updated located in the auxiliary rack of the data object;

[0154] If the number of data blocks to be updated located in the auxiliary rack is less than the number of data blocks not yet updated located in the auxiliary rack, then the incremental update mode (Delta Based Approach, or DBA mode for short) is entered.

[0155] Otherwise, it enters Reconstruct Writes (RCW mode).

[0156] As an optional embodiment, the step of performing a hierarchical architecture update process on the data object based on the updated data includes:

[0157] Obtain the data block to be updated located in the auxiliary rack corresponding to the updated data and the data block not updated located in the auxiliary rack of the data object;

[0158] If the number of data blocks to be updated located in the auxiliary rack is less than the number of data blocks not yet updated located in the auxiliary rack, then enter DBA mode;

[0159] If the number of data blocks to be updated located in the auxiliary rack is greater than the number of data blocks not yet updated located in the auxiliary rack, then enter RCW mode.

[0160] As an optional embodiment, after obtaining the data block to be updated located in the auxiliary rack corresponding to the updated data and the unupdated data block of the data object located in the auxiliary rack, the method further includes:

[0161] If the number of data blocks to be updated located in the auxiliary rack is equal to the number of data blocks not updated located in the auxiliary rack, then obtain the data blocks to be updated and the data blocks not updated located in the auxiliary storage node in the main rack.

[0162] In the main rack, if the number of data blocks to be updated on the secondary storage node is less than the number of data blocks not yet updated on the secondary storage node, the system enters the DBA mode; if the number of data blocks to be updated on the secondary storage node is not less than the number of data blocks not yet updated on the secondary storage node, the system enters the RCW mode.

[0163] As an optional embodiment, the RCW mode further includes:

[0164] Read t new data blocks of the updated data and kt unupdated data blocks of the data object, where k≥t≥1;

[0165] The t new data blocks and the kt unupdated data blocks are reconstructed to obtain the reconstructed data object;

[0166] The reconstructed data object is encoded to generate m new check blocks, and the m new check blocks are respectively sent to the check nodes where the historical check blocks of the data object are stored, so that each check node replaces the historical check block with the new check block.

[0167] The t new data blocks are sent to the storage node where the historical data block corresponding to each new data block is stored, so that each storage node replaces the corresponding historical data block with the new data block.

[0168] As an optional embodiment, the DBA mode further includes:

[0169] Read t new data blocks of the updated data and the historical data blocks corresponding to each of the t new data blocks;

[0170] Each new data block is incrementally compared with its corresponding historical data block to generate an incremental data block, thus obtaining t incremental data blocks;

[0171] Based on the length of the data object, the t incremental data blocks and kt empty data blocks are combined to obtain the combined data object;

[0172] The combined data object is encoded to generate m incremental verification blocks. The m incremental verification blocks are then sent to the verification nodes where the historical verification blocks of the data object are stored, so that each verification node performs incremental calculations on the incremental verification blocks and the historical verification blocks to obtain the target verification block and replaces the historical verification block with the target verification block.

[0173] Each new data block is sent to the storage node where the corresponding historical data block is stored, so that each storage node replaces the corresponding historical data block with the new data block.

[0174] Since the data update method of the erasure coding storage system described in this embodiment is the same as the data update method used in the erasure coding storage system of Embodiment 1 of this application, those skilled in the art can understand the specific implementation and various variations of the data update method of the erasure coding storage system in this embodiment based on the erasure coding storage system described in Embodiment 1 of this application. Therefore, how the data update method of this erasure coding storage system implements the erasure coding storage system of Embodiment 1 of this application will not be described in detail here. As long as those skilled in the art implement the data update method used in the erasure coding storage system of Embodiment 1 of this application, they all fall within the scope of protection of this application.

[0175] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects.

[0176] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0177] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. An erasure coding storage system, characterized in that, include: Multiple data centers, multiple racks in each data center, and multiple computer nodes in each rack; the data centers are interconnected via a network, and computer nodes in the same rack and in different racks are interconnected via network devices; The plurality of computer nodes include several client nodes, several management nodes, and several storage nodes; the plurality of storage nodes corresponding to each data object include one primary storage node and several secondary storage nodes, wherein the primary storage node is a storage node of the data object that meets the set requirements, and the secondary storage nodes are the storage nodes of the data object other than the primary storage node; The primary storage node is used to obtain the updated data of the object access request sent by the client node and the encoding storage mapping table of the data object corresponding to the updated data sent by the management node. Based on the storage locations of the n encoded blocks of the data object in the encoded storage mapping table, the storage architecture of the data object is determined, wherein the n encoded blocks include k historical data blocks and m historical check blocks, where n, k, and m are all integers not less than 1, and n = k + m; If the storage architecture is a planar architecture, then according to the updated data, a planar architecture update process is performed on the data object, wherein the planar architecture is an architecture in which the storage locations of the n coded blocks are located in different racks; If the storage architecture is a hierarchical architecture, then according to the updated data, the hierarchical architecture update process is performed on the data object, wherein the hierarchical architecture is an architecture in which the storage locations of at least two coded blocks are located in the same rack; Each client node is used for: Receive data access requests from remote or local applications, including access requests sent by file systems, object storage, and / or block storage devices; The data access request is converted into an object access request, which is an access request for a fixed-length data object / data segment. Query the main management node for the main storage node of the data object corresponding to the updated data of the object access request, and send the updated data to the main storage node of the corresponding data object; The management node is used for: The management nodes maintain the consistency of cluster information of the system through a consensus algorithm and elect the master management node for each data object; Maintain the status information, location information, and storage device information of each storage device of each storage node; Maintain each data object and the encoding storage mapping table for each data object, wherein the encoding storage mapping table is a list of mapping relationships between each encoding block of the data object and the storage device where each encoding block is stored; Respond to the client node's request to obtain the primary storage node for each data object; Respond to the request from the primary storage node of each data object to obtain the encoded storage mapping table of each data object.

2. The system as described in claim 1, characterized in that, The multiple auxiliary nodes of each data object include: multiple data nodes and multiple verification nodes, wherein the data nodes are storage nodes for storing data blocks of the data object, and the verification nodes are storage nodes for storing verification blocks of the data object; The rack where the primary storage node is located is the main rack, and the rack where the secondary storage node is located is the secondary rack; Each storage node includes a processor, memory, a network interface card (NIC) connected to the processor, and several storage devices.

3. A data update method for an erasure coding storage system, characterized in that, Applied to the main storage node of the erasure coding storage system as described in any one of claims 1-2, the method includes: Obtain the updated data of the object access request sent by the client node and the encoding storage mapping table of the data object corresponding to the updated data sent by the management node; Based on the storage locations of the n encoded blocks of the data object in the encoded storage mapping table, the storage architecture of the data object is determined, wherein the n encoded blocks include k historical data blocks and m historical check blocks, where n, k, and m are all integers not less than 1, and n = k + m; If the storage architecture is a planar architecture, then according to the updated data, a planar architecture update process is performed on the data object, wherein the planar architecture is an architecture in which the storage locations of the n coded blocks are located in different racks; If the storage architecture is a hierarchical architecture, then a hierarchical architecture update process is performed on the data object according to the updated data, wherein the hierarchical architecture is an architecture in which at least two coded blocks are stored in the same rack.

4. The method as described in claim 3, characterized in that, The process of performing a planar architecture update on the data object based on the updated data includes: Obtain the data block to be updated located in the auxiliary rack corresponding to the updated data and the data block not updated located in the auxiliary rack of the data object; If the number of data blocks to be updated located in the auxiliary rack is less than the number of data blocks not yet updated located in the auxiliary rack, then the system enters an incremental update mode, i.e., DBA mode. Otherwise, it enters rebuild-write mode, i.e., RCW mode.

5. The method as described in claim 3, characterized in that, The process of performing a hierarchical architecture update on the data object based on the updated data includes: Obtain the data block to be updated located in the auxiliary rack corresponding to the updated data and the data block not updated located in the auxiliary rack of the data object; If the number of data blocks to be updated located in the auxiliary rack is less than the number of data blocks not yet updated located in the auxiliary rack, then enter DBA mode; If the number of data blocks to be updated located in the auxiliary rack is greater than the number of data blocks not yet updated located in the auxiliary rack, then enter RCW mode.

6. The method as described in claim 5, characterized in that, After obtaining the data block to be updated located in the auxiliary rack corresponding to the updated data and the unupdated data block of the data object located in the auxiliary rack, the process further includes: If the number of data blocks to be updated located in the auxiliary rack is equal to the number of data blocks not updated located in the auxiliary rack, then obtain the data blocks to be updated and the data blocks not updated located in the auxiliary storage node in the main rack. In the main rack, if the number of data blocks to be updated on the secondary storage node is less than the number of data blocks not yet updated on the secondary storage node, the system enters the DBA mode; if the number of data blocks to be updated on the secondary storage node is not less than the number of data blocks not yet updated on the secondary storage node, the system enters the RCW mode.

7. The method according to any one of claims 4-6, characterized in that, The RCW mode also includes: Read t new data blocks of the updated data and kt unupdated data blocks of the data object, where k≥t≥1; The t new data blocks and the kt unupdated data blocks are reconstructed to obtain the reconstructed data object; The reconstructed data object is encoded to generate m new check blocks, and the m new check blocks are respectively sent to the check nodes where the historical check blocks of the data object are stored, so that each check node replaces the historical check block with the new check block. The t new data blocks are sent to the storage node where the historical data block corresponding to each new data block is stored, so that each storage node replaces the corresponding historical data block with the new data block.

8. The method according to any one of claims 4-6, characterized in that, The DBA mode also includes: Read t new data blocks of the updated data and the historical data blocks corresponding to each of the t new data blocks; Each new data block is incrementally compared with its corresponding historical data block to generate an incremental data block, thus obtaining t incremental data blocks; Based on the length of the data object, the t incremental data blocks and kt empty data blocks are combined to obtain the combined data object; The combined data object is encoded to generate m incremental verification blocks. The m incremental verification blocks are then sent to the verification nodes where the historical verification blocks of the data object are stored, so that each verification node performs incremental calculations on the incremental verification blocks and the historical verification blocks to obtain the target verification block and replaces the historical verification block with the target verification block. Each new data block is sent to the storage node where the corresponding historical data block is stored, so that each storage node replaces the corresponding historical data block with the new data block.

Images