Simplified RAID implementation for byte-addressable memory

Abstract

The present disclosure provides a simplified RAID implementation of a byte-addressable memory. One aspect of the present disclosure may provide a storage system. The storage system may include multiple byte-addressable storage devices and multiple media controllers. The corresponding byte-addressable storage devices are used to store parity blocks or data blocks of data stripes, and the corresponding media controllers are coupled to the corresponding byte-addressable storage devices. Each media controller may include a tracker logic block to serialize critical sections of multiple media access sequences associated with addresses on the corresponding byte-addressable storage devices. Each media access sequence includes one or more read and / or write operations, and the data stripes may be inconsistent during the critical section of the media access sequence.

Description

Background Art

[0001] The present disclosure generally relates to redundant array of independent disks (RAID) technology. More specifically, the present disclosure relates to implementing RAID for byte-addressable memory devices. BRIEF DESCRIPTION OF THE DRAWINGS

[0002] Figure 1 A memory system implementing a redundant array of independent disks (RAID) according to one aspect of the present disclosure is illustrated.

[0003] Figure 2 A RAID group spanning multiple byte-addressable memory devices is illustrated according to one aspect of the present disclosure.

[0004] Figure 3 Illustrated is a block diagram of a RAID-enabled media controller according to one aspect of the present disclosure.

[0005] Figure 4 A write sequence according to one aspect of the present disclosure is illustrated.

[0006] Figure 5 Serialization of requests to a data lane and a parity lane is illustrated according to one aspect of the present disclosure.

[0007] Fig. 6A A read sequence with data errors according to one aspect of the present disclosure is illustrated.

[0008] Figure 6B A read sequence with multiple errors according to one aspect of the present disclosure is illustrated.

[0009] Figure 7 A scenario of serialization of multiple tracker occupancy events on a parity way controller according to one embodiment is illustrated.

[0010] Fig. 8A A write sequence with parity errors according to one aspect of the present disclosure is illustrated.

[0011] Figure 8B A write sequence with data errors according to one aspect of the present disclosure is illustrated.

[0012] Fig. 9 An apparatus for facilitating RAID operations in a byte-addressable memory system according to one aspect of the present disclosure is illustrated.

[0013] In the drawings, like reference numerals refer to like drawing elements. DETAILED DESCRIPTION

[0014] The following description is presented to enable any person skilled in the art to make and use the examples, and is provided in the context of a specific application and its requirements. Various modifications to the disclosed examples will be apparent to those skilled in the art, and the general principles defined herein may be applied to other examples and applications without departing from the spirit and scope of the present disclosure. Therefore, the scope of the present disclosure is not limited to the examples shown, but is intended to conform to the maximum scope consistent with the principles and features disclosed herein.

[0015] The present disclosure provides a simplified implementation of a redundant array of independent disks (RAID) for byte-addressable memory (e.g., storage class memory (SCM)) devices. The disclosed memory system includes a plurality of memory devices, each of which is coupled to a media controller and each of which maintains a RAID group mapping table. The media controllers cooperatively provide RAID functionality at a granularity level suitable for byte-addressable memory (e.g., at a cache line level). Note that although the term RAID is used herein, the memory devices described in the present disclosure are not necessarily associated with one or more disks. The memory devices may include solid-state byte-addressable storage devices. The term "RAID" herein refers generally to the use of redundant arrays of memory devices, which use checksum schemes and protection schemes similar to those used in conventional RAID systems, but are applied to solid-state byte-addressable memory devices. The RAID checksum scheme used by the various examples disclosed herein uses an exclusive-OR (XOR) function, in which the parity path of the RAID stripe stores the bitwise XOR of the values ​​in each stripe data path. The XOR checksum scheme is exemplary only, and other more advanced checksum schemes may also be used. The scope of the present disclosure is not limited by the type of checksum scheme.

[0016] During a write operation that requires reading pre-write data and pre-write parity from the data path and parity path, respectively, the data path media controller and the parity path media controller exchange three messages. More specifically, after receiving a write request, the data path media controller sends a sequence start message to the parity path controller before starting its own read operation, thereby allowing the pre-write data read and the pre-write parity read to be performed simultaneously. When performing a read operation, a tracker in the data path media controller is occupied to delay other accesses to the same access address. In addition, a tracker in the parity path media controller is also occupied to delay access to the parity path on behalf of other data paths. After receiving an ACK for the sequence start message and after completing the reading of the pre-write data and the writing of the post-write data, the data path media controller releases its tracker and sends a sequence completion message to the parity path media controller. The parity path media controller releases its tracker after receiving the sequence completion message. In this way, the tracker on the parity path media controller can serialize access (including read access and write access) to the same stripe address (e.g., a memory block on the same stripe). More specifically, accesses to the same stripe address can be serialized to prevent the risk of race conditions in multiple accesses to different data paths belonging to the same stripe. Compared to existing methods, this solution significantly reduces the complexity of implementation.

[0017] Redundant Array of Independent Disks (RAID) is a well-known data storage virtualization technology for providing data redundancy in storage systems. However, the extended transaction processing time of traditional software-based RAID technology makes it unsuitable for byte-addressable memory devices, such as storage class memory (SCM) devices and dynamic random access memory (DRAM) devices. Existing hardware-based RAID technology designed specifically for byte-addressable memory usually requires complex hardware logic to implement.

[0018] The solution disclosed herein improves upon existing RAID implementations for byte-addressable memory by reducing the complexity of the hardware logic used to implement RAID on the media controller of the byte-addressable memory devices.

[0019] Figure 1 A memory system implementing RAID according to one aspect of the present disclosure is illustrated. Memory system 100 may include a plurality of byte-addressable memory devices (e.g., memory devices 102, 104, 106, and 108), a plurality of RAID-enabled media controllers (e.g., media controllers 112, 114, 116, and 118), a memory semantic structure 110, and one or more processing nodes (e.g., processing nodes 120 and 130).

[0020] The memory devices 102-108 may be byte-addressable memory devices, such as SCM devices or DRAM devices. Each memory device may be coupled to a RAID-enabled media controller. For example, the byte-addressable memory device 102 is coupled to a RAID-enabled media controller 112. Each RAID-enabled media controller may handle media access functions (e.g., handle read requests and write requests) like a conventional media controller. In addition, each RAID-enabled media controller may include logic blocks for performing RAID-specific functions. For example, each RAID-enabled media controller may access a RAID group mapping table that stores information useful for performing RAID-specific functions. Figure 1 , media controllers 112, 114, 116, and 118 access RAID group mapping tables 122, 124, 126, and 128, respectively. Media controllers 112-118 provide RAID functionality at the granularity most frequently addressed by memory devices 102-108 without employing a master RAID controller. In other words, all media controllers are similarly configured peer controllers. For example, the granularity may be a cache line granularity, such as 64 bytes, used by processing nodes 120 and 130 to cache data stored in memory devices 102-108.

[0021] The memory semantic structure 110 is used to interconnect processing nodes and memory devices, thereby allowing the processing node to access the memory device (e.g., send a memory semantic request) regardless of where the memory device is located on the structure. The memory semantic request may include a read request, a write request, an atomic operation request, etc. The memory semantic structure 110 may include an underlying high-speed input / output (I / O) network that can connect the processing node to the physical media, which may be much longer than other possible distances. According to one aspect, the memory semantic structure 110 may include a Gen-Z memory structure. According to further aspects, the memory semantic structure 110 may include one or more switches (e.g., SCM switches) and other components that physically interconnect the media controller with the processing node. The read request and write request issued by the processing node can cause the media controller of the processing node to communicate with the media controller of the memory device through the memory semantic structure 110, so that the media controller can perform read operations and write operations on the attached memory device.

[0022] exist Figure 1In the example shown, there are two processing nodes and four media controllers. In fact, the number of processing nodes can be greater than or less than the number of media controllers. A processing node can be a computing device capable of issuing a read / write request to a memory device. Each processing node may include one or more processors, each of which has one or more processing cores. Examples of processing nodes may include, but are not limited to, CPUs, GPUs, input / output (I / O) devices, accelerators, network interface cards (NICs), and the like. A processing node may include one or more media controllers, which may be integrated within or outside its processor. The processing cores of a processor may each include multiple caches, such as a level 1 (L1) cache and a level 2 (L2) cache, and multiple cores of the same processor may share another cache, such as a level 3 (L3) cache. Each processing node may also access a RAID group mapping table. For example, processing node 120 may access RAID group mapping table 132, and processing node 120 may access RAID group mapping table 134. Note that all RAID group mapping tables store the same information. Details about the RAID group mapping table will be provided later.

[0023] Figure 2 A RAID group across multiple byte-addressable memory devices is illustrated according to one aspect of the present disclosure. Figure 2 In RAID group 200, a plurality of byte-addressable memory devices (eg, memory devices 202-208) are included, each memory device being coupled to its own media controller (eg, media controllers 212-218). Figure 2 Also shown is a RAID stripe 210 distributed across multiple byte-addressable memory devices. In this example, the RAID stripe 210 may include multiple data blocks (e.g., data blocks 222-228) located respectively on multiple memory devices (e.g., memory devices 202-208), wherein one of the data blocks (e.g., block 204) stores parity data and the other data blocks store data. According to one aspect, the size of each data block can be the same as the width of a cache line (e.g., 64 bytes). Note that for each RAID stripe, the memory devices that store user data are referred to as data path devices of the RAID stripe, and the memory devices that store parity data are referred to as parity path devices of the RAID stripe. Similarly, the corresponding media controller may be referred to as a data path media controller or a parity path media controller of the RAID stripe. In Figure 2In the example shown, media controller 214 is the parity path media controller for RAID stripe 210, and all other media controllers are data path media controllers for RAID stripe 210. It should also be noted that parity data may be stored on different memory devices for different stripes. FIG. 2 also shows that parity data for RAID stripe 220 is stored on memory device 202. With respect to RAID stripe 220, memory device 202 is a parity path device, and media controller 212 is a parity path media controller. Figure 2 As shown, a RAID group can store many RAID stripes, and the parity data of different stripes can be spread across multiple memory devices to prevent excessive wear on any particular memory device and balance traffic between multiple memory devices. In other words, for a specific memory access operation (e.g., a read operation or a write operation), depending on the memory address being accessed (e.g., the RAID stripe being accessed), one media controller can be called a parity path media controller, one is called a data path media controller, and the other media controllers are not directly involved unless required for error correction purposes.

[0024] Each media controller can access the RAID group mapping table that stores information associated with the RAID group. Each entry in the RAID group mapping table can describe a RAID group, and each memory access is searched for any access address mapped to the RAID group. The information searched in the table is sufficient to clearly identify the parity path media controller and the data path media controller associated with any given access address in combination with the access address. For example, the table can specify an ordered list of the number of media controllers and their identities. Arithmetic and logical operations acting on a subset of address bits can serve as an interleaving function, thereby determining which controller in the table supply list is used as the data path controller for a given access address and which is used as the parity path controller for a given access address. In one example, the controller can use the access address and information included in the mapping table (which lists all controllers and interleaving schemes) to perform calculations to determine which controller is the parity controller and which controllers are the corresponding data controllers in the stripe.

[0025] Entries in the RAID group mapping table may include identity information associated with the media controllers involved in the RAID group. In addition, the table entries may provide sufficient information to unambiguously determine the parity path / data path media controller used for any given access address. According to one aspect, the table may contain only one entry, and all memory capacity under the control of the media controller is mapped to a single RAID group. According to another aspect, the table may contain multiple entries, and the memory capacity under the control of the media controller may be subdivided or partitioned between multiple RAID groups that each respond to different access address ranges.

[0026] A RAID stripe is considered consistent if all paths of the stripe can be read and the values ​​stored in its parity path devices are a correct checksum of the values ​​stored in its various data path devices. A RAID stripe is inconsistent if all paths of the stripe can be read but the values ​​stored in its parity path devices are not a correct checksum of the values ​​stored in its various data path devices. A RAID stripe is correctable if it was previously consistent but one of its paths resulted in a device failure rendering it unreadable. Since the stripe was consistent prior to the failure, the data associated with the unreadable path can be reconstructed from the surviving paths.

[0027] The RAID system must, whenever possible, leave each stripe in a consistent state after each access, or in a correctable state if a device failure prevents each stripe from reaching a consistent state. The RAID system may leave a stripe in an uncorrectable state after an access only if more than one path is unreadable. This indicates that the data loss exceeds the protection capabilities of the RAID scheme. In no case should the RAID system leave a stripe in an inconsistent state, as this will result in silent data corruption in the event of a subsequent device failure.

[0028] The inherent race condition between the updating of the data and parity values ​​stored in their respective memory devices may result in temporarily inconsistent stripe states during a write sequence. Without further precautions, the failure of another data path device within the same RAID stripe may cause the rebuild sequence to use these temporarily inconsistent stripe values, resulting in corrupted reconstructed data. To avoid this danger, stripe rebuild sequences should be serialized so that they do not occur simultaneously with writes to the same stripe. Since both the rebuild sequence and the write sequence are coordinated by the parity path of the stripe, the parity path media controller may provide a mechanism for enforcing such serialization.

[0029] The portion of the stripe access that must be serialized for any given stripe to avoid the risk of data corruption described above is called a critical region. In the case of write accesses, the critical region is any time when the stripe contents may be temporarily inconsistent (data updated but parity paths not updated, or vice versa). In the case of a rebuild sequence, the critical region begins when the first read is issued to compute the reconstructed data, and ends when the last such read is completed and, if applicable, the reconstructed data has been written back to the appropriate data path devices.

[0030] It should be noted that the same address serialization inherent in a processing node, for example due to its cache architecture, is not sufficient to ensure the required serialization for these critical sections. A processing node that performs such serialization may still issue multiple simultaneous accesses to different data addresses corresponding to different data paths within the same stripe. Furthermore, multiple simultaneous accesses to the same stripe or even to the exact same data address may also occur in a manner that allows multiple processing nodes to communicate via e.g. Figure 1 The present invention relates to a RAID system for accessing a storage array using the memory semantic structure shown. Existing methods for serializing critical sections of multiple simultaneous stripe accesses require complex implementations. For example, these methods may require that all accesses pass through a single controller that performs serialization but introduces performance bottlenecks. Alternatively, these methods may require explicit lock tracking by the media controller and corresponding explicit lock and unlock handshakes using messages between the data path media controller and the parity path media controller or between the processing node and the media controller. In order to ensure the consistency of the stripe or parity in a simpler way, the disclosed solution uses a write sequence and a rebuild sequence so that the tracker on the data path media controller serializes the critical section of the same block access, and the tracker on the parity path media controller serializes the critical section of all other same stripe accesses. This ensures that the critical section of a subsequent write or rebuild operation for the same stripe does not start while the critical section of a previous write or rebuild operation is still in progress.

[0031] Figure 3 A block diagram of a RAID-enabled media controller is illustrated in accordance with one aspect of the present disclosure. Figure 3, a RAID-enabled media controller 300 may include a media controller sub-block 310 and a RAID controller sub-block 320. The media controller sub-block 310 may be similar to a conventional media controller used in a storage system that implements a memory semantic structure. More specifically, the media controller sub-block 310 may include a media interface 312 for interfacing with an attached memory device and a media access logic 314 that manages various aspects of media access operations (e.g., read, write, write buffering, wear leveling, etc.) specific to the media type (e.g., DRAM, flash memory, memristor, etc.) of the attached memory device. The RAID controller sub-block 320 may be responsible for receiving memory semantic requests (e.g., read, write, atomic operation access, etc.) from a request source (e.g., a processing node), performing various RAID-specific functions, and performing reads and writes of data blocks and parity blocks using the media controller sub-block 310.

[0032] RAID controller sub-block 320 may include a fabric interface 322, a controller interface 324, a table lookup logic 326, a RAID group mapping table 328, a tracker logic 330, and a conflict detection logic 332. The various logic blocks in RAID controller sub-block 320 may be implemented as hardware logic that may provide greater efficiency and greater speed for performing various RAID-related functions than software-based solutions. Fabric interface 322 interfaces with a memory semantic structure. More specifically, fabric interface 322 may receive read requests and write requests from processing nodes via the memory semantic structure and send responses to completed requests. In addition, fabric interface 322 may send and receive messages to and from other media controllers to facilitate various RAID-related operations (e.g., requests for data or parity).

[0033] The controller interface 324 interfaces with the media controller sub-block 310 and may be responsible for sending read requests and write requests to the media controller sub-block 310. The controller interface 324 may also receive data or error messages from the media controller sub-block 310.

[0034] Table lookup logic 326 may be responsible for looking up a RAID group mapping table 328 that stores information useful for RAID operations. For example, a RAID group mapping table 328 may store information about the number and identity of peer RAID-enabled media controllers. For each RAID group, a RAID group mapping table 328 may store information identifying all path controllers and allow the identity of parity path controllers as well as data path controllers to be determined based on addresses within the RAID group.

[0035] Tracker logic 330 may be responsible for tracking the status and progress of all requests (read requests or write requests) being processed. Tracker logic 330 may generate a tracker entry for each in-progress request and close or delete the entry after completing all activities associated with the request, including media access, interaction with the memory controller, and sending of responses.

[0036] The conflict detection logic 332 can be responsible for detecting whether a newly received request is for the same address as an ongoing request based on the active tracker entries in the tracker logic 330. Therefore, the conflict detection logic 332 can delay the execution of the newly received request until the previous request is completed. In other words, requests for the same address can be serialized one at a time by the conflict detection logic 332. In the data path, when multiple processing nodes attempt to access the same address (e.g., write to the same address), the conflict detection logic 332 can ensure that the multiple requests will be serialized to prevent stripe consistency race conditions. Similarly, when there are multiple requests for different data paths within the same RAID stripe, access to the corresponding parity paths can be serialized by the conflict detection logic 332 to prevent stripe consistency race conditions.

[0037] Figure 4 A write sequence according to one aspect of the present disclosure is illustrated. For an error-free write sequence, three parties may be involved: a processing node 402 that sends a write request, a data path controller 404 that writes to a corresponding address in the medium, and a parity path controller 406 that updates the parity data. Note that calculating the updated parity data may depend on the values ​​of the pre-write data and the pre-write parity. Both the data path controller 404 and the parity path controller 406 are RAID-enabled media controllers. Note that a controller is referred to as a data path controller or a parity path controller based on whether its corresponding memory device stores user data or parity at the address specified by the write request.

[0038] Additionally, multiple events (e.g., reads, writes, tracker occupancy, etc.) can occur on the same entity, with each event taking a different amount of time. To better understand the timing of different events, Figure 4 Use different modes of progress bars to represent different events. The corresponding relationship between progress bars and different events is shown in Figure 4 4 is shown in the legend field 410. Note that similar reference numerals are used throughout the disclosure.

[0039] During a write operation, processing node 402 sends a write request 412 to data path controller 404. Assume that processing node 402 also stores RAID group mapping information so that processing node 402 can identify which RAID-enabled media controller to send the write request to. In this example, based on the memory address specified in write request 412, processing node 402 sends the request to data path controller 404. After receiving write request 412, data path controller 404 can send an acknowledgement (ACK) response 414 back to processing node 402. The timing of sending ACK response 414 can depend on the actual implementation. Alternatively, data path controller 404 can send ACK response 414 after a later stage of the write operation is completed (e.g., after data path write media event 434 is completed), rather than sending ACK response 414 immediately after receiving write request 412. Figure 4 The earliest time that the datapath controller 404 can issue an ACK 414 is depicted, which is consistent with the assignment of the tracker 420. Tracker occupancy ensures that any subsequent same-address read requests issued after the processing node receives the ACK response 414 will be serialized, at least until the data from the write request 412 has been written to the buffer and is therefore visible to subsequent reads.

[0040] After receiving the write request 412, the tracker logic in the data path controller 404 generates a tracker entry for the address (tracker occupied event 420). In addition, the data path controller 404 sends a sequence start (SeqStart) message 416 to the parity path controller 406 to notify the parity path controller 406 of the start of the write sequence. To this end, the data path controller 404 can look up the RAID group mapping table to determine which peer controller is the parity path controller. The data path controller 404 also reads the data currently stored at the address specified by the write request (read event 418). The data currently stored at the address can be referred to as pre-write data to distinguish it from the data to be written to the address. The data path controller 404 can be configured so that additional access to the address will be delayed for the duration of the tracker occupied event 420 to serialize access to the address.

[0041] After receiving the sequence start message 416, the tracker logic in the parity path controller 406 generates a tracker entry for the parity address (tracker occupied event 426). In addition, the parity path controller 406 sends a sequence confirmation (SeqACK) message 422 to confirm the allocation of the tracker for the write sequence (tracker occupied event 426). The parity path controller 406 reads the current parity data, which is called pre-write parity (read event 424). Similar to the data path controller 404, the parity path controller 406 can be configured so that additional same stripe accesses to the parity path will be delayed until the tracker occupied event 426 is completed to serialize access to the stripe. Note that within a given stripe, the media address of the data and the media address of the parity are the same and can be referred to as the RAID stripe address. Since there are multiple data paths per stripe, there are multiple access addresses associated with a single stripe address. The stripe address is different from the access address (e.g., the memory address used by the processor to issue memory semantic requests). It should also be noted that the stripe address of a data / parity block indicates at which stripe of the medium (ie, memory device) the data / parity block resides. Figure 2 In the example, data / parity blocks 222-228 have the same stripe address. Each access address can be uniquely mapped to a (stripe address, data path) tuple.

[0042] Read event 424 may begin before read event 418 is completed. In fact, the reading of the pre-write data and the reading of the pre-write parity may be performed substantially in parallel (assuming that the transmission time of sequence start message 416 is small), thereby improving write efficiency compared to conventional methods in which the reading of the pre-write parity occurs after the reading of the pre-write data.

[0043] Any media write operation can actually include two events: a write buffer event (when data is written to the write buffer) and a write media event (when data is written to the storage medium). Once the data is written to the write buffer, the data becomes visible to subsequent read operations as data stored at the specified address. Therefore, the write operation can be considered complete, but it may take time to actually write the data to the storage medium. Similar to the data path controller 404, the parity path controller 406 can be configured so that additional same-stripe accesses to the parity path will be delayed until the tracker occupied event 426 is completed, thereby serializing critical sections as needed to avoid data corruption risks.

[0044] Returning to the data path controller 404, after the read of the pre-write data is completed without error, the data path controller 404 can write the data included in the write request 412 to the write buffer (write buffer event 428). Once the write is completed (e.g., the write buffer event 428 is completed successfully) and the sequence confirmation message 422 is received, the tracker logic in the data path controller 404 can close the tracker occupied event 420 (e.g., by releasing the tracker for the address) to allow subsequent access to the address. Note that the tracker on the data path is only released after a successful read and write (e.g., the read event 418 and the write event 428 are completed successfully or without error), and the write buffer event 428 is only started after the sequence confirmation message 422 is received.

[0045] After successfully writing the data to the write buffer, the data path controller 404 may send a sequence completion message 430 along with the data to the parity path controller 406. After receiving the data, the parity path controller 406 updates the parity (e.g., by performing an XOR operation based on the pre-write parity and the received data) and writes the updated parity to the write buffer (write buffer event 432). Once the updated parity is successfully written to the write buffer, the tracker logic in the parity path controller 406 closes the tracker occupied event 426 (e.g., by releasing the tracker at the corresponding address). Note that the tracker on the parity path is only released after receiving the sequence completion message 430 (which indicates the successful completion of the read activity and the write activity on the data path, e.g., events 418 and 428) and the successful completion of the read activity and the write activity on the parity path (e.g., events 424 and 432). Figure 4 Also shown are write media events 434 and 436 on the data path and parity path, respectively. During these events, data and parity are transferred to the actual media. The timing of these events is not very important, because data and parity become visible to subsequent reads after being written into the write buffer.

[0046] like Figure 4As shown, both the data path media write operation and the parity path media write operation associated with a single write request (e.g., request 412) occur entirely during the time that the parity path tracker is occupied (i.e., during tracker occupied event 426). A write buffer event 428 on the data path controller cannot begin until after sequence confirmation message 422 is received, and the parity path tracker remains occupied (i.e., event 426) until write buffer event 432 on the parity path is completed and sequence completion message 430 is received, which in turn indicates completion of write buffer event 428 on the data path. Thus, the critical region of the write sequence where the stripe may become temporarily inconsistent falls entirely within the duration of tracker occupied event 426 on the parity path.

[0047] The value written to the stripe parity path is an XOR function of three data values: pre-write parity, pre-write data, and to-be-written data. Parity path controller 406 obtains the pre-write parity value via read event 424, but relies on messages from data path controller 404 to obtain the other two values. Figure 4 In the example shown, both the pre-write data and the data to be written can be sent to the parity path controller 406 together with the sequence completion message 430. The pre-write data and the data to be written can also be transmitted as two separate values, or in order to save fabric bandwidth, the data path controller 404 can combine them into a single value using an XOR function before sending. As another alternative, the data to be written can be sent together with the sequence start message 416, and the pre-write data can then be sent together with the sequence completion message 430.

[0048] from Figure 4 As can be seen in FIG, the data path tracker is allocated before any media access activity (reading or writing to the buffer) on the data path or the parity path begins (e.g., the tracker occupied event 420 is turned on), and is released only after all media access activities (reading and writing to the buffer) on the data path are completed (e.g., the tracker occupied event 420 is turned off). Similarly, the parity path tracker is allocated before any media access activity (reading or writing to the buffer) on the data path or the parity path begins (i.e., the tracker occupied event 426 is turned on), and is released only after all media access activities (reading or writing to the buffer) on both the parity path and the data path are completed (i.e., the tracker occupied event 426 is turned off). This ensures consistency between the current parity and the current data.

[0049] As discussed previously, the tracker is critical to ensuring stripe consistency because the tracker enforces serialization of accesses to the same address (both data and parity paths). Figure 5 Serialization of data paths and parity paths according to one aspect of the present disclosure is illustrated. Note that Figure 5 The data path controller or parity path controller shown in is for a specific RAID stripe. The same media controller can perform different roles (e.g., as a data path controller or a parity path controller) for different RAID stripes, depending on whether the corresponding memory blocks in the RAID stripe store data or parity. Figure 5 , processing node 502 sends a subsequent write request 542 to controller 504 (which acts as a data path controller for a particular stripe address and will be referred to as data path controller 504 hereinafter). Write request 542 arrives at data path controller 504 while the tracker for a previous write request at the same address is still occupied. Therefore, write request 542 is delayed in processing by data path controller 504 until the tracker is released, as indicated by dashed arrow 544. Once the previous tracker is released, activities in response to write request 542 (e.g., sending sequence start message 546, reading data, starting a new tracker occupation event, etc.) can be performed on data path controller 504. This delay in starting subsequent same-address write accesses ensures that the pre-write data used to calculate the final parity update of the write sequence in response to request 542 is the value written by the previous write, thereby ensuring that consecutive writes are sequentially consistent. The remainder of the write operation can be similar to Figure 4 and will not be discussed in detail here.

[0050] Figure 5 The parity path controller 506 is also shown receiving a subsequent sequence start message 548 for the same stripe address from another data path controller 540 representing a different access address of a different data path mapped to the same stripe, the subsequent sequence start message indicating that the data path controller 540 is performing a write operation. The sequence start message 548 arrives at the parity path controller 506 when the tracker for the previous request for parity data is still occupied. This means that the parity data may still contain pre-write parity values. In this case, processing of the sequence start message 548 is delayed until the tracker for the previous operation is released, as shown by the dashed arrow 550. Once the previous tracker is released, activities in response to the sequence start message 548 (e.g., starting a new tracker occupation event, sending a sequence confirmation message 552, reading parity data, etc.) can be performed on the parity path controller 506. The remainder of the write operation can be similar to Figure 4 and will not be discussed in detail here.

[0051] Stripe consistency is a prerequisite for correcting data when a media read fails, regardless of whether the media read is part of a write access or a read access from the processor. Fig. 6A A read sequence that results in a media read error on a data path according to one aspect of the present disclosure is illustrated. Fig. 6A 6A , the processing node 602 sends a read request 612 to the data path controller 604. After receiving the read request 612, the data path controller 604 starts a tracker occupied event 616 and reads the data stored at the address specified by the read request 612 (read event 614). Note that if no error occurs, the data will be read and returned to the processing node 602, and the tracker is released. However, in the example shown in FIG. 6A , the read event 614 encounters an error. Therefore, the data path controller 604 sends a rebuild message 618 to the parity path controller 606. As previously discussed, the data path controller 604 can look up its RAID group mapping table to identify which controller stores the parity path data. The rebuild message is a special message used in RAID implementations. The rebuild message can be sent by the data path controller to the parity path controller, thereby requesting the parity path controller to start a reconstruction sequence of its data and return the reconstructed data value to the requesting data path controller in a response message. The reconstruction sequence may include the parity path controller rebuilding the data values ​​on behalf of the requesting data path controller by performing a checksum operation on the parity value of the stripe stored in its medium and the peer stripe data values ​​from all peer data controllers in the RAID group. The parity controller obtains the stripe data values ​​from the peer data controllers by sending a Rebuild-Read (RbldRd) message to each peer data controller and receiving back a response containing the peer path data of the stripe address.

[0052] A rebuild-read message may be similar to a read-request message in that both messages cause the target data path media controller to read the specified data and return the value as a response message. However, a rebuild-read message does not allocate a tracker entry on the responding media controller, the rebuild-read message completes regardless of the pre-existing tracker occupancy of the stripe on the responding media controller, and no corrective action is taken in the event of a media read error. A rebuild-read message returns either the read data or an error indication.

[0053] After receiving the rebuild message 618, the parity path controller 606 may start its own tracker (tracker busy event 620), read the parity data (read event 622), and send a rebuild-read (RbldRd) message 624 to each peer data path controller (eg, controller 608). Fig. 6A A peer data path controller is shown. In fact, there can be multiple peer data path controllers, and a rebuild-read message should be sent to each peer data path controller. In response, each peer data path controller (e.g., controller 608) reads its own data (read event 626) in the corresponding stripe, and uses message 628 to return its own data to the parity path controller 606. Note that the design of the structure and media controller ensures that the rebuild-read operation is completed unconditionally, and these rebuild-read operations may never be blocked by a read request or write request in progress or in operation, a rebuild request, or a sequence start / sequence confirmation / sequence completion message. Similarly, a rebuild request and a sequence start / sequence confirmation / sequence completion message may never be blocked by a read request or write request in progress or in operation.

[0054] Once data from all datapath controllers is received, the parity path controller 606 can reconstruct the data for the datapath controller 604 by XORing the data values ​​from its parity media read (read event 622) and from all peer read responses (e.g., message 628), releasing the tracker (e.g., end tracker occupation event 620) and sending the reconstructed data to the datapath controller 604 using message 630. The datapath controller 604 can then return the reconstructed data to the processing node 602 using message 632. The datapath controller 604 can optionally erase the error by writing the reconstructed data to the media (write buffer event 634 and write media event 636). Depending on the type of error, the erase write may succeed or fail. If the original error corrected by the erase is a "soft error" (e.g., a bit flipped due to a random condition such as an alpha strike), the memory location can be successfully erased. If the original error is a hardware fault (e.g., a stuck-at fault of a memory cell), the erase write to the memory location will typically fail. Note that when an erase is performed, the tracker on the datapath controller 604 will only be released after the write buffer event 634 is completed to maintain stripe consistency. If an erase is not performed or fails, the data stored at the address remains unreadable or poisoned, and each subsequent read of that address will be performed. Fig. 6A The data reconstruction process shown.

[0055] Figure 6B A read sequence with multiple errors according to one aspect of the present disclosure is illustrated. Figure 6B The start of the read sequence in can be similar to Fig. 6A , including a read error when the data path controller 604 and the parity path controller 606 send a rebuild-read message to the peer data path controller 608. Fig. 6A The situation shown is different. Figure 6B In FIG. 6 , peer data path controller 608 encounters an error during read event 642 and returns an error (NAK) message 644 to parity path controller 606. This means that more than one path in the same RAID stripe has failed. In this case, although parity path controller 606 can still receive correct data from other surviving data path controllers, it cannot reconstruct data for data path controller 604. Figure 6B As depicted, both failed paths are data paths. Similar results would occur if one of the failures was a parity media read (e.g., read event 622). As long as two or more of the paths are unreadable, the stripe is uncorrectable. Therefore, parity path controller 606 sends error message 646 to data path controller 604, thereby notifying data path controller 604 that the data reconstruction effort failed and releasing its own tracker. Similarly, data path controller 604 may send error message 648 to processing node 602, indicating that the read operation failed and releasing its own tracker. This is an example of a failure that exceeds the error correction capabilities of the RAID scheme.

[0056] Return to reference Fig. 6AUnlike the write sequence, the read-rebuild sequence does not cause any temporary stripe inconsistencies. The stripe state either remains correctable or transitions directly from a correctable state to a consistent state after a successful erase is performed. However, the rebuild sequence does rely on the existence of invariant values ​​in all modes of operation of the stripe. If the write sequence for the same stripe is allowed to modify data values ​​and / or parity values ​​during the rebuild-read operation, the read-rebuild may return corrupted data. Therefore, the rebuild portion of the read-rebuild sequence is a critical section that needs to be serialized with respect to any writes to the same stripe. As with writes, the critical section of the read-rebuild sequence is completely contained within the time period during which it occupies the parity path tracker 620. The parity path read event 622 and various data path reads (e.g., event 626) all occur completely within the duration of the tracker occupancy event 620. Therefore, if the parity path tracker occupancy of these two operations is serialized, the read-rebuild will not be affected by temporary stripe inconsistencies caused by simultaneous writes to the same stripe.

[0057] The same-stripe parity lane occupation provides the required serialization between critical regions regardless of the mix of write activity and read-rebuild activity that caused the occupation. Figure 7 FIG. 4 illustrates a scenario of serialization of multiple tracker occupancy events on a parity path controller according to one embodiment. Figure 7 In , there are multiple media controllers (controllers 702-706) that can act as data path controllers or parity path controllers for different RAID stripes. Figure 7 , a processor issues a read request 712 to an address for which controller 702 is acting as a datapath controller; and the same processor or a different processor issues a first write request 722 to an address within the same RAID stripe for which controller 706 is a datapath controller. Controller 704 is a parity path controller for the stripe containing the addresses of read request 712 and write request 722. In response to the first write request 722 arriving when the tracker for its stripe address is not occupied, controller 706 initiates a tracker occupied event 724, reads the currently stored data (read event 726), and sends a sequence start message 728 to controller 704 as the parity path controller for the stripe address. Upon receiving the sequence start message 728 arriving when the tracker for its stripe address is not occupied, controller 704 initiates a tracker occupied event 730, reads the currently stored parity (read event 732), and sends a sequence confirmation message 734 back to datapath controller 706.

[0058] In response to receiving the read request 712, the controller 702 initiates a tracker busy event 714 and a read event 716. The read event 716 encounters an error, which causes the controller 702 to send a rebuild message 718 to the controller 704, which acts as the parity path controller for the RAID stripe. Note that the sequence start message 728 from the controller 706 arrives at the controller 704 before the rebuild message 718 from the controller 702. When the rebuild message 718 arrives, the parity path tracker for the particular stripe address is already busy, as indicated by the tracker busy event 730. This means that the processing of the rebuild message 718 will be delayed.

[0059] Operations in response to the write request 722 are performed on both the controller 706 and the controller 704, including the controller 706 starting a tracker busy event 724, sending an acknowledgement message 740 to the processing node that issued the write request 722, writing the data to the write buffer (write buffer event 736), sending a sequence complete message 742 and the data to the parity path controller 704, and writing the data to the storage medium (write media event 738); and the parity path controller 704 writing the data to the write buffer (write buffer event 744) and writing the data to the storage medium (write media event 746). Note that the data path controller 706 ends the tracker busy event 724 after the data has been written to the write buffer and thus becomes visible to subsequent reads. It should also be noted that the parity path controller 704 ends its own tracker busy event 730 after receiving the sequence complete message 742 and writing the data to the write buffer (write buffer event 744). As previously discussed, releasing the tracker in this manner can ensure that all media activity associated with the write operation is completed, making the data and parity visible to subsequent reads consistent.

[0060] Once the parity path tracker is released (e.g., the tracker busy event 730 ends), the parity path controller 704 can process the previously received rebuild message 718. More specifically, the parity path controller 704 can initiate a tracker busy event 750 to track the rebuild process, identify peer data path controllers based on the RAID group mapping table and send a rebuild-read (RbldRd) message 748 to each identified peer data path controller (e.g., controller 706), initiate a tracker busy event 750 to track the rebuild process, and read the current parity (read event 752). Note that the write operation has updated the current parity in response to the first write request 722. After receiving the RbldRd message 748, the data path controller 706 reads its own data (read event 754) and sends the data to the parity path controller 704 via a data message 756. Note that, although similar to a read request, a rebuild-read message received at a datapath controller does not block the normal read process or write process; therefore, the tracker on the responding controller does not track its progress. After receiving data from all peer datapath controllers, controller 704 can reconstruct the requested data for controller 702 based on the peer data and parity. Controller 704 can then send the reconstructed data to datapath controller 702 via data message 758. Controller 702 can optionally attempt a memory erase by writing the data back to the address (write buffer event 760 followed by write media event 762). Datapath controller 702 can also send the data back to the requesting processing node via data message 764 and end tracker busy event 714.

[0061] For controller 704, during the rebuild (e.g., while tracker busy event 750 is ongoing), controller 704 may receive another sequence start message 768 for the same stripe address from controller 706, which controller 706 sent in response to second write-request message 766. Note that the RbldRd message (e.g., message 748) does not trigger the tracker on controller 706. Therefore, controller 706 may begin its own operations, including initiating tracker busy event 770, sending sequence start message 768, and reading current data (read event 772). However, because controller 704 receives sequence start message 768 while its tracker is busy (tracker busy event 750 is ongoing), processing of sequence start message 768 will be delayed until tracker busy event 750 is completed.

[0062] Once the tracker occupied event 750 is over, the controller 704 processes the sequence start message 768 by starting a new tracker occupied event 780, sending a sequence confirmation message 774 to the controller 706, and reading the current parity (read event 782). The rest of the write sequence may include standard operations performed by the controller 704 and the controller 706, which may include the controller 706 writing the data (write buffer event 776 and write media event 778) and the controller 704 writing the data (write buffer event 788 and write media event 790). In addition, the controller 706 sends a confirmation message 784 to the processing node that issued the write request 766, and sends a sequence completion message 786 to the controller 704 along with the data after writing the data to the write buffer to make the data visible. The tracker on controller 706 is released (or tracker occupied event 770 is completed) after data is written to the write buffer (or write buffer event 776 is completed), and the tracker on controller 704 is released (or tracker occupied event 770 is completed) after data has been written to the write buffer in both ways.

[0063] from Figure 7 As can be seen in FIG, there are three operations (one rebuild read operation and two write operations) that overlap in time. All three operations request access to the parity path. To ensure stripe consistency, the parity path controller 704 can implement a single tracker module to serialize the same stripe requests for the parity path, as indicated by tracker occupancy events 730, 750, and 780. Note that subsequent requests for the parity path will not be processed unless the tracker for the previous request has been released. Compared to existing methods that rely on additional mechanisms (e.g., stripes entering degraded mode) to serialize requests, this method can significantly reduce the complexity of the implementation. Note that the terms "data path tracker" and "parity path tracker" are relative terms because each RAID-enabled media controller is equipped with a hardware module for tracking the progress of requests to a specific address (e.g., generating and maintaining tracker entries). If the address stores data, the tracker instance is called a "data path" tracker and tracks data path activity associated with the address, but if the address stores parity, the tracker instance is called a "parity path" tracker and tracks parity path activity associated with the address.

[0064] Apart from FIG. 6A to FIG. 6B and Figure 7 In addition to the read errors shown, other types of errors may occur. The tracker can behave similarly in various error scenarios to ensure stripe consistency. Fig. 8AA write sequence with a parity error according to one aspect of the present disclosure is illustrated. In this example, a processing node 802 sends a write request 812 for a specific address to a data path controller 804 that generates a tracker entry for the address (tracker occupied event 820), and then sends a confirmation message 814 to the processing node 802. The data path controller 804 also sends a sequence start message 816 to the parity path controller 806 and reads the data currently stored at the address (read event 818). Upon receiving the sequence start message 816, the parity path controller 806 generates a tracker entry (tracker occupied event 824), then sends a sequence confirmation message 822 to the data path controller 804, and reads the current parity (read event 826). The read event 826 encounters an error. On the other hand, the data path controller 804 writes the data (write buffer event 828 and write media event 830), and sends a sequence completion message 832 together with the data to the parity path controller 806. After the data becomes visible (or the write buffer event 828 is completed), the data path controller 804 releases its tracker (or ends the tracker occupied event 820). However, because the parity path controller 806 encountered a read error, the parity will not be updated. After receiving the sequence complete message 832, the parity path controller 806 ends the tracker occupied event 824. Note that in this example, there is no need to erase the corrupted parity data; the parity data is still unavailable. When an erase is required, the parity path controller 806 can send an RbldRd message to all peer data paths, recalculate the parity value using an XOR operation together with the write data included in the sequence complete message 832, and then attempt to write it to the parity path medium before releasing the parity path tracker entry.

[0065] Figure 8BA write sequence with data errors according to one aspect of the present disclosure is illustrated. In this example, a processing node 802 sends a write request 842 for a specific access address to a data path controller 804 that generates a tracker entry for the access address (tracker occupied event 850), and sends a confirmation message 844 to the processing node 802. The data path controller 804 also sends a sequence start message 846 to the parity path controller 806 and reads the data currently stored at the address (read event 848). Note that the read event 848 encounters an error and cannot obtain the pre-written data. In some cases, the data path controller 804 can erase the data (write buffer event 858 and attempt to write media event 860) after receiving the sequence confirmation message 852. Because a partial write of any subset of a block will depend on the data read in an earlier data path read (e.g., read event 848), an erase write can only be attempted when the data covers an entire block of data (e.g., cache line data in a RAID system using cache line sized blocks). Regardless of whether a datapath erase write is attempted or not, and whether it succeeds or not, a parity update must be completed to reflect the written data in the checksum value.

[0066] In response to receiving the sequence start message 846, the parity path controller 806 sends back a sequence confirmation message 852, reads the current parity (read event 854), and generates a tracker entry (tracker occupied event 856). In response to receiving the sequence confirmation message 852 and the complete write buffer event 858 (if applicable), the data path controller 804 releases its tracker (or completes the tracker occupied event 850) and sends a sequence completion message 862 together with the written data and an error message indicating that the read failed. Note that updating the parity requires correct pre-write data. When the data path controller 804 fails to read the pre-write data, the parity path controller 806 can rebuild the pre-write data from the current parity and data from (multiple) other peer data path controllers. In this example, the parity path controller 806 sends a rebuild-read (RbldRd) message 864 to the peer data path controller 808, and the peer data path controller reads the data at the corresponding address (read event 866) and sends the data to the parity path controller 806. Once data from all peer data path controllers is received, the parity path controller 806 can use the received peer data and pre-write parity data to rebuild the pre-write data. The parity path controller 806 can further calculate the updated parity based on the pre-write data, the current data and the pre-write parity, and write the updated parity (write buffer event 870 and write media event 872). Note that once the updated parity is written to the buffer or becomes visible, the parity path controller 806 can release its tracker (or end tracker occupation event 856) to allow other processes to access the parity path.

[0067] Fig. 9 An apparatus for facilitating RAID operations in a byte-addressable memory system according to one aspect of the present disclosure is illustrated. Apparatus 900 may include a plurality of units or devices that may communicate with each other via wired, wireless, quantum optical, or electrical communication channels. Apparatus 900 may be implemented using one or more integrated circuits and may include fewer or more units or devices than the units or devices shown in FIG. 9 . According to one aspect, the various units in apparatus 900 may be implemented using hardware logic to ensure a higher processing speed for memory requests. Apparatus 900 may be part of a media controller used in a Gen-Z memory system. A media controller may be used to control media access to a byte-addressable storage medium (e.g., SCM). For example, apparatus 900 may be embedded in a Gen-Z media controller to facilitate RAID operations on multiple SCM devices.

[0068] The device 900 may include a processor interface 902 for communicating with a processing node requesting memory access, a peer interface 904 for communicating with peer devices embedded in other media controllers, a media interface 906 for accessing data stored in physical media, a mapping table 908 for storing RAID group information, a table lookup unit 910 for looking up the mapping table 908, a tracker unit 912 for tracking ongoing operations (e.g., by generating and deleting tracker entries), and a request serialization unit 914 for serializing media access requests based on active tracker entries.

[0069] For example, in a situation similar to Figure 5 During the write operation shown, for the data path, the processor interface 902 receives a write request from a processing node (e.g., a CPU of a host computer) via a memory semantic structure and sends back a write confirmation message; the peer interface 904 communicates with the parity path controller via the memory semantic structure (e.g., sends a sequence start message, receives a sequence confirmation message, and sends a sequence completion message with data); the medium interface 906 accesses the attached medium (reads pre-written data and writes data to the medium); the tracker unit 912 starts and ends tracker occupancy events (e.g., activates and deactivates tracker entries); and the request serialization unit 914 can ensure that any subsequent requests for the data path address will be delayed until the data path tracker is released (e.g., the corresponding tracker entry is deactivated).

[0070] Similarly, for a parity path, the peer interface 904 communicates with the data path controller (e.g., receives sequence start messages, sends sequence confirmation messages, and receives sequence completion messages with data); the media interface 906 accesses the attached media (reads pre-written parity and writes updated parity to the media); the tracker unit 912 starts and ends tracker occupancy events (e.g., activates and deactivates tracker entries); and the request serialization unit 914 can ensure that any subsequent requests for a parity path address will be delayed until the parity path tracker is released (e.g., the corresponding tracker entry is deactivated).

[0071] In general, the present disclosure provides a RAID solution for a byte-addressable memory system (e.g., an SCM system). Most particularly, the RAID solution is ownerless, meaning that a master RAID controller is not involved. The media controller in the existing SCM architecture can be enhanced (e.g., by including a RAID controller unit) to implement collaborative distributed RAID operations. The proposed RAID solution can also provide consistency even if multiple requests are made to the same memory location. For example, multiple read operations or write operations for the same memory location can be serialized by the corresponding data path media controller using the tracker entry of the address. Similarly, the parity path activities (including parity queries and updates) requested by the corresponding data path of the same stripe can be serialized by the parity path media controller using tracker entries. In essence, the parity path tracker is used to keep occupied in all critical areas that need to be serialized to ensure non-hazardous operations.

[0072] One aspect of the present disclosure may provide a storage system. The storage system may include multiple byte-addressable storage devices and multiple media controllers. The corresponding byte-addressable storage devices are used to store parity blocks or data blocks of data stripes, and the corresponding media controllers are coupled to the corresponding byte-addressable storage devices. Each media controller may include a tracker logic block to serialize critical sections of multiple media access sequences associated with addresses on the corresponding byte-addressable storage devices. Each media access sequence includes one or more read and / or write operations, and the data stripes may be inconsistent during the critical section of the media access sequence.

[0073] In variations of this aspect, the medium access sequence includes one or more of: a read sequence, a write sequence, and a read-rebuild sequence.

[0074] In a variation of this aspect, the tracker logic block is used to generate a tracker entry for the address to serialize access to the address.

[0075] In a variation of this aspect, each media controller may further include a mapping table that identifies one or more media controllers associated with a data block and a media controller associated with a parity block of a data stripe.

[0076] In variations of this aspect, each media controller may include a processor interface for communicating with one or more processing nodes via a memory semantic structure and a peer interface for communicating with one or more peer media controllers.

[0077] In a variation of this aspect, during a write operation to a data block in a data stripe, a media controller associated with the data block is operable to send a sequence start message to a media controller associated with a parity block to allow reads of the data block and the parity block to be performed substantially simultaneously.

[0078] In a further variation, during a write operation, a tracker logic block of a media controller associated with the data block is used to delay subsequent access to the data block until the media controller associated with the data block has received a sequence confirmation message from the parity path controller in response to the sequence start message and until data for the write operation has been written to a write buffer on the media controller associated with the data block.

[0079] In a further variation, during a write operation, a media controller associated with the parity block is used to update the parity block based on data of the write operation, and a tracker logic block of the media controller associated with the parity block is used to delay a subsequent media access sequence involving the parity block until the data of the write operation has been written to a write buffer on the media controller associated with the data block and the updated parity has been written to the write buffer on the media controller associated with the parity block.

[0080] In a further variation, a media controller associated with the data block is operable to send a message to a media controller associated with the parity block to indicate that data of the write operation has been written to the write buffer.

[0081] In a variation of this aspect, a media controller associated with the data block is operable to send a reconstruction message to a media controller associated with the parity block in response to detecting an error when reading the data block, the media controller associated with the parity block is operable to reconstruct the data block based on the parity block, and a tracker logic block of the media controller associated with the parity block is operable to block other accesses to the parity block until reconstruction of the data block is completed.

[0082] One aspect may provide a method. The method may include: in response to a media access request for a data block within a data stripe including a parity block and a plurality of data blocks, identifying a parity path controller and one or more data path controllers corresponding to the data stripe. The data stripe is distributed across a plurality of byte-addressable storage devices, the parity path controller is coupled to a storage device storing the parity block, and the data path controller is coupled to a storage device storing the data block. The method may also include executing a media access sequence in response to the request, including the data path controller and the parity path controller serializing critical sections of a plurality of media access sequences associated with the data stripe. Each memory access sequence includes one or more read and / or write operations, and the data stripes may be inconsistent during a critical section of the media access sequence. One aspect may provide a redundant array of independent disks (RAID) enhanced media controller. The RAID enhanced media controller may include a media interface for attaching a byte-addressable memory device coupled to the RAID enhanced media controller, a processor interface for communicating with one or more processing nodes that issue memory access requests, and a peer interface for communicating with one or more peer RAID enhanced media controllers to collaboratively provide RAID functionality across multiple byte-addressable memory devices that are respectively attached to the multiple RAID enhanced media controllers. The byte-addressable memory device is used to store parity blocks or data blocks of a data stripe. The RAID enhanced media controller may further include a tracker logic block to serialize critical sections of multiple media access sequences associated with addresses on the byte-addressable storage device. Each media access sequence includes one or more read and / or write operations, and the data stripes may be inconsistent during the critical section of the media access sequence.

[0083] The methods and processes described in the detailed description may be embodied as code and / or data, which may be stored in a computer-readable storage medium as described above. When a computer system reads and executes the code and / or data stored on a computer-readable storage medium, the computer system executes the methods and processes embodied as data structures and codes and stored in the computer-readable storage medium.

[0084] In addition, the above methods and processes may be included in hardware modules or devices. Hardware modules or devices may include, but are not limited to, ASIC chips, field programmable gate arrays (FPGAs), dedicated or shared processors that execute specific software modules or codes at specific times, and other programmable logic devices now known or later developed. When the hardware modules or devices are activated, these hardware modules or devices execute the methods and processes included therein.

[0085] The foregoing description has been presented for the purpose of illustration and description only. The description is not intended to be exhaustive or to limit the scope of the present disclosure to the disclosed forms. Accordingly, many modifications and variations will be apparent to those of ordinary skill in the art.

Claims

1. A storage system, comprising: a plurality of byte-addressable storage devices, wherein respective byte-addressable storage devices are used to store parity blocks or data blocks of a data stripe; and a plurality of media controllers, wherein respective media controllers are coupled to corresponding byte-addressable storage devices; wherein each media controller includes a tracker logic block to serialize critical regions of a plurality of media access sequences associated with addresses on the corresponding byte-addressable storage device, wherein each media access sequence includes one or more read and / or write operations, wherein the data stripes are inconsistent or consistent during the critical region of the media access sequence, and wherein the tracker logic block is used to generate tracker entries for the addresses to serialize access to the addresses.

2. The storage system according to claim 1, wherein: The medium access sequence includes one or more of the following: Read the sequence; Write sequence; and Read-reconstruct sequence.

3. The storage system according to claim 1, wherein: Each media controller further includes a mapping table that identifies one or more media controllers associated with the data blocks and a media controller associated with a parity block for the data stripe.

4. The storage system according to claim 1, wherein: Each media controller includes: a processor interface for communicating with one or more processing nodes via a memory semantic structure; and A peer interface is used to communicate with one or more peer media controllers.

5. The storage system of claim 1, wherein: During a write operation to a data block in the data stripe, a media controller associated with the data block is operable to send a sequence start message to a media controller associated with the parity block to allow reading of the data block and the parity block to be performed substantially simultaneously.

6. The storage system of claim 5, wherein: During the write operation, a tracker logic block of the media controller associated with the data block is used to delay subsequent access to the data block until the media controller associated with the data block has received a sequence confirmation message from a parity path controller in response to the sequence start message and until data of the write operation has been written to a write buffer on the media controller associated with the data block.

7. The storage system of claim 6, wherein: During the write operation, the media controller associated with the parity block is used to update the parity block based on data of the write operation; and A tracker logic block of the media controller associated with the parity block is used to delay a subsequent media access sequence involving the parity block until data of the write operation has been written to a write buffer on the media controller associated with the data block and updated parity has been written to a write buffer on the media controller associated with the parity block.

8. The storage system according to claim 6, wherein: The media controller associated with the data block is used to send a message to the media controller associated with the parity block to indicate that the data of the write operation has been written into the write buffer.

9. The storage system of claim 1, wherein: In response to detecting an error while reading the data block, the media controller associated with the data block is operable to send a rebuild message to the media controller associated with the parity block; The media controller associated with the parity block is used to reconstruct the data block based on the parity block; and A tracker logic block of the media controller associated with the parity block is used to block other accesses to the parity block until reconstruction of the data block is completed.

10. A method comprising: identifying, in response to a media access request for a data block within a data stripe including a parity block and a plurality of data blocks, a parity path controller and one or more data path controllers corresponding to the data stripe, wherein the data stripe is distributed across a plurality of byte-addressable storage devices; wherein the parity path controller is coupled to a storage device storing the parity block, and wherein the data path controller is coupled to a storage device storing the data blocks; and Executing a media access sequence in response to the request includes the data path controller and the parity path controller serializing critical sections of a plurality of media access sequences associated with the data stripes, wherein each memory access sequence includes one or more read and / or write operations, and wherein the data stripes are inconsistent or consistent during the critical sections of the media access sequences.

11. The method of claim 10, wherein: The medium access sequence includes one or more of the following: Read the sequence; Write sequence; and Read-reconstruct sequence.

12. The method of claim 10, wherein: Serializing access to the data block or the parity block includes generating a tracker entry for an address associated with the data stripe.

13. The method of claim 10, wherein: Identifying the parity path controller and the one or more data path controllers includes looking up a mapping table and performing a calculation based on an address of the data block targeted by the media access request and information included in the mapping table.

14. The method of claim 10, wherein: Performing the write operation includes the data path controller sending a sequence start message to the parity path controller to allow reading of the data block and the parity block to be performed substantially simultaneously.

15. The method of claim 14, wherein: Performing the write operation includes delaying subsequent access to the data block until the data path controller has received a sequence confirmation message from the parity path controller in response to the sequence start message and until data for the write operation has been written to a write buffer on the data path controller.

16. The method of claim 15, wherein: Executing the write operation includes: updating the parity block based on data of the write operation; and Execution of a subsequent media access sequence involving the parity block is delayed until data for the write operation has been written to a write buffer on the data path controller and updated parity has been written to a write buffer on the parity path controller.

17. The method of claim 15, further comprising sending a message by the data path controller to the parity path controller to indicate that the data of the write operation has been written into the write buffer.

18. The method of claim 10, further comprising: In response to detecting an error when reading the data block, sending a rebuild message by the data path controller to the parity path controller; reconstructing, by the parity path controller, the data block based on the parity block; as well as Further access to the parity block is delayed until reconstruction of the data block is complete.

19. A redundant array of independent disks (RAID) enhanced media controller comprising: a media interface for attaching a byte-addressable memory device coupled to the RAID enhanced media controller; a processor interface for communicating with one or more processing nodes that issue memory access requests; a peer interface for communicating with one or more peer RAID enhanced media controllers to cooperatively provide RAID functionality across a plurality of byte-addressable memory devices respectively attached to the plurality of RAID enhanced media controllers, wherein the byte-addressable memory devices are used to store parity blocks or data blocks of a data stripe; and A tracker logic block, the tracker logic block is used to serialize critical sections of multiple media access sequences associated with addresses on the byte-addressable storage device, wherein each media access sequence includes one or more read and / or write operations, wherein the data stripes are inconsistent or consistent during the critical section of the media access sequence, and wherein the tracker logic block is used to generate a tracker entry for the address to serialize access to the address.

Images