System and method for reconstructing data from a degraded raid volume using an accelerator engine

Abstract

A system and method for reconstructing data from a degraded RAID storage device using an accelerator engine is disclosed. An article of manufacture can include a non-transitory memory having machine-readable instructions that, when executed by a processor, cause the processor to transmit a first command to a first storage device and a second storage device to trigger the first storage device and the second storage device to write striped data to a memory in an accelerator engine. The instructions can also cause the processor to transmit a second command to the accelerator engine to perform an operation on the written striped data for reconstructing data stored on a third failed storage device. The first storage device, the second storage device, and the third storage device can be part of a RAID volume. Further, the instructions can cause the processor to receive an output of the operation from the accelerator engine.

Description

priority

[0001] This application claims priority to Indian Patent Application No. 202411017420, filed on March 11, 2024, the entire disclosure of which is incorporated herein by reference. Technical Field

[0002] This disclosure relates to rebuilding data from a degraded redundant array of independent disks (RAID), for example, where one of the RAID drives has failed, and in particular to rebuilding data from a degraded RAID using an accelerator engine and peer-to-peer direct memory access (DMA) capabilities. Background Technology

[0003] When a group of drives is represented as a single unit, that is, when an array is referred to as a single unit, the array can be called a RAID volume. While the term RAID historically referred to "disks," it can be used in relation to other storage devices, but is not limited to that. Therefore, the term RAID can be considered as a redundant array of individual drives (such as solid-state drives) or a redundant array of individual storage devices.

[0004] A typical Redundant Array of Independent Disks (RAID) stack running on the host machine utilizes host instructions for parity generation, which is a CPU and memory-intensive operation, even on high-performance x86_64 servers. CPU instructions may require up to two inputs and can be inefficient for large data stripes. Some server CPUs use Advanced Vector Extensions (AVX) instructions for parity generation, which increases the load on the host and host dynamic random access memory (DRAM) for any parity calculation operations.

[0005] XOR parity generation is one of the building blocks of RAID algorithms. XOR parity generation is also used in various other operations, such as error detection, encryption, and pseudo-random number generators, but is not limited to these. The software stack running in the host server typically uses regular CPU instructions or high-level vector instructions (such as AVX-256 or AVX-512) to perform XOR operations. Performing an XOR operation on multiple stripes of a distributed data buffer consumes significant host and memory controller bandwidth. Traditional hardware (HW) RAID architectures can be a performance bottleneck when scaling via high-performance non-volatile memory fast (NVMe) drives.

[0006] PCIe (Peripheral Component Interconnect Fast) is a high-speed serial computer expansion bus standard that replaced the older PCI, PCI-X, and Accelerated Graphics Port (AGP) bus standards. The PCIe root complex connects PCIe endpoints to the CPU and memory subsystem to route communication between connected devices. PCIe endpoints or switches are not connected to the CPU or memory directly but via the PCIe root complex. PCIe uses a point-to-point topology, allowing for faster communication between devices. PCIe-enabled motherboards and systems use different sizes of PCIe devices, such as x1, x4, x8, or x16, referring to the number of lanes they use. PCIe devices connect to the motherboard or system using PCIe slots, allowing the device to be recognized by the motherboard or system.

[0007] Non-volatile Memory Fast (NVMe) is an open logical device interface specification for accessing non-volatile storage media in computers typically attached via the PCIe bus. NVMe can be used with NAND flash memory that goes into plug-in PCIe cards.

[0008] However, existing RAID stacks use DRAM in the host for transfer buffers and perform calculations on the host for rebuilding degraded RAID storage devices. These calculations consume significant bandwidth of the host's memory and impair the host's ability to run other applications. Summary of the Invention

[0009] Various aspects provide systems and methods for reconstructing data from a degraded Redundant Array of Independent Disks (RAID) volume using an accelerator engine. An example may include an artifact. This artifact may include non-transitory memory having machine-readable instructions that, when executed by a processor, cause the processor to send a first command to a first and a second storage device to trigger the first and second storage devices to write striped data to memory in the accelerator engine. The instructions may also cause the processor to send a second command to the accelerator engine to perform an operation on the written striped data for reconstructing data stored on a third failed storage device. The first, second, and third storage devices may be part of a RAID volume. Furthermore, the instructions may cause the processor to receive the output of an operation from the accelerator engine.

[0010] In conjunction with any of the examples above, this instruction can also enable the processor to use the PCIe bus, which includes the Peripheral Component Interconnect Fast (PCIe) root complex, to transmit a first command, transmit a second command, and receive output.

[0011] Combining any of the examples above, this operation could be an XOR operation on the written stripe data.

[0012] Combining any of the examples above, the first storage device, the second storage device, and the accelerator engine can communicate using peer-to-peer direct memory access.

[0013] Combining any of the examples above, the accelerator engine, as well as the first and second storage devices, can be PCIe endpoints.

[0014] In conjunction with any of the examples above, this instruction can also cause the processor to receive a message from the first and second storage devices indicating that the first and second storage devices have completed writing the striped data into the memory of the accelerator engine. This message can trigger the processor to transmit a second command.

[0015] Individually or in combination with any of the examples above, examples of this disclosure may include a method. The method may include transmitting a first command to a first storage device and a second storage device to trigger the first and second storage devices to write striped data to memory in an accelerator engine. The first and second storage devices may be part of a RAID volume. The method may additionally include transmitting a second command to the accelerator engine to perform an operation on the written striped data. This operation may be used to rebuild data stored on a third failed storage device of the RAID volume. The method may also include receiving the output of the operation from the accelerator engine.

[0016] In conjunction with any of the examples above, the method may also include using a PCIe bus that includes a Peripheral Component Interconnect Fast (PCIe) root complex to transmit a first command, transmit a second command, and receive output.

[0017] Combining any of the examples above, the PCIe root complex can be used to communicate between the first and second storage devices and the memory in the accelerator engine based on the base address register (BAR) of the memory in the accelerator engine.

[0018] Combining any of the examples above, this operation could be an XOR operation on the written stripe data.

[0019] Combining any of the examples above, the first command can instruct the first and second storage devices to communicate with the accelerator engine using peer-to-peer direct memory access.

[0020] Combining any of the examples above, the accelerator engine, as well as the first and second storage devices, can be PCIe endpoints.

[0021] In conjunction with any of the examples above, the method may further include receiving a message from a first storage device and a second storage device indicating that the first and second storage devices have completed writing the striped data into the memory of the accelerator engine. This message may trigger the transmission of a second command.

[0022] Individually or in combination with any of the examples above, examples of this disclosure may include a system. The system may include a memory bus, accelerator engine circuitry coupled to the memory bus, and a host coupled to the memory bus. The host may include a processor and non-transitory memory including machine-readable instructions that, when executed by the processor, cause the processor to transmit a first command to a first and a second storage device coupled to the memory bus to trigger the first and second storage devices to write striped data into memory within the accelerator engine circuitry. These instructions may also cause the processor to transmit a second command to the accelerator engine circuitry to perform an operation on the written striped data. This operation may be used to reconstruct data previously stored on a third failed storage device. The first, second, and third storage devices may be part of a RAID volume. These instructions may also cause the processor to receive the output of the operation from the accelerator engine circuitry.

[0023] In conjunction with any of the examples above, the memory bus can be a PCIe bus that includes the Peripheral Component Interconnect Fast (PCIe) root complex.

[0024] Combining any of the examples above, the PCIe root complex can be used to route communication between the first and second storage devices and the memory in the accelerator engine circuitry based on the base address register (BAR) of the memory in the accelerator engine circuitry.

[0025] Combining any of the examples above, this operation could be an XOR operation on the written stripe data.

[0026] Combining any of the examples above, the first storage device, the second storage device, and the accelerator engine circuitry can communicate using peer-to-peer direct memory access.

[0027] Combining any of the examples above, the accelerator engine circuitry, as well as the first and second storage devices, can be PCIe endpoints.

[0028] In conjunction with any of the examples above, this instruction can also cause the processor to receive a message from the first and second storage devices indicating that the first and second storage devices have completed writing the striped data into the memory of the accelerator engine circuitry. This message can trigger the processor to transmit a second command.

[0029] Although example implementations have been described above, other variations and implementations may be made in accordance with this disclosure without departing from the substance and scope of these implementations. Attached Figure Description

[0030] The accompanying figures illustrate examples of the systems and methods.

[0031] Figure 1 A block diagram illustrating a system for reconstructing data from a degraded RAID volume using an accelerator engine, according to an example of this disclosure;

[0032] Figure 2 An example of a method for reconstructing data from a degraded RAID volume using an accelerator engine, according to an example of this disclosure, is illustrated; and

[0033] Figure 3 An example of a method for reconstructing data from a degraded RAID volume using an accelerator engine, according to an example of this disclosure, is illustrated.

[0034] Reference numerals for any illustrated element appearing in multiple different figures have the same meaning in all figures, and any reference or discussion of any illustrated element in the context of any particular figure also applies to each other figure (if any) in which the same illustrated element is shown. Detailed Implementation

[0035] According to aspects of the present invention, a system and method are provided for reconstructing data from a degraded Redundant Array of Independent Disks (RAID) volume using an accelerator engine. A RAID volume may be degraded when a storage device that is part of it fails. As described above, a RAID volume is a logical device created using multiple storage devices, such as drives or disks, but is not limited thereto. The system and method reconstruct data from the failed RAID volume using peer-to-peer communication (e.g., Peripheral Component Interconnect Fast (PCIe) Peer-to-Peer Direct Memory Access (DMA) capabilities), allowing an accelerator engine to reconstruct data in the case of a degraded RAID volume without using host memory as a transfer buffer. Using an accelerator engine for data reconstruction reduces the use of host memory and improves the performance of the degraded RAID volume. The memory in the accelerator engine (rather than host memory) can be used as a transfer buffer, reducing overhead at the host.

[0036] The aspects include features disclosed in Indian Patent Application No. 202311056027, filed August 21, 2023, and U.S. Patent Application No. 18 / 527,579, filed December 4, 2023, which are incorporated herein by reference in their entirety for all purposes.

[0037] Figure 1 A block diagram illustrating a system for rebuilding data from a degraded RAID volume using an accelerator engine, according to an example of this disclosure, is provided. System 100 may include a host 110, which may include a processor 112, a memory 114, and a memory bus 116. Processor 112 may use memory bus 116 to write data to memory 114 and read data from memory 114. Memory 114 may be non-transitory memory, such as non-volatile random access memory (NVRAM), including machine-readable instructions that, when executed by processor 112, cause processor 112 to perform one or more actions as described herein. Processor 112 may be a central processing unit (CPU), general-purpose processor, special-purpose processor, microcontroller, programmable logic controller (PLC), digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic device, discrete gate or transistor logic device, discrete hardware component, other programmable device, or any combination thereof, designed to perform the functions disclosed herein. Processor 112 can drive signals onto memory bus 116 to read data from storage devices 122a, 122b, and 122c (collectively, "storage device 122"), and can drive signals onto memory bus 116 to write data to storage device 122. Processor 112 may include RAID driver stack 118 to manage storage device 122 and accelerator engine 130. Storage device 122 can be controlled as RAID volume 120 in response to operation of RAID driver stack 118. (See also: [link to relevant documentation]) Figure 2 As described, when performing a data reconstruction operation (e.g., an XOR operation), the RAID drive stack 118 can invoke a Non-Volatile Memory Fast (NVMe) offload command to perform the operation. The NVMe offload command can be vendor-defined. In doing so, the RAID drive stack 118 will use memory 132 in the accelerator engine 130 instead of using memory 114 in the host 110 for the transfer buffer during the data reconstruction operation.

[0038] The memory bus 116 can receive data from the processor 112 and can send data to one or more circuits coupled to the memory bus 116. The memory bus 116 can be a Peripheral Component Interconnect Fast (PCIe) bus or another bus type not specifically mentioned. The memory bus 116 can be a PCIe root complex, which is the root of a hierarchical structure connected to the host 110, RAID volume 120, and accelerator engine 130. The host 110, the storage device 122 in RAID volume 120, and the accelerator engine 130 can be PCIe endpoints. The PCIe endpoints are not directly connected to the processor 112 or the memory 116, but are connected via the memory bus 116.

[0039] Storage device 122 may be a non-transitory storage device, such as a solid-state drive or hard disk drive, including but not limited to dynamic random access memory (DRAM), non-volatile memory (NVM), embedded non-volatile memory (eNVM), non-volatile memory fast (NVMe), or another type of non-transitory storage device not specifically mentioned. A given storage device 122 may include circuitry for moving data to and from the storage device 122. Storage device 122 may use open logic device interface specifications to access non-volatile storage media, where these specifications may include NVMe or the Non-Volatile Memory Host Controller Interface Specification (NVMHCIS). Memory bus 116 facilitates data transmission to and from storage device 122. Processor 112 may move data to and from storage device 122 via memory bus 116, and processor 112 may receive data from storage device 122 via memory bus 116. Additionally, storage device 122 and accelerator engine 130 may communicate with each other using memory bus 116.

[0040] RAID volume 120 can be a redundant array of independent disks formed by storage devices 122. In some examples, RAID volume 120 can be a RAID-5 system including at least three storage devices 122. While there is no limit to the maximum number of drives, three storage devices will be used for illustrative purposes, but it is not limited to this. RAID volume 120 can use disk striping with parity when storing information. Data and parity information can be striped evenly across the storage devices 122 in RAID volume 120. As an example of disk striping, in a RAID volume with three storage devices and assuming each stripe is 16KB, each row of the RAID volume will contain two data stripes and one parity information stripe. The two data stripes are collectively referred to as "stripes" and will be accompanied by a parity information stripe for a given stripe. For example, when RAID volume 120 includes three storage devices 122, the stripe can include data stored on two of the three storage devices 122 and parity information stored on the third of the three storage devices 122. Specifically, as Figure 1 As illustrated, stripe 124a may include data A1 and A2 stored on storage devices 122a and 122b respectively, and parity information P1(A1, A2) of data A1 and A2 stored on storage device 122c. Any two of the three storage devices 122 may store the data portion of the stripe, while the remaining storage device 122 may store the parity information. Figure 1As shown, parity information for stripe 124b is stored on storage device 122b, parity information for stripe 124c is stored on storage device 122a, and parity information for stripe 124d is stored on storage device 122b. Striping allows data writes and reads to occur in parallel across multiple storage devices 122, and avoids any single storage device 122 becoming a bottleneck. Striping also allows users to rebuild data in the event of a failure of one storage device in storage device 122 (i.e., RAID volume 120 degradation). If any storage device 122 of RAID volume 120 is lost or has failed, RAID volume 120 can be considered degraded.

[0041] System 100 may also have one or more accelerator engines 130 coupled to host 110 and storage device 122 via memory bus 116. Accelerator engines 130 may be used to offload computational operations from host 110 to PCIe endpoints using NVMe transport protocol instruction communication and DMA data communication. Accelerator engines 130 may include memory 132, engine 136, and processor 138. Accelerator engines 130 may support XOR offload commands for rebuilding data stored on failed storage device 122 of a degraded RAID volume 120. Accelerator engines 130 support XOR offload commands by providing memory 132 in accelerator engines 130 instead of memory 114 in host 110, which can be used as a buffer during data rebuilding. Specifically, memory 132 may include instructions for creating regions of memory 132, such as dynamic random access memory (DRAM) windows, which are exposed as base address registers (BARs) (e.g., BAR4) on the RAID drive of host 110. Memory 132 may be, but is not limited to, Double Data Rate (DDR) memory or Double Data Rate Synchronous Dynamic Random Access Memory (DDRSDRAM). Engine 136 may be an engine or circuit for performing operations, such as a parity generation engine or parity generation circuit, and may be used to reconstruct data based on parity information. Engine 136 may perform XOR operations. Processor 138 may be a general-purpose processor, special-purpose processor, microcontroller, programmable logic controller (PLC), digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware component, other programmable device, or any combination thereof, designed to perform the functions disclosed herein.

[0042] When storage device 122 in RAID volume 120 fails, causing RAID volume 120 to degrade, system 100 can use the remaining storage device 122 (e.g., the unfailed storage device 122) and the memory 132 in accelerator engine 130 via peer-to-peer direct memory access communication via memory bus 116 to reconstruct data from the failed storage device 122. When memory bus 116 is a PCIe bus, PCIe peer-to-peer communication can be used to reconstruct data from the failed storage device 122, which allows the two PCIe devices (e.g., storage device 122 and accelerator engine 130) to directly transfer data between each other without using the memory 116 in host 110 as a temporary storage device.

[0043] For example, in an example where storage device 122b has failed, data reconstruction from storage device 122b can begin when processor 112 writes to the doorbell registers of storage devices 122a and 122c using memory bus 116. The doorbell registers provide a way for one device coupled to memory bus 116 to send messages to another device coupled to memory bus 116 without using interrupts. Therefore, when processor 112 writes to the doorbell registers of storage devices 122a and 122c, the doorbell registers indicate to the processors or control circuitry of storage devices 122a and 122c that a command is pending.

[0044] A doorbell register write allows storage devices 122a and 122c to read a first command from host 110 via memory bus 116. The first command allows storage devices 122a and 122c to write striped data to memory 132. The striped data written to memory 132 can be data associated with a given data stripe. For example, when the given data stripe is data stripe 124a, storage device 122a can write data A1 to memory 132, and storage device 122c can write parity information P1(A1, A2) to memory 132. Storage devices 122a and 122c can use direct memory access (DMA) via memory bus 116 to write striped data to memory 132, and a region of memory 132 can be exposed to storage devices 122a and 122c as BAR4.

[0045] After writing the striped data to memory 132, storage devices 122a and 122c can transmit a message to the RAID drive stack via memory bus 116, indicating that storage devices 122a and 122c have completed writing the striped data. The message can be in the form of an interrupt or polling. For example, the message can be a Message Signaling Interrupt (MSI) signal, such as MSI or MSI-X. In response to the message indicating that storage devices 122a and 122c have completed writing the striped data, host 110 initiates a second command addressed to accelerator engine 130, for example, by performing a write to the doorbell register at accelerator engine 130. The second command can be an XOR command. The second command can be transmitted from host 110 to accelerator engine 130 via memory bus 116.

[0046] In response to a write to the doorbell register of accelerator engine 130, accelerator engine 130 can retrieve a second command from host 110 via memory bus 116. Engine 136 in accelerator engine 130 can then perform the operation specified in the second command on the striped data written to memory 132 by storage devices 122a and 122c. For example, the second command could be an XOR command. The result of the second command is reconstructed data from faulty storage device 122b.

[0047] Once the accelerator engine 130 has reconstructed the data from storage device 122b, it can transfer the reconstructed data to memory 114 via memory bus 116. After transferring the reconstructed data to memory 114, the accelerator engine 130 can also send a message indicating that the data reconstruction is complete to the RAID driver stack via memory bus 116. Upon receiving the message, the RAID driver stack can perform any further processing on the reconstructed data. This message can be in the form of polling or interrupts, such as MSI or MSI-X.

[0048] The above process can use peer-to-peer DMA operations between storage devices 122a and 122c and accelerator engine 130 without transferring striped data to memory 114, and without processor 112 performing XOR or other operations on the data to recover lost data, thereby improving efficiency and reducing overhead in host 110 when rebuilding data from degraded storage devices.

[0049] Figure 2 An example of a method for reconstructing data from a degraded RAID volume using accelerator engine circuitry, according to an example of this disclosure, is illustrated. Method 200 can be used... Figure 1Any of the components shown, such as host 110, RAID volume 120, and accelerator engine 130, individually or in combination, or any other component operable to implement method 200. Although examples have been described above, other variations and examples may be made in accordance with this disclosure without departing from the spirit and scope of these disclosed examples.

[0050] Method 200 begins at box 210, in which the RAID drive stack can transmit a first command to a storage device (hereinafter referred to as the "functional storage device") that has not yet failed or caused an error, to retrieve striped data from the functional storage device and instruct the functional storage device to write the striped data to memory in the accelerator engine circuitry, such as... Figure 1 The memory 132 in the accelerator engine 130 is shown. The RAID driver stack can transmit a first command after initiating a stripe read of the RAID volume and discovering that the storage device of the RAID volume has failed or is faulty. In response to receiving the first command, the functional storage device can write the stripe data to the memory in the accelerator engine circuitry.

[0051] At box 250, the RAID driver stack can send a second command, such as an XOR command, to the accelerator engine circuitry to trigger the accelerator engine circuitry to perform an operation on the striped data written at box 210, such as, but not limited to, an XOR operation. The second command can utilize a memory write transaction being sent to a doorbell register in the accelerator engine circuitry and is a vendor-defined NVMe XOR command. In response to receiving the second command, the accelerator engine circuitry can perform an operation on the striped data from the memory in the accelerator engine circuitry, such as an XOR operation or another operation.

[0052] At box 280, the RAID drive stack can receive the output of the operation from the accelerator engine circuitry. The accelerator engine circuitry can write the output of the operation to the host memory. The accelerator engine circuitry can also write the output of the operation to the host memory via DMA. The output of the XOR operation can be a reconstruction of data stored on a failed or faulty storage device.

[0053] although Figure 2 A specific number of operations related to method 200 are disclosed, but method 200 is available in more... Figure 2 The number of operations described can be increased or decreased to perform the task. Furthermore, although... Figure 2 The specific order of operations to be performed in method 200 is disclosed, but the operations including method 200 can be performed in any suitable order.

[0054] Figure 3An example of a method for reconstructing data from a degraded RAID volume using accelerator engine circuitry, according to an example of this disclosure, is illustrated. Method 300 can use... Figure 1 Any of the components shown, such as host 110, RAID volume 120, and accelerator engine 130, individually or in combination, or any other component operable to implement method 300. Although examples have been described above, other variations and examples may be made in accordance with this disclosure without departing from the spirit and scope of these disclosed examples.

[0055] Method 300 begins at box 310, in which the RAID drive stack can initiate a request to a RAID volume (such as...) Figure 1 The striped read of RAID volume 120 (as shown) reveals that the RAID volume has been degraded due to a failure or error in one of the storage devices that make up the storage device. The RAID drive stack can be accessed via a memory bus (such as...). Figure 1 The memory bus 116 shown sends NVMe commands to the storage devices in the RAID volume and writes to the doorbell register in the storage device that has not yet failed. For example, see Reference Figure 1 If storage device 122b has failed, the RAID driver stack can write to the doorbell register in storage devices 122a and 122c. The doorbell register can notify pending commands to the still-functioning storage devices and trigger them to pull commands from the host, causing the still-functioning storage devices (e.g., storage devices 122a and 122c) to write striped data to the accelerator engine circuitry (such as...). Figure 1 The memory in the accelerator engine 130 shown is as described with respect to boxes 320 and 330.

[0056] At box 320, the storage device receiving the doorbell register write (at box 210) can at least partially respond to the doorbell register write by using the memory in the host (such as...) Figure 1 The memory read of the memory (114) shown is used to retrieve the first command. The first command may instruct the storage device to write striped data into the memory in the accelerator engine circuitry, as described with respect to box 330.

[0057] At box 330, a storage device receiving a write to the doorbell register can trigger direct memory access (DMA) to write striped data to memory in the accelerator engine circuitry. Memory in the accelerator engine circuitry may include a DDR region of the storage device exposed as a BAR.

[0058] At box 340, the storage device that receives the doorbell register write can send a first message to the RAID driver stack indicating that the command has been completed. This message can convey to the RAID driver stack that the storage device has completed writing the striped data to the memory in the accelerator engine circuitry, and the RAID driver stack can continue the data reconstruction process. The first message can be in the form of an interrupt, such as MSI or MSI-X, or a poll.

[0059] At box 350, the RAID driver stack can, at least in part, respond to the receipt of the first message (at box 340) by initiating a second command addressed to the accelerator engine circuitry by writing to the doorbell register in the accelerator engine circuitry. The second command can be an XOR command or a vendor-defined NVMe command.

[0060] At box 360, in response to a doorbell register write (box 350), the accelerator engine circuitry can retrieve a second command from the host. The second command can be in memory at the host or any BAR.

[0061] At block 370, the engine or circuitry in the accelerator engine circuitry may, at least in part, respond to the second command by performing an operation on striped data from the memory in the accelerator engine circuitry. This operation may be an XOR operation. The striped data used for the XOR operation may be data written to the memory in the accelerator engine circuitry at block 330.

[0062] At box 380, the accelerator engine circuitry can transfer the output of the operation (from box 370) to the host memory. The accelerator engine circuitry can also write the output of the operation to the host memory via DMA. The output of the operation can be a reconstruction of data stored on a faulty or erroneous storage device.

[0063] At box 390, the accelerator engine circuitry can transmit a second message to the host indicating that a command has been completed. This second message can be in the form of an interrupt, such as MSI or MSI-X, or a poll.

[0064] although Figure 3 A specific number of operations related to method 300 are disclosed, but method 300 is available in more... Figure 3 The number of operations described can be increased or decreased to perform the task. Furthermore, although... Figure 3 The specific order of operations to be performed in method 300 is disclosed, but the operations including method 300 can be performed in any suitable order.

[0065] Examples of this disclosure may include an article of manufacture. This article of manufacture may include non-transitory memory containing machine-readable instructions that, when executed by a processor, cause the processor to transmit a first command to a first storage device and a second storage device to trigger the first and second storage devices to write striped data into memory in an accelerator engine. The instructions may cause the processor to transmit a second command to the accelerator engine to perform an operation on the written striped data. This operation may be used to rebuild data stored on a third failed storage device. The first, second, and third storage devices may be part of a RAID volume. The instructions may cause the processor to receive the output of the operation from the accelerator engine.

[0066] Combining any of the examples above, this instruction enables the processor to use the PCIe bus, which includes the Peripheral Component Interconnect Fast (PCIe) root complex, to transmit a first command, transmit a second command, and receive output.

[0067] Combining any of the examples above, this operation could be an XOR operation on the written stripe data.

[0068] In conjunction with any of the examples above, the first storage device, the second storage device, and the accelerator engine communicate using peer-to-peer direct memory access.

[0069] Combining any of the examples above, the accelerator engine, as well as the first and second storage devices, can be PCIe endpoints.

[0070] Combining any of the examples above, this instruction can cause the processor to receive a message from both the first and second storage devices, indicating that the first and second storage devices have completed writing the striped data into the memory of the accelerator engine. The message can then trigger the processor to transmit a second command.

[0071] Examples of this disclosure may include a system. The system may include a memory bus, accelerator engine circuitry coupled to memory, and a host coupled to the memory bus. The host may include a processor and non-transitory media. The non-transitory media may include any of the non-transitory media of any of the examples above, such as those of the example article. Instructions may cause the processor to transmit a first command to a first storage device and a second storage device coupled to the memory bus to trigger the first and second storage devices to write striped data to memory in the accelerator engine circuitry, and to transmit a second command to the accelerator engine circuitry to perform an operation on the written striped data. This operation may be used to reconstruct data previously stored on a third failed storage device. The first, second, and third storage devices may be part of a RAID volume. Instructions may be used to receive the output of the operation from the accelerator engine circuitry.

[0072] In conjunction with any of the examples above, the memory bus can be a PCIe bus that includes a PCIe root complex.

[0073] Combining any of the examples above, the PCIe root complex can be used to route communication between the first and second storage devices and the memory in the accelerator engine circuitry based on the base address register (BAR) of the memory in the accelerator engine circuitry.

[0074] Combining any of the examples above, this operation could be an XOR operation on the written stripe data.

[0075] Combining any of the examples above, the first storage device, the second storage device, and the accelerator engine circuitry can communicate using peer-to-peer direct memory access.

[0076] Combining any of the examples above, the accelerator engine circuitry, as well as the first and second storage devices, can be PCIe endpoints.

[0077] In conjunction with any of the examples above, the processor can be used to receive a message from the first and second storage devices indicating that the first and second storage devices have completed writing striped data into the memory of the accelerator engine circuitry. This message can trigger the processor to transmit a second command.

[0078] Examples of this disclosure may include a method. This method may include operations of any of the examples described above. The method may include transmitting a first command to a first storage device and a second storage device to trigger the first and second storage devices to write striped data to memory in an accelerator engine, wherein the first and second storage devices are part of a RAID volume. The method may include transmitting a second command to the accelerator engine to perform an operation on the written striped data for rebuilding data stored on a third failed storage device of the RAID volume. The method may include receiving the output of the operation from the accelerator engine.

[0079] Combining any of the examples above, transmitting the first command, transmitting the second command, and receiving output can be performed using a PCIe bus that includes the Peripheral Component Interconnect Fast (PCIe) root complex.

[0080] Combining any of the examples above, the PCIe root complex can route communication between the first and second storage devices and the memory in the accelerator engine based on the base address register (BAR) of the memory in the accelerator engine.

[0081] Combining any of the examples above, the first command can instruct the first and second storage devices to communicate with the accelerator engine using peer-to-peer direct memory access.

[0082] Combining any of the examples above, the accelerator engine, as well as the first and second storage devices, can be a fast endpoint for interconnecting peripheral components.

[0083] Combining any of the examples above, the method may include receiving a message from a first storage device and a second storage device indicating that the first and second storage devices have completed writing striped data into the memory of the accelerator engine. This message may trigger the transmission of a second command.

[0084] Although examples have been described above, other variations and examples may be made in accordance with this disclosure without departing from the substance and scope of these disclosed examples.

Claims

1. An article comprising: Non-transitory memory, the non-transitory memory including machine-readable instructions, which, when executed by a processor, cause the processor to: A first command is sent to the first storage device and the second storage device to trigger the first storage device and the second storage device to write striped data into the memory in the accelerator engine; A second command is sent to the accelerator engine to perform an operation on the written striped data, the operation being used to rebuild data stored on a third failed storage device, wherein the first storage device, the second storage device, and the third storage device are part of a RAID volume; as well as Receive the output of the operation from the accelerator engine.

2. The article of manufacture according to claim 1, wherein the processor is configured to use a PCIe bus including a Peripheral Component Interconnect Fast (PCIe) root complex to transmit the first command, transmit the second command, and receive the output.

3. The article of manufacture according to any one of claims 1 to 2, wherein the operation is an XOR operation on the written strip data.

4. The article of manufacture according to any one of claims 1 to 3, wherein the first storage device and the second storage device and the accelerator engine communicate using peer-to-peer direct memory access.

5. The article of manufacture according to claim 4, wherein the accelerator engine and the first and second storage devices are PCIe endpoints.

6. The article of manufacture according to any one of claims 1 to 5, wherein the processor is configured to receive a message from the first storage device and the second storage device, the message indicating that the first storage device and the second storage device have completed writing striped data into the memory in the accelerator engine; and The message triggers the processor to transmit the second command.

7. A method, the method comprising: A first command is sent to a first storage device and a second storage device to trigger the first storage device and the second storage device to write striped data to memory in the accelerator engine, wherein the first storage device and the second storage device are part of a RAID volume; A second command is sent to the accelerator engine to perform an operation on the written striped data, the operation being used to rebuild data stored on a third faulty storage device of the RAID volume; as well as Receive the output of the operation from the accelerator engine.

8. The method of claim 7, wherein transmitting the first command, transmitting the second command, and receiving the output are performed using a PCIe bus including a Peripheral Component Interconnect Fast (PCIe) root complex.

9. The method of claim 8, wherein the PCIe root complex is used to route communications from the first storage device and the second storage device to the memory in the accelerator engine based on the base address register (BAR) of the memory in the accelerator engine.

10. The method according to any one of claims 7 to 9, wherein the operation is an XOR operation on the written stripe data.

11. The method according to any one of claims 7 to 10, wherein the first command instructs the first storage device and the second storage device to communicate with the accelerator engine using peer-to-peer direct memory access.

12. The method according to any one of claims 7 to 11, wherein the accelerator engine and the first and second storage devices are peripheral component interconnect fast endpoints.

13. The method of any one of claims 7 to 12, the method comprising receiving a message from the first storage device and the second storage device, the message indicating that the first storage device and the second storage device have completed writing striped data into the memory in the accelerator engine; and The message mentioned above triggers the transmission of the second command.

14. A system comprising: Memory bus; An accelerator engine circuit, the accelerator engine circuit being coupled to the memory bus; and A host computer coupled to the memory bus, the host computer including a processor and non-transitory memory, the non-transitory memory including machine-readable instructions, the machine-readable instructions causing the processor, when executed by the processor, to: A first command is transmitted to a first storage device and a second storage device coupled to the memory bus to trigger the first storage device and the second storage device to write striped data into the memory in the accelerator engine circuit. A second command is sent to the accelerator engine circuit to perform an operation on the written striped data, the operation being used to reconstruct data previously stored on a third failed storage device, wherein the first storage device, the second storage device, and the third storage device are part of a RAID volume; as well as The output of the operation is received from the accelerator engine circuit.

15. The system of claim 14, wherein the memory bus is a PCIe bus including a Peripheral Component Interconnect Fast (PCIe) root complex.

16. The system of claim 15, wherein the PCIe root complex is used to route communications from the first storage device and the second storage device to the memory in the accelerator engine circuit based on the base address register (BAR) of the memory in the accelerator engine circuit.

17. The system according to any one of claims 14 to 16, wherein the operation is an XOR operation on the written striped data.

18. The system according to any one of claims 14 to 17, wherein the first storage device and the second storage device and the accelerator engine circuit communicate using peer-to-peer direct memory access.

19. The system according to any one of claims 14 to 18, wherein the accelerator engine circuitry and the first and second storage devices are peripheral component interconnect fast endpoints.

20. The system according to any one of claims 14 to 19, wherein the processor is configured to receive a message from the first storage device and the second storage device, the message indicating that the first storage device and the second storage device have completed writing striped data into the memory in the accelerator engine circuitry; and The message triggers the processor to transmit the second command.

Images