Storage System

Abstract

The present invention provides a storage system for supporting data protection on a field programmable gate array (FPGA) solid-state drive (SSD), including a storage system having a first group of solid-state drives connected to the field programmable gate array. The field programmable gate array includes a first data protection controller, and the first data protection controller is configured to perform the following operations: manage input / output requests to and from the first group of solid-state drives according to a data protection configuration; generate parity bits according to the data protection configuration; and store the parity bits on at least a parity solid-state drive from the first group of solid-state drives.

Description

Technical Field

[0001] Some embodiments of the present disclosure generally relate to non-volatile memory express (NVMe) solid state drives (SSD), field programmable gate arrays (FPGA), and redundant array of independent disks (RAID). Background Art

[0002] The rapid development of data has promoted the corresponding development of data centers and the increasing use of cloud computing. 1U and 2U servers are becoming more and more popular in order to maximize the use of limited data center space. Supporting cloud computing requires data centers to maintain a high level of reliability while allowing simple system configuration. In many cases, cloud computing involves the use of relatively high-performance servers that can each operate multiple virtual machines. The virtual machines can then divide the resources of the server on which the virtual machines operate.

[0003] RAID is an important tool for providing data protection and improving overall system performance. Implementing RAID with NVMe-based SSDs typically requires the use of external PCIe add-in-cards (AICs) (e.g., RAID controllers). Due to the physical limitations of the standard form factor of PCIe cards and the limited number of available connections (e.g., U.2 connections), adding AICs to 1U and 2U servers can be problematic. For example, each RAID controller can be configured to support 4 NVMe SSDs. A typical 2U chassis contains 24 NVMe SSDs, which in turn require 6 AICs and form 6 RAID domains. The inclusion of these AICs and domains increases the overall cost (e.g., in terms of hardware cost and power consumption) and increases the complexity of configuring and using the system.

[0004] Adding RAID AICs to a 1U chassis can create additional issues of space and complexity. For example, only a limited number of RAID AICs can be physically added to a 1U chassis. Therefore, the only way to configure each drive in a RAID setting is to use a completely software-based system, or to use a combination of hardware and software-based RAID. Both of these setups result in reduced performance and increased complexity.

[0005] Erasure coding is another data protection technology that is becoming increasingly popular due to its configurability. Erasure coding allows data to be split into multiple smaller coded blocks, where the original data can be recovered from a subset of the coded blocks. The system administrator defines how many fragments the data is divided into and the number of fragments required to recover the original data after detecting data corruption (e.g., parity). In the storage system, the fragments are distributed to the corresponding drives. For example, if the erasure code is configured as a 10 / 16 system, the data will be split into 16 fragments and 10 fragments can be used for reconstruction. The system will split the 16 fragments on 16 drives / nodes / etc. Although providing increased flexibility, erasure coding can be computationally expensive and can also result in additional network usage. In addition, implementing erasure coding can further increase system configuration complexity.

[0006] The above information is only for enhancement of understanding of the background of the embodiments of the present disclosure and therefore may contain information that does not form prior art. Summary of the invention

[0007] [Technical issues]

[0008] One object of the present invention relates to providing a storage system that employs data protection functionality while reducing cost and complexity.

[0009] [Problem Solution]

[0010] Some embodiments of the present disclosure provide a system and method for data protection on a field programmable gate array (FPGA) solid-state disk (SSD). In various embodiments, the system includes a storage system having a first group of SSDs connected to a first FPGA. The first FPGA includes a first data protection controller, which is configured to perform the following operations: manage input / output requests to and from the first group of solid-state disks according to a data protection configuration; generate parity bits according to the data protection configuration; and store the parity bits on at least a parity solid-state drive from the first group of solid-state drives.

[0011] In various embodiments, the storage system further includes a processor connected to the first FPGA via a high-speed bus and a BMC connected to the first FPGA via the high-speed bus, wherein the BMC is configured to provide a data protection configuration.

[0012] In various embodiments, the storage system further includes a processor connected to the first FPGA via a high-speed bus and a BMC connected to the first FPGA via a low-speed bus, wherein the BMC is configured to provide a data protection configuration.

[0013] In various embodiments, the storage system further includes a first switch connecting the first FPGA to the first group of solid-state drives.

[0014] In various embodiments, the storage system further includes a second set of solid-state drives connected to a second FPGA, the second FPGA connected to the second set of solid-state drives via a second switch. The second FPGA includes a second data protection controller, and the first FPGA and the second FPGA are connected by a connector between the first switch and the second switch.

[0015] In various embodiments, the first data protection controller sends the replicated data to the second data protection controller via the first switch and the second switch.

[0016] In various embodiments, the first data protection controller includes a redundant array of independent disks (RAID) controller.

[0017] In various embodiments, the first data protection controller includes an erasure code controller.

[0018] In various embodiments, a storage system includes a first group of SSDs, a second group of SSDs, an FPGA connected to the first group of SSDs, and a second FPGA connected to the second group of SSDs. The FPGA includes a first redundant array of independent disks (RAID) controller, and the second FPGA includes a second RAID controller. The first FPGA and the second FPGA are configured to: manage input / output requests to and from the first group of solid-state disks and the second group of solid-state disks according to a data protection configuration; generate parity bits for stored data according to the data protection configuration; store the parity bits on at least one parity SSD from the first group of SSDs; and replicate the first group of solid-state drives on the second group of solid-state drives.

[0019] In various embodiments, the storage system further includes a processor connected to the first FPGA and the second FPGA through a bus and a BMC connected to the first FPGA and the second FPGA through the bus, wherein the BMC is configured to provide a data protection configuration.

[0020] In various embodiments, the bus comprises a peripheral component interconnect express (PCIe) bus.

[0021] In various embodiments, the storage system further includes a first PCIe switch connecting the first FPGA to the first set of solid-state drives and a second PCIe switch connecting the second FPGA to the second set of solid-state drives, wherein the first switch and the second switch connect the first FPGA to the second FPGA.

[0022] In various embodiments, the data protection configuration is a hybrid RAID configuration.

[0023] In various embodiments, a storage system includes a first group of solid-state drives and a first FPGA connected to the first group of SSDs, a processor connected to the first FPGA via a bus, and a BMC connected to the first FPGA via a bus and configured to provide a data protection configuration for the first FPGA. The first FPGA includes a first data protection controller, which is configured to perform the following operations: manage input / output requests to and from the first group of solid-state drives according to the data protection configuration; generate parity bits for data stored in the first group of solid-state drives according to the data protection configuration; and store the parity bits on at least one parity solid-state drive from the first group of solid-state drives.

[0024] In various embodiments, the first data protection controller includes a redundant array of independent disks (RAID) controller.

[0025] In various embodiments, the data protection configuration includes at least one of a RAID 1, RAID 5, or RAID 6 configuration.

[0026] In various embodiments, the first data protection controller includes an erasure coding controller configured to stripe data on a first portion of the first plurality of solid-state drives and to generate erasure codes based on data on a second portion of the first plurality of solid-state drives.

[0027] In various embodiments, the system further includes a second group of solid-state drives and a second FPGA, the second FPGA is connected to the processor through a bus, to the BMC through a bus, and to the second group of solid-state drives. The second FPGA includes a second data protection controller, the second data protection controller is configured to perform the following operations: manage input / output requests to and from the second group of solid-state disks according to the data protection configuration; generate parity bits for data stored in the second group of solid-state drives according to the data protection configuration; and store the parity bits on at least one parity solid-state drive from the second group of solid-state drives.

[0028] In various embodiments, the system includes a second plurality of solid-state drives, a first peripheral component interconnect express (PCIe) switch connecting a first FPGA to a first downstream PCIe switch and a second downstream PCIe switch. The first downstream PCIe switch is connected to a first set of solid-state drives and connects the first set of solid-state drives to the first FPGA via the first PCIe switch, and the second downstream PCIe switch is connected to a second set of solid-state drives and connects the second set of solid-state drives to the first FPGA via the first PCIe switch.

[0029] In various embodiments, the data protection configuration includes a hybrid RAID configuration.

[0030] [Advantageous effects of the present invention]

[0031] According to the present invention, a storage system providing a data protection function while reducing cost and complexity is provided by adopting a field programmable gate array having a data protection function and connecting them using a switch. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Some embodiments may be understood in more detail through the following description taken in conjunction with the accompanying drawings, in which:

[0033] Figure 1A and Figure 1B A storage system with an FPGA SSD according to various embodiments is depicted.

[0034] Figure 2 An embodiment including an FPGA with a PCIe switch is depicted in accordance with various embodiments.

[0035] Figure 3 A storage system including multiple FPGAs connected to an SSD is depicted according to various embodiments.

[0036] Figure 4 Depicted is an example storage system with multiple PCIe switches in accordance with various embodiments.

[0037] Figure 5 An example storage system with a multilayer switch is depicted in accordance with various embodiments.

[0038] Figure 6 Methods of performing system initialization according to various embodiments are described.

[0039] Figure 7 Methods of pre-processing an SSD according to various embodiments are described.

[0040] Figure 8 Methods of generating parity for performing I / O according to various embodiments are described.

[0041] Fig. 9 Methods of performing heat removal and addition after drive failure are described in accordance with various embodiments.

[0042] Description of Figure Numbers

[0043] 100A, 100B, 200, 400, 500: storage system;

[0044] 110: processor;

[0045] 120: memory;

[0046] 130: mainboard management control unit;

[0047] 140, 150, 220, 310, 320, 410, 420, 510: FPGA;

[0048] 145A, 155A: Data protection controller / RAID controller;

[0049] 145B, 155B: Data protection controller / erasure coding controller;

[0050] 160~174, 230~252, 330~344, 430~460, 540~570: NVMe SSD;

[0051] 210: Drive visible to host;

[0052] 300, 414, 424, 514, 520, 530: PCIe switches;

[0053] 315, 325: Erasure code controller;

[0054] 412, 422, 512: data protection controller;

[0055] 514: a first PCIe switch;

[0056] 520: a first downstream PCIe switch;

[0057] 530: a second downstream PCIe switch;

[0058] S600, S610, S620, S630, S700, S710, S720, S730, S800, S810, S820, S830, S840, S850, S860, S870, S900, S910, S920, S930: steps. DETAILED DESCRIPTION

[0059] The features of the inventive concept and the methods for realizing the features can be more easily understood by referring to the following detailed description of the embodiments and the accompanying drawings. Hereinafter, the embodiments will be described in more detail with reference to the accompanying drawings, in which the same reference numerals refer to the same elements throughout the text. However, the present invention can be embodied in various different forms and should not be construed as being limited to the embodiments described herein only. On the contrary, these embodiments are provided as examples so that the present disclosure will be thorough and complete, and the various aspects and features of the present invention will be fully conveyed to those skilled in the art. Therefore, it may not be described that processes, elements and technologies that are not necessary for a complete understanding of the aspects and features of the present invention by those of ordinary skill in the art are described. Unless otherwise noted, the same reference numerals represent the same elements throughout the accompanying drawings and written descriptions, and therefore their descriptions will not be repeated. In the accompanying drawings, for clarity, the relative sizes of the elements, layers and regions may be magnified.

[0060] In the following description, for the purpose of explanation, numerous specific details are set forth to provide a thorough understanding of various embodiments. However, it is apparent that various embodiments can be practiced without these specific details or with one or more equivalent arrangements. In other cases, well-known structures and devices are shown in block diagram form in order to avoid unnecessary obfuscation of various embodiments.

[0061] It should be understood that when an element, layer, region, or component is referred to as being "on," "connected to," or "coupled to" another element, layer, region, or component, it may be directly on, connected to, or coupled to another element, layer, region, or component, or there may be one or more intervening elements, layers, regions, or components. However, "directly connected / directly coupled" refers to one component being directly connected or coupled to another component without an intermediate component. At the same time, other expressions describing the relationship between components, such as "between," "immediately between," or "adjacent to," and "immediately adjacent to," may be similarly understood. In addition, it will also be understood that when an element or layer is referred to as being "between" two elements or two layers, it may be the only element or layer between the two elements or two layers, or there may also be one or more intervening elements or intervening layers.

[0062] The terms used herein are only used for the purpose of describing specific embodiments, and are not intended to limit the present invention. As used herein, unless the context clearly indicates otherwise, the singular form "a / an" is also intended to include the plural form. It will be further understood that the terms "comprises / comprising", "have / having", "includes / including" when used in this specification represent the presence of stated features, integral bodies, steps, operations, elements and / or components, but do not exclude the presence or increase of one or more other features, integral bodies, steps, operations, elements, components and / or groups thereof. As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items.

[0063] As used herein, the terms "substantially", "about", "approximately" and similar terms are used as terms of approximation and not as terms of degree, and are intended to take into account the inherent deviations in the measured or calculated values ​​that will be recognized by those of ordinary skill in the art. Taking into account the measurement in question and the errors associated with the measurement of a particular amount (i.e., the limitations of the measurement system), as used herein, "about" or "approximately" includes the stated value and means within the acceptable range of the deviation of the particular value determined by those of ordinary skill in the art. For example, "about" may mean within one or more standard deviations, or within ±30%, 20%, 10%, 5% of the stated value. In addition, when describing embodiments of the present invention, the use of "may" refers to "one or more embodiments of the present invention". As used herein, the terms "use", "using", and "used" may be considered synonymous with the terms "utilize", "utilizing", and "utilized", respectively. In addition, the term "exemplary" is intended to refer to an example or illustration.

[0064] When a certain embodiment can be implemented in different ways, the specific processing order can be performed differently from the described order. For example, two processes described in succession can be performed substantially simultaneously or in the reverse order of the described order.

[0065] Various embodiments are described herein with reference to cross-sectional illustrations as schematic illustrations of embodiments and / or intermediate structures. Thus, differences from the illustrated shapes as a result of, for example, manufacturing techniques and / or tolerances should be expected. In addition, for the purpose of describing embodiments according to the concepts of the present disclosure, the specific structural or functional descriptions disclosed herein are illustrative only. Therefore, the embodiments disclosed herein should not be understood as being limited to the specific illustrated shapes of the regions, but rather include shape deviations caused by, for example, manufacturing. For example, an implant region illustrated as a rectangle will typically have a circular or curved feature and / or a gradient of implant concentration on its edge, rather than a binary change from an implant region to a non-implant region. Similarly, a buried region formed by implantation may produce some implantation in the region between the buried region and the surface where the implantation occurs. Therefore, the regions illustrated in the drawings are schematic in nature and their shapes are not intended to illustrate the actual shape of the region of the device and are not intended to be restrictive.

[0066] The electronic or electronic devices and / or any other related devices or components according to the embodiments of the present invention described herein may be implemented using any suitable hardware, firmware (e.g., application specific integrated circuits), software, or a combination of software, firmware, and hardware. For example, the various components of these devices may be formed on an integrated circuit (IC) chip or on a separate IC chip. In addition, the various components of these devices may be implemented on a flexible printed circuit film, a tape carrier package (TCP), a printed circuit board (PCB), or formed on a substrate. In addition, the various components of these devices may be processes or threads that run on one or more processors in one or more computing devices, execute computer program instructions, and interact with other system components to perform the various functions described herein. Computer program instructions are stored in a memory that can be implemented in a computing device using a standard memory device such as a random access memory (RAM). Computer program instructions may also be stored in other non-temporary computer-readable media such as a CD-ROM, a flash drive, or the like. Furthermore, those skilled in the art will recognize that the functionality of various computing devices may be combined or integrated into a single computing device, or that the functionality of a particular computing device may be distributed among one or more other computing devices without departing from the spirit and scope of the exemplary embodiments of the invention.

[0067] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by ordinary technicians in the field to which the invention belongs. It will be further understood that terms (such as those defined in commonly used dictionaries) should be interpreted as having a meaning consistent with their meaning in the context of the relevant technology and / or this specification, and should not be interpreted in an idealized or overly formal sense unless so explicitly defined in this article.

[0068] Embodiments of the present invention include systems and methods for supporting data protection in a storage system. In various embodiments, the storage system includes at least one FPGA configured to operate in conjunction with one or more solid-state drives (SSDs), such as non-volatile memory express (NVMe) SSDs. In various embodiments, the FPGA may be configured to perform computing operations (e.g., in-storage computing operations) on data stored in the attached SSDs. In various embodiments, each FPGA may be configured with a data protection controller that is used to protect the integrity of data stored in the storage system. For example, in various embodiments, the data protection controller may implement RAID or erasure coding of the attached SSDs, rather than using an AIC or software-based implementation. In various embodiments, each FPGA includes computing resources and high-speed input / output (I / O) connectivity for performing RAID as well as erasure code parity generation and data routing.

[0069] In modern data centers, the current goal is to maximize storage density and performance while minimizing power consumption. Typical RAID configurations utilize one or more external PCIe add-in cards (AICs). In order to maximize storage density, modern and future storage server form factors are designed to contribute as much space as possible to data storage drives (e.g., SSDs). Therefore, 1U and 2U form factors are becoming more and more popular. These form factors are not without disadvantages. For example, their small size does not provide much space to add AICs, the number of PCIe ports may be limited, and their increased density may make it more difficult to cool. Therefore, it is desirable to limit the number of AICs added to the system.

[0070] Current data protection implementations use AICs to perform RAID or use processing power to perform erasure coding. These approaches have multiple disadvantages. For example, AICs are more expensive, consume extra power, and generate extra heat while also using bandwidth on the PCIe bus. In addition, if every PCIe slot is utilized to implement RAID, the chassis will have no room for future expansion. Erasure coding provides many of the same benefits as RAID and can provide even more configuration flexibility, however, erasure coding is computationally expensive because most of the coding is done by the server's processor.

[0071] Embodiments of the present disclosure improve upon these previous approaches by utilizing the processing power of the FPGA to which the storage is attached. By implementing a data protection controller in the FPGA, a data protection scheme can be employed without having to use every available PCIe slot (thus allowing for future system expansion) and saving costs in terms of AIC, power, cooling, and processor workload. In addition, bandwidth on the primary PCIe bus can be used for other operations. Thus, storage density and performance can be increased while reducing power consumption. Additionally, the use of a data protection controller allows for greater configuration flexibility because the data protection controller is implemented in the FPGA.

[0072] Figure 1A and Figure 1B A storage system with an FPGA SSD according to various embodiments is depicted.

[0073] refer to Figure 1A and Figure 1BIn various embodiments, the storage system 100A / 100B is configured to operate using a data protection configuration implemented in conjunction with a data protection controller operation on an FPGA. In various embodiments, the storage system 100A is configured to operate using RAID, and the storage system 100B is configured to operate using erasure coding. The memory systems 100A / 100B each include one or more processors 110, a memory 120, a motherboard management control unit BMC 130, an FPGA 140, an FPGA 150, and SSDs 160 to 174 (e.g., NVMe SSDs, cross-architecture NVMe (NVMe-oF) or any other solid-state or rotating disk drives). In various embodiments, the processor 110 is connected to the NVMe SSDs 160 to 174 via the FPGAs 140 and 150. Although the processor 110 is configured to handle most of the primary computing operations, in various embodiments, the FPGAs 140, 150 are configured to perform a variety of computing operations (e.g., in-storage computing operations) on data stored in the corresponding attached SSDs 160 to 174. The FPGAs 140, 150 are connected to the processor 110 via a high-speed bus (e.g., PCIe x8 / x16) that is configured to transfer data to and from the FPGAs 140, 150 and to transfer commands executed by the FPGAs 140, 150. In some embodiments, the FPGAs 140, 150 are directly connected to the BMC 130 via a relatively low-speed bus (e.g., PCIe x1) to perform configuration and control operations on the FPGAs 140, 150 and the attached SSDs 160 to 174. In various embodiments, FPGA 140, FPGA 150 may each include one or more data protection controllers 145A / B, data protection controllers 155A / B. In various embodiments, data protection controllers 145A / B, data protection controllers 155A / B may be implemented by FPGA 140, FPGA 150. In other embodiments, data protection controllers 145A / B, data protection controllers 155A / B may be implemented as application-specific integrated circuits (ASICs) on the same die as FPGA 140, FPGA 150.

[0074] In various embodiments, the BMC 130 may be used to specify a data protection configuration for the data protection controller 145A / B, the data protection controller 155A / B. In some embodiments, the BMC 130 may have a hard-coded path, where a specific data protection configuration is automatically applied when the system is initialized. For example, the BMC 130 may apply the data protection configuration according to a basic input and output system (BIOS) configuration or a software interface. In other embodiments, the administrator has the option of setting a preset or hard-coded setting by modifying the BIOS or interfacing software. In various embodiments, the administrator selects different data protection configurations to be used during the configuration step. For example, the administrator may choose to implement erasure coding or RAID according to its application. In some embodiments, the BMC 130 may be used to configure multiple data protection controllers 145A / B, data protection controllers 155A / B on each FPGA 140, FPGA 150. For example, in some embodiments, the server 100A / 100B may operate multiple virtual machines that may be utilized by multiple users, and the data protection controller 145A / B, data protection controller 155A / B may be implemented for each virtual machine.

[0075] In various embodiments, each FPGA 140, FPGA 150 may operate with a number of SSDs 160-174. In the depicted example, each FPGA 140, FPGA 150 is configured to operate with four SSDs, however, it should be understood that the ratio of FPGAs to SSDs may be modified to include one FPGA for each SSD to multiple SSDs for each FPGA.

[0076] refer to Figure 1A In various embodiments, each FPGA 140, FPGA 150 has a data protection controller that operates a data protection scheme according to a data protection configuration (e.g., a configuration specified by BMC 130). For example, in various embodiments, the data protection controller may be configured to operate as a RAID controller 145A, RAID controller 155A. In various embodiments, a user (e.g., a system administrator) may configure the RAID controller 145A, RAID controller 155A to implement a desired RAID configuration (e.g., RAID 0, RAID 1, RAID 5, RAID 6, etc.) and specify the drives 160 to 174 to be used (e.g., creating a group based on the total number of drives connected to the RAID controller 145A, RAID controller 155A).

[0077] For example, in various embodiments, FPGA 140, FPGA 150 may each be connected to four SSDs 160 to SSD 174 and configured to each operate in a RAID 5 configuration, where data is striped across three drives and the fourth drive is used for parity. In another example, FPGA 140, FPGA 150 may be connected to five SSDs 160 to SSD 174 and configured to each operate in a RAID 6 configuration.

[0078] In various embodiments, the RAID controller 145A, RAID controller 155A is managed by the BMC 130. Therefore, the system administrator can access the BMC 130 for initial configuration of the RAID controller 145A, RAID controller 155A and for performing any changes in configuration, performance monitoring, performing system maintenance, etc. For example, the BMC 130 may provide a user interface (e.g., a graphical user interface or a command line interface) for configuration, and may provide a software interface for some configurations.

[0079] refer to Figure 1B In various embodiments, the data protection controller of each FPGA 140, FPGA 150 operates according to the data protection configuration to specify the erasure code operation. For example, in various embodiments, the data protection controller can be operated as an erasure coding controller 145B, an erasure coding controller 155B. In various embodiments, the erasure coding controller 145B, the erasure coding controller 155B are each configured to operate in an M / N erasure coding configuration (for example, data is split across M drives and can be recovered using only N drives). For example, in various embodiments, the FPGA 140, FPGA 150 can each be connected to 4 SSDs 160 to SSD 174 and configured to operate in a 1 / 4, 2 / 4, or 3 / 4 configuration. In another example, each of the FPGA 140, FPGA 150 can be connected to 8 SSDs 160 to SSD 174 and configured to each operate in, for example, a 5 / 8 configuration. It should be understood that the erasure coding configuration and the number of SSDs connected to the FPGA can be modified according to the available SSDs and user design requirements.

[0080] In various embodiments, the configuration of the erasure coding controller 145B, the erasure coding controller 155B is managed by the BMC 130. The BMC 130 may provide a user interface to perform initial settings, check device status, perform equipment maintenance, modify configuration, etc. Therefore, the administrator can easily configure the storage system to have the data protection scheme required by the administrator.

[0081] Figure 2 An embodiment including an FPGA with a PCIe switch is depicted in accordance with various embodiments.

[0082] refer to Figure 2 In various embodiments, the storage system 200 may be configured with a variety of data protection schemes. In various embodiments, the attached NVMe SSD 230 to the NVMe SSD 252 may be viewed by the host as a single host-visible drive 210. Therefore, from the host's perspective, the configuration of the storage system 200 is not a problem in operation. In various embodiments, the FPGA 220 may be attached to an integrated PCIe switch or have an integrated PCIe switch. In various embodiments, adding a PCIe switch provides further configuration flexibility and allows the use of fewer FPGAs 220, and thus further reduces system cost. In addition, the business on the system PCIe bus can be reduced because RAID / erasure code operations and any in-storage computing operations of the FPGA 220 are limited to a separate PCIe bus served by the PCIe switch.

[0083] In various embodiments, the storage system 200 may be configured (e.g., by an administrator accessing the BMC 130) to operate using RAID 5, using an X / Y erasure code configuration, or using RAID 0. In the depicted embodiment, the storage system 200 includes 12 attached SSDs 230 to SSD 252. Thus, when the storage system 200 is configured in a RAID 5 configuration, 11 drives 230 to drive 250 may be used to store data (e.g., data is striped across the 11 drives), and 1 drive 252 is used to store parity bits. Similarly, when the storage system 200 is configured to operate using erasure codes, x drives may be used to store data, and y drives may be used to store code. In another embodiment, the system may be configured as RAID 0 (without parity stripes), and all drives are used as data drives.

[0084] Figure 3 A storage system including multiple FPGAs connected to an SSD is depicted according to various embodiments.

[0085] refer to Figure 3 In various embodiments, the storage system may further include one or more PCIe switches 300 connected to or operating on each FPGA 310, FPGA 320. The PCIe switch 300 may be used to increase the number of NVMe drives 330 to NVMe drives 344 to which each FPGA 310, FPGA 320 is connected to provide additional data protection configurations, and the NVMe drives 330 to NVMe drives 344 connect two or more than two FPGAs 310, FPGA 320. Therefore, additional data security layers and configurability may be selected by an administrator.

[0086] For example, in the depicted embodiment, each FPGA 310 or FPGA 320 is connected to four NVMe SSDs 330 to NVMe SSD 344 using a PCIe switch 300. In addition, the FPGAs 310, FPGA 320 are also connected to each other via the PCIe switch 300. Therefore, when the FPGAs 310, FPGA 320 operate the RAID or erasure code controller 315, RAID or erasure code controller 325, the RAID or erasure code configuration can be incorporated with additional FPGAs and their associated drivers, and further configuration flexibility is provided. For example, in various embodiments, the first FPGA 310 can be paired with one or more FPGAs, such as the second FPGA 320. Additional FPGAs (e.g., FPGA 320) and their corresponding drivers (e.g., NVMe SSD 338 to NVMe SSD 344) can be used for data replication (e.g., data mirroring), thereby providing an additional layer of data security. In various embodiments, multiple FPGAs and their corresponding drivers may be used to implement hybrid RAID or nested RAID configurations, such as RAID 10, RAID 50, etc.

[0087] Figure 4 Depicted is an example storage system with multiple PCIe switches in accordance with various embodiments.

[0088] refer to Figure 4 In various embodiments, each FPGA 410, FPGA 420 of the storage system 400 may include a switch 414, switch 424 (e.g., a PCIe switch). In various embodiments, the PCIe switch 414, PCIe switch 424 coupled to the data protection controller 412, data protection controller 422 (e.g., a RAID or erasure code controller) provides additional configuration flexibility. For example, in the depicted embodiment, the use of the PCIe switch 414, PCIe switch 424 allows more drives to be logically connected to each FPGA 410, FPGA 420. For example, in the depicted embodiment, each of the FPGA 410 or FPGA 420 is connected to 8 NVMe SSDs 430 to NVMe SSD 460. In other embodiments, the FPGA 410, FPGA 420 may be logically connected to more or fewer drives. Furthermore, as discussed above, a hybrid RAID configuration may be employed using two or more FPGAs 410, FPGA 420, and their associated drivers 430-460.

[0089] Figure 5 An example storage system with a multilayer switch is depicted in accordance with various embodiments.

[0090] See also Figure 5 In various embodiments, the storage system 500 may include a single FPGA 510 connected to a plurality of NVMe SSDs 540 to NVMe SSDs 570 through a plurality of PCIe switches 514, 520, and 530. In various embodiments, the FPGA 510 may include a data protection controller 512 and be logically connected to a first PCIe switch 514, the data protection controller 512 being configured to perform RAID and / or erasure code operations. The first PCIe switch 514 (e.g., at the first layer) may connect the FPGA 510 to two or more downstream PCIe switches 520, 530 (e.g., at the second layer), each of which is connected to a portion of the plurality of SSDs 540 to 570. In the depicted example, the first downstream PCIe switch 520 may be logically connected to eight SSDs 540 to 554, and the second downstream PCIe switch 530 may be logically connected to eight SSDs 556 to 570. The use of additional PCIe switches allows for further configuration options for protecting data stored on multiple NVMe SSDs 540 to 570. As described above with reference to Figure 4 As discussed, multiple PCIe switches can be used to implement a hybrid RAID configuration. For example, RAID 10 can be implemented by having the drives under the first downstream switch 520 implement RAID 1 (mirroring) between each drive and striping these drives to the drives connected to the second downstream switch 530. In another arrangement, the first downstream switch 520 and the second downstream switch 530 can implement RAID 5, and they can be striped using RAID 0 to form RAID 50. It should be understood that these RAID implementations are used as examples, and any hybrid RAID implementation can be configured using an appropriate number of drives and switches.

[0091] Figure 6 Methods of performing system initialization according to various embodiments are described.

[0092] refer to Figure 6 as well as Figure 1A and Figure 1BIn various embodiments, the BMC 130 may be configured to perform an initial setup of a data protection scheme employed by the storage system 100A / 100B. In various embodiments, the storage system 100A / 100B may be initialized for use by an administrator. For example, the storage system 100A / 100B may be a cloud storage server that has applied for client and demand configuration. In various embodiments, the BMC 130 initializes and pre-processes the SSD according to the RAID or erasure code configuration (S600) (as will be referred to in reference Figure 7 , discussed above). For example, as discussed above, the BMC 130 may have a hard-coded path or a configurable setting in which a specific data protection scheme is also applied. Based on the configuration provided by the BMC 130, the data protection controller 145A / B, the data protection controller 155A / B represents the data drives and parity (or erasure code) drives that meet the requirements of the specified data protection scheme (S610). In various embodiments, the data protection controller 145A / B, the data protection controller 155A / B may normalize any attached data drives or parity drives with different capacities (S620). The controller 145A / B, the controller 155A / B may then make the parity drive invisible to any subject (S630) (e.g., it is removed from the total capacity of visible drives).

[0093] Figure 7 Methods of pre-processing an SSD according to various embodiments are described.

[0094] Refer to 7 and continue to refer to Figure 1A and Figure 1BIn various embodiments, after initializing system 100A / B, the system can pre-process the drive by erasing all the content stored thereon. For example, in cloud computing applications, there may be multiple system users, each of which has private data stored on the attached drive. Before the storage server can be used by another user, any previous user data should be safely deleted. In conventional systems, erasing data from a large drive of a remote host may be a very long process because it usually involves writing multiple times on each storage bit. As the drive capacity increases, this operation is then increased during the period. In various embodiments, storage system 100A / B can speed up data erasure processing by using BMC 130 and FPGA 140, FPGA 150 to distribute tasks. For example, before assigning a drive to a new user, an erase command (S700) can be received from an administrator at BMC 130. In various embodiments, an erase command can be used for a specific namespace or range. The BMC 130 may then determine which FPGAs are associated with the command (e.g., allocated to the drivers of the host sending the command) and broadcast the command using a low-speed bus connected to each FPGA 140, FPGA 150 (S710). The erase command is then forwarded to each of the target drives (S720). Each drive processes the command and erases its contents (e.g., previous user data) (S730). Thus, the erase commands are executed in parallel and the total execution time can be greatly reduced. In addition, offloading the erase of the BMC / FPGA allows the system central processor (CPU) to be free to handle other tasks. In addition, a secure channel is formed using the data path (e.g., the BMC path) for maintenance operations.

[0095] Figure 8 Methods of generating parity for performing I / O according to various embodiments are described.

[0096] refer to Figure 8 In various embodiments, the storage system is configured to generate parity bits according to a data protection scheme. After the system is initialized, the system may begin with some configuration steps. For example, each attached drive (e.g., drive 160 to drive 174) may include a unique member identification, such as a slot number or an assigned identification number from a data protection controller (e.g., data protection controller 145A / B) (S800). The drive may then be available for general transactions (S810). For example, a user or application may use the BMC to access the system and select a data protection scheme type, and the drive may be designated as a data drive or a parity drive (e.g., by data protection controller 145A / B). After being designated as a parity drive, the drive may no longer be visible to the host or upper-level data protection controller (e.g., in a controller such as Figure 5). After initialization, the drive may then be discovered by the host and I / O transactions may begin (S820). The host may then begin writing data to the drive, and in some embodiments, the drive may be configured to identify coefficients to be used to generate parity values ​​(S830). The drive may then use the coefficients to generate a delta product for use by one or more parity drives (S840). For example, the drive may perform a read of a target logical block address (LBA), XOR the read data with new data, multiply by the coefficients, write the new data to the LBA, and then transfer the delta product to the parity drive using the LBA information. The parity drive may receive the delta product information, read the LBA data, XOR the delta product information with the read information, and write the information, and send the commit to the corresponding parity drive (S850). After the parity check is completed (S860), the transaction may be reported to the host as completed (S870).

[0097] Fig. 9 Methods of performing heat removal and addition after drive failure are described in accordance with various embodiments.

[0098] refer to Fig. 9In various embodiments, the storage system may be configured to recover from failure using a BMC and a data protection controller (e.g., BMC 130 and data protection controller 145A / B, data protection controller 155A / B of FIG. 1 ). In various embodiments, BMC 130 may detect when a drive fails or is removed (S900). In various embodiments, BMC 130 may detect the failure or removal of a drive in several ways. For example, in one embodiment, all drives have presence pins connected to BMC 130 via a midplane. Therefore, when a drive is physically removed from the chassis, BMC 130 detects the removal. In various embodiments, BMC 130 also periodically monitors the stability of each attached drive. When BMC 130 detects any serious error condition reported by a drive, BMC 130 may determine to stop the SSD service. In either case, BMC 130 reports the location of the failed drive (e.g., the corresponding physical slot number) to the controller in FPGA 140, FPGA 150. In another embodiment, FPGA 140, FPGA 150 can support hot plug-in, where all drives are connected via PCIe sideband signals. Thus, FPGA 140, FPGA 150 can detect certain error conditions, including when a drive is unplugged or plugged in, or the PCIe link to the drive is no longer operational. In various embodiments, in such error conditions, a controller in FPGA 140, FPGA 150 can isolate the failed drive, or BMC 130 can achieve the same by disabling power to the failed drive and then starting to rebuild the drive.

[0099] In various embodiments, when a drive fails, the data protection controller is responsible for maintaining the performance of I / O requests. Therefore, the data protection controller can use one or more parity drives or erasure code drives to rebuild data and perform normal I / O until a new drive is provided (S910). In various embodiments, when a new drive is inserted, the BMC 130 can intervene again and configure the new drive as part of one or more groups or start a rebuild operation (S920). The controller can then start the rebuild (S930). Therefore, the recovery path in this system is configured to provide the smallest possible performance impact while providing lower latency in the data access path.

[0100] The foregoing describes example embodiments and should not be construed as limiting thereof. Although several example embodiments have been described, it will be readily apparent to those skilled in the art that many modifications to the example embodiments are possible without substantially departing from the novel teachings and advantages of the example embodiments. Accordingly, all such modifications are intended to be included within the scope of the example embodiments as defined in the appended claims. In the claims, the means-plus-function clause is intended to cover structures described herein as performing the functions recited, and not only structural equivalents but also equivalent structures. Therefore, it should be understood that the foregoing describes example embodiments and should not be construed as being limited to the specific embodiments disclosed, and modifications to the disclosed example embodiments and other example embodiments are intended to be included within the scope of the appended claims. The inventive concept is defined by the above claims, in which the equivalents of the claims will be included.

Claims

1. A storage system, include: The first set of solid-state drives; as well as A first field programmable gate array is connected to the first group of solid-state drives, wherein the first field programmable gate array includes a first data protection controller, the first data protection controller is configured to perform the following operations: managing input / output requests to and from the first set of solid-state drives in accordance with a data protection configuration; generating parity bits according to the data protection configuration; as well as storing the parity bits on at least a parity solid state drive from the first group of solid state drives, The first data protection controller is implemented for a first virtual machine used by a first user, and The first data protection controller is further configured to specify an erasure code operation according to the data protection configuration.

2. The storage system according to claim 1, further comprising: include: a processor connected to the first field programmable gate array via a high-speed bus; as well as A baseboard management controller is connected to the first field programmable gate array through the high-speed bus, wherein the baseboard management controller is configured to provide the data protection configuration.

3. The storage system according to claim 1, further comprising: include: a processor connected to the first field programmable gate array via a high-speed bus; as well as A baseboard management controller is connected to the first field programmable gate array via a low-speed bus, wherein the baseboard management controller is configured to provide the data protection configuration. 4 . The storage system of claim 1 , further comprising a first switch connecting the first field programmable gate array to the first group of solid-state drives.

5. The storage system according to claim 4, further comprising: include: a second set of solid-state drives; as well as a second field programmable gate array connected to the second set of solid-state drives via a second switch, wherein: The second field programmable gate array includes a second data protection controller; and The first field programmable gate array and the second field programmable gate array are connected by a connection element between the first switch and the second switch. 6 . The storage system according to claim 5 , wherein the first data protection controller sends the replicated data to the second data protection controller via the first switch and the second switch.

7. The storage system of claim 1, wherein the first data protection controller comprises a redundant array of independent disks controller.

8. The storage system of claim 1, wherein the first data protection controller comprises an erasure code controller.

9. A storage system, include: The first set of solid-state drives; a second set of solid-state drives; a first field programmable gate array connected to the first set of solid-state drives, wherein the first field programmable gate array includes a first redundant array of independent disks controller; as well as a second field programmable gate array connected to the second set of solid-state drives, wherein the second field programmable gate array comprises a second redundant array of independent disks controller, wherein the first field programmable gate array and the second field programmable gate array are configured to: managing input / output requests to and from the first group of solid-state drives and the second group of solid-state drives according to a data protection configuration; generating parity bits for the stored data according to the data protection configuration; storing the parity bits on at least one parity solid-state drive from the first group of solid-state drives; as well as duplicating the first set of solid-state drives on the second set of solid-state drives, wherein the first redundant array of independent disks controller is implemented for a first virtual machine used by a first user, and the second redundant array of independent disks controller is implemented for a second virtual machine used by a second user, and Each of the first field programmable gate array and the second field programmable gate array is further configured to specify an erasure code operation according to the data protection configuration.

10. The storage system according to claim 9, further comprising: include: a processor connected to the first field programmable gate array and the second field programmable gate array via a bus; as well as A baseboard management controller is connected to the first field programmable gate array and the second field programmable gate array through the bus, wherein the baseboard management controller is configured to provide the data protection configuration.

11. The memory system of claim 10, wherein the bus comprises a Peripheral Component Interconnect Express bus.

12. The storage system according to claim 9, further comprising: include: a first peripheral component interconnect express switch connecting the first field programmable gate array to the first set of solid state drives; as well as A second PCI Express switch connects the second FPGA to the second set of solid state drives, wherein the first PCI Express switch and the second PCI Express switch connect the first FPGA to the second FPGA.

13. The storage system of claim 9, wherein the data protection configuration comprises a hybrid redundant array of independent disks configuration.

14. A storage system, include: The first set of solid-state drives; a first field programmable gate array connected to the first set of solid-state drives; a processor connected to the first field programmable gate array via a high-speed bus; as well as a motherboard management controller connected to the first field programmable gate array via a low-speed bus and configured to provide data protection configuration to the first field programmable gate array, wherein the first field programmable gate array includes a first data protection controller, and the first data protection controller is configured to perform the following operations: managing input / output requests to and from the first set of solid-state drives in accordance with the data protection configuration; generating parity bits for data stored in the first group of solid-state drives according to the data protection configuration; as well as storing the parity bits on at least one parity solid state drive from the first group of solid state drives, The first data protection controller is implemented for a first virtual machine used by a first user, and The first data protection controller is further configured to specify an erasure code operation according to the data protection configuration.

15. The storage system of claim 14, wherein the first data protection controller comprises a redundant array of independent disks controller.

16. The storage system of claim 15, wherein the data protection configuration comprises at least one of a Redundant Array of Independent Disks 1 configuration, a Redundant Array of Independent Disks 5 configuration, or a Redundant Array of Independent Disks 6 configuration.

17. The storage system of claim 14, wherein the first data protection controller comprises an erasure coding controller configured to stripe data on a first portion of the first group of solid-state drives and to generate erasure codes based on data on a second portion of the first group of solid-state drives.

18. The storage system according to claim 14, further comprising: include: a second set of solid-state drives; as well as a second field programmable gate array connected to the processor via the high-speed bus, connected to the motherboard management controller via the low-speed bus, and connected to the second set of solid-state drives, wherein the second field programmable gate array includes a second data protection controller configured to perform the following operations: managing input / output requests to and from the second set of solid-state drives in accordance with the data protection configuration; and generating parity bits for data stored in the second set of solid-state drives according to the data protection configuration; as well as The parity bits of the second group of solid-state drives are stored on at least one parity solid-state drive from the second group of solid-state drives.

19. The storage system according to claim 14, further comprising: include: a second set of solid-state drives; as well as a first peripheral component interconnect express switch connecting the first field programmable gate array to a first downstream peripheral component interconnect express switch and a second downstream peripheral component interconnect express switch, wherein: the first downstream PCI Express switch is connected to the first set of solid-state drives, and the first set of solid-state drives is connected to the first field programmable gate array via the first PCI Express switch; and The second downstream PCI Express switch is connected to the second set of solid-state drives and connects the second set of solid-state drives to the first field programmable gate array via the first PCI Express switch.

20. The storage system of claim 14, wherein the data protection configuration comprises a hybrid redundant array of independent disks configuration.

Images