Raid system with parallel structure and method thereof

Abstract

The invention discloses a Raid system of a parallel structure and a method thereof, and relates to the technical field of Raid cards, the Raid system comprises a Raid module and a storage component which are respectively in communication connection with a bus, the Raid module and the storage component form the parallel structure, the novel parallel structure adopted by the invention can enable the storage component to be directly connected with the bus, performance bottleneck is avoided, the performance of the Raid card can be effectively improved, and meanwhile, the Raid card can be conveniently connected with the storage component. The number of the Raid cards can be dynamically expanded, computing resources can be dynamically allocated according to task conditions, hot plugging can be achieved, maintenance convenience can be improved, the problem of single-point failure can be avoided, the effect of improving the working performance of the Raid cards is achieved, in addition, packaging can be conducted according to the type of an interface, operation logic of a traditional Raid card is achieved, and the service life of the Raid cards is prolonged. And the use compatibility is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of Raid cards, and in particular to a parallel-structured Raid system and a method thereof. Background Art

[0002] RAID (Redundant Array of Independent Disks) is a storage technology that is attached to HBA (Host Bus Adapter) and developed based on the need to improve the reliability of storage components. It combines multiple independent storage components to provide higher redundancy, data performance and capacity. RAID technology provides a variety of different Raid levels (Raid0 / 1 / 3 / 5 / 6, different storage component combinations, data distribution forms) to support data reliability, performance, space expansion and other requirements, and achieve multi-faceted expansion of storage component performance.

[0003] At the same time, because traditional RAID technology is attached and developed on HBA, it must retain the most basic HBA structural characteristics (the purpose of HBA is to adapt to and match high-speed buses and low-speed storage components, so one end of the HBA structure is connected to the high-speed internal bus (currently the PCI bus (including PCIe / PCIx / PCI), previously VESA, EISA, ISA, etc.), and the other end is connected to a relatively low-speed external bus interface (such as SAS, SATA, ATA, etc.) to provide matching, connection and management between devices of different speeds).

[0004] With the increase in data processing speed of the media in the storage components (even greatly increased, such as HDD (disk) to SSD (solid state disk)), the increase in interface speed has become inevitable. The current interface type and related speed have quickly transitioned from the original SATA / SAS to NVMe, and the speed has also developed from 6Gb / s of SATA3.0 and 12Gb / s of SAS3.0 to 8GB / s of NVMe1.4 PCIeGen4 U.2FF. Due to the change in the interface type, the original external bus (external interface) has completely aligned with the internal bus in terms of speed and can be directly interfaced to the internal bus (NVMe U.2 / E1 / E3 / AIC / M.2 is actually directly interfaced to the PCIe bus through the PCIe protocol), and the main functional requirements of the original HBA card to coordinate the internal and external speed differences have disappeared. For RAID technology based on HBA, although there is still a large demand for the redundancy technology, performance and capacity expansion capabilities of the storage components provided by RAID, it is inevitable that the performance of RAID technology using the traditional internal and external bus connection architecture will have an insurmountable bottleneck.

[0005] Traditional RAID technologies include hardware RAID and software RAID:

[0006] Hardware RAID is implemented through hardware devices independent of the system. Since it still uses HBA-based technology, its structure consists of a RAID card and several storage components connected to it. This leads to the following problems during use: 1. When the RAID card is damaged, the data in the storage components connected to it cannot be read normally, resulting in a single point failure problem; 2. The RAID card cannot be hot-maintained (online plug-in and unplug-out of the RAID card for maintenance); 3. The performance cannot be expanded, and the performance is insufficient when the task data volume surges; 4. The interface type is single and cannot be encapsulated according to the interface.

[0007] Software RAID can be simplified as a virtual RAID card generated by using system resources, but it will occupy system resources during use. For example, as the amount of data increases, it will lead to excessive occupation and consumption of the host's core processing and computing resources. In extreme cases, it will cause the host's core computing resources to be exhausted and unable to carry out other tasks except Raid-related work. At the same time, accidental power outages or failures will cause current data loss.

[0008] In order to match the current technological development of external storage components and to meet the market demand for the features provided by RAID technology, changing the implementation mode of RAID technology developed based on HBA has become a new innovation point. Summary of the invention

[0009] The embodiment of the present invention provides a parallel-structured Raid system and method thereof, which adopts a novel parallel structure and solves the problem that the above-mentioned traditional Raid technology cannot meet the demand.

[0010] A parallel Raid system, comprising:

[0011] A Raid module, which has at least one Raid card, and the Raid card is communicatively connected with the bus;

[0012] and, at least one storage component communicatively coupled to the bus;

[0013] The Raid module is in communication connection with the storage component, and the Raid module is configured to manage the storage component.

[0014] Furthermore, encapsulation can be performed according to the interface type to make it suitable for various types of interfaces.

[0015] A management method for a parallel-structured Raid system includes: managing Raid cards; and managing storage components.

[0016] Further, when the number of Raid cards is greater than two, including the allocation of the main Raid card and the slave Raid card, the steps are as follows:

[0017] Get the running status of all Raid cards;

[0018] Assign one of the Raid cards that is in normal operation as the master Raid card, and the other Raid cards as slave Raid cards.

[0019] Furthermore, the method also includes monitoring the main Raid card, and when a failure is detected, allocating one of the slave Raid cards as the main Raid card, wherein the reallocation of the main Raid card is performed based on two dimensions: the fastest completion of allocation or the minimum impact on normal tasks.

[0020] Furthermore, the master Raid card manages each slave Raid card.

[0021] Furthermore, the master Raid card dynamically allocates the function of each slave Raid card so that the functions of the slave Raid cards are the same or different.

[0022] Furthermore, the storage components are managed from the Raid card according to the assigned functions.

[0023] Further, when a new Raid card is added, it is allocated as a slave Raid card, and the master Raid card dynamically allocates the functions of the newly added slave Raid card.

[0024] Further, when deleting the Raid card, the main Raid card reallocates or does not reallocate the functions of other slave Raid cards according to the function of deleting the Raid card.

[0025] The beneficial effects of the above technical solution provided by the embodiment of the present invention include at least:

[0026] 1. The parallel structure adopted allows the storage components to be directly connected to the bus, avoiding performance bottlenecks and effectively improving the performance of the Raid card.

[0027] 2. The number of Raid cards can be expanded, and computing resources can be dynamically allocated according to the task situation to improve the working performance of the Raid card.

[0028] 3. Hot-swappable, able to achieve online maintenance of Raid card.

[0029] 4. When one of the Raid cards is damaged, other Raid cards can take over the work tasks of the damaged Raid card to avoid the problem of single point failure.

[0030] 5. The parallel-structured Raid system can be packaged according to the type of interface. For example, the use of NVMe-compliant FormFact can simplify the host device type and reduce the overall complexity of the host. At the same time, with the addition of hot-swap function, the convenience of maintenance can be further greatly improved.

[0031] 6. This parallel-structured Raid system adopts the same series management structure as the traditional Raid card, which can realize the operating logic of the traditional Raid card, improve the compatibility of use, and can directly replace the traditional Raid card without targeted adjustment.

[0032] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description, claims, and drawings.

[0033] The technical solution of the present invention is further described in detail below through the drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0035] Figure 1 The physical and logical connection structure of the HBA card in the prior art;

[0036] Figure 2 It is a connection architecture between a traditional hard Raid card and the storage components it manages and a host in the prior art;

[0037] Figure 3 This is a schematic diagram of the traditional Raid card mode and related interface rates;

[0038] Figure 4 A schematic diagram of the structure of a parallel Raid system provided in an embodiment of the present invention;

[0039] Figure 5 A logical structure diagram of control command processing and data transmission processing of a Raid card and related components in a parallel Raid system provided by an embodiment of the present invention.

[0040] Reference numerals:

[0041] 1. Raid module; 11. Raid card; 2. Storage component; 3. Bus; 4. CPU; 5. Memory. DETAILED DESCRIPTION

[0042] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0043] Figure 1 The physical and logical connection structure of the HBA card in the prior art is shown, which combines multiple independent hard disks to provide higher redundancy, data performance and capacity.

[0044] The core technical logic of RAID technology is to reasonably slice (strip) the source data and perform XOR algorithm based on the data slices to generate XOR data, and then associate the source slice data and XOR data and store them on different hard disks. When a hard disk fails and cannot read the source data fragment, the remaining source data fragments and XOR data can be used to generate the lost data through the inverse XOR operation (also called XOR budget), and the fragment data can be combined into the original data and provided to the upper system / device (this function is the most common Raid5 / Raid3 function in RAID technology, which can deal with the situation where one data area / hard disk data is lost / damaged; the function of Raid6 is to generate secondary / two sets of XOR data on top of Raid5 to improve fault tolerance and provide for the situation where two data areas / hard disks are lost / damaged at the same time). At the same time, the slice data will be reasonably distributed on each hard disk to make full use of the read / write / storage space capabilities of each hard disk, thereby greatly expanding the overall performance and capacity.

[0045] like Figure 2 As shown in the figure, the current traditional Raid technology is implemented based on the HBA architecture. Even for the NVMe SSD that has begun to be used on a large scale (its current typical interface rate has reached 4GB / s (PCIe x2)-16GB / s (PCIe x8), and the most common U.2 single port is the PCIe x4 interface with a rate of 8GB / s), it also uses the same connection architecture as SATA / SAS HDD / SSD.

[0046] from Figure 2It is not difficult to see that in the traditional HBA connection architecture, the interface between the HBA card / Raid card 11 and Bus 3 becomes an inevitable bottleneck when supporting NVMeSSD. (The Bus 3 interface of the traditional HBA card / Raid card 11 is only PCIex8 (the current Broadcom LSI Raid card 11 has begun to promote the PCIex16 interface, but is limited by the current PCIe interface specification design of only PCIex16, which is also the maximum number of channels of a single interface in the current Bus 3 interface specification))

[0047] RAID cards 11 based on HBA architecture are still mainstream in the current market. Enterprise-level products include the series of Raid cards 11 from Broadcom LSI, which has the largest number of users in the world (reference: https: / / www.broadcom.com / products / storage / raid-controllers), Microchip's AdaptecSmartRaid series (reference: https: / / www.microchip.com / en-us / products / storage / adaptec-smartraid-raid-adapters), the emerging Highpoint's RocketAIC series cards (reference: https: / / www.highpoint-tech.com / ), and the domestic Suzhou Guoxin's CCore-Raid series (reference: https: / / www.china-core.com / products_show.php?id=41&lm=40), etc.

[0048] From the analysis of RAID technology, we can see that although RAID technology is developed from HBA, its technical principles and implementation paths do not necessarily rely on the core functions provided by HBA - that is, the matching and connection of fast and slow devices. This situation can also be confirmed from the current technology and application landing in the industry. These include system-level soft RAID technology that has been implemented in enterprise-level environments (such as the mdadm soft Raid function module in the Linux environment), consumer-level soft Raid that has been implemented in batches (such as Synology's built-in soft Raid), and emerging GPU-accelerated soft Raid technology in enterprise-level environments (such as GRaid's SupermeRAIDSR-1000 series NVGPU-accelerated soft Raid, reference: https: / / www.graidtech.com / ), etc.

[0049] The interface of current storage component 2 (mainly hard disk) has begun to accelerate the transition from SATA / SAS to NVMe. At the same time, with the improvement of the performance of SSD media itself, a high-speed interface with the same performance as bus 3 has also become a necessity for future SSD components. In this case, the Raid card 11 of the traditional structure path will also be replaced by the new Raid card 11.

[0050] Analysis of the technology, performance and functions of the current traditional hard Raid card 11:

[0051] The traditional HBA-based RAID card 11 mode is limited by the functions and overall structural logic of the HBA. The interface between the Raid card 11 and the bus 3 will inevitably become a bottleneck for performance improvement and greatly hinder the implementation of high-speed interface hard disks when the current hard disk interface characteristics are already aligned with the bus 3. At the same time, the traditional Raid technology has been developed since its creation, because its architecture is developed on the HBA card. In this structure, although the traditional hard Raid technology provides performance, capacity scalability and operational reliability above the HBA card, due to its structural characteristics, it also brings about the single point failure problem of the HBA / Raid card 11 and the derived problem of being unable to perform hot maintenance.

[0052] 1. Technical analysis:

[0053] The traditional Raid function is developed based on HBA and was developed to expand the hard disk capacity and partially improve the performance at a low cost when the HD capacity was small, the performance was low, and the interface rate was low. Later, some reliability issues were solved with the development of technology. However, from the current technical situation, some basic functions and underlying architecture of Raid are no longer suitable for the technical status of current equipment.

[0054] 2. Performance issues:

[0055] like Figure 3 As shown, the traditional Raid card 11 is developed based on HBA and matches the relatively low-speed interface (SATA / SAS). In order to simplify the overall analysis, the impact of communication and data processing losses on performance can be removed, the delay characteristics of each protocol can be ignored, and the structural model can be simplified, that is, the Raid / HBA information processing module is used as a central point for the collection of upper and lower layer data and the central point processing capacity and delay are ignored. After the model is simplified, the lower data channel of the central point is the bandwidth of the channel connecting the central point and each hard disk, and the sum is added to form the total bandwidth. The upper data channel is the interface between the Raid / HBA card and the bus 3, and its interface bandwidth is also the interface bandwidth of the card.

[0056] For the downstream bandwidth, the communication and data processing losses of the interface are also simplified and the latency characteristics are ignored. Based on the current SATA and SAS interface protocol versions SATA 3.0, SAS 3.0 and NVMe 1.4Gen4, if a three-mode interface control card (that is, supporting SATA / SAS / NVMe three modes) is used, the interface rates are SATA 6Gb / s, SAS12Gb / s and NVMe Gen4 PCIe x4's 64Gb / s respectively. Taking into account that in current traditional applications, the number of hard disks connected downstream to the Raid / HBA card is usually 4-32 and uses the same type of interface hard disk (based on the current more traditional host support structure for hard disks, usually no more than 24 hard disks are connected to a single HBA / RAID card 11, and the more common number is 24 / 16 / 8), then the total rate of the highest connected aggregate interface in SATA interface mode is 32*6Gb / s=192Gb / s (or 24*6=144, 16*6=96, 8*6=48), and in SAS interface mode it is 32*12Gb / s=384Gb / s (or 24*12=288, 16*12=192, 8*12=96).

[0057] For the uplink channel, the analysis is also simplified, and its uplink bandwidth is 128Gb / s of PCIe x8 (based on the current mainstream PCIe Gen4 bus version 3, the peak bandwidth of the PCIe x8 interface is 16GB / s, or 128Gb / s. If some manufacturers upgrade to PCIe Gen4 x16 cards, the peak bandwidth is 32GB / s, or 256Gb / s; if the PCIe version is upgraded to PCIe Gen5 bus version 3 and the PCIe x8 channel is retained, the peak bandwidth is also 32GB / s / 256Gb / s; if it is expanded to PCIex16 based on the Gen5 version, the peak bandwidth is 64GB / s / 512Gb / s).

[0058] As shown in the following table (uplink and downlink rate and matching analysis table):

[0059] Table 1

[0060]

[0061] Table 2

[0062]

[0063] Based on the analysis of Tables 1 to 2, in the case of simplified analysis, the traditional Raid card 11 mode has shown a fundamental bottleneck caused by its traditional structure. This bottleneck has become an insurmountable obstacle for supporting the new high-speed NVMe storage component 2, and even brings some restrictions to the traditional SATA / SAS interface protocol hard disk supported by the traditional Raid card 11. This is also the reason why the current traditional mode Raid card 11 cannot fully exert the hard disk performance under some configurations in actual use (see the gray text part in the figure for the part that is not compatible in the current traditional mode Raid card 11 environment). Of course, this simplified analysis only considers the theoretical bandwidth of the upstream and downstream, and ignores the communication loss and basic delay that must exist in the actual interface; at the same time, due to the simplified analysis factors, the processing capacity of the Raid task is not analyzed, and only the data reading task under the normal operation of the Raid card 11 and the connected hard disk is used as the simplified analysis point. However, the most basic conclusion can be drawn from the simplified architecture analysis, that is, the traditional Raid card 11 developed based on the HBA mode is no longer suitable for the needs of the current new high-speed NVMe storage component 2.

[0064] 3. Functional issues:

[0065] The Raid technology has been developed from its early stage to the present day because its architecture has always used the basic architecture of HBA. Although the Raid technology solves the single point failure problem of the managed storage component 2 in the Raid system, the single point failure of the Raid card 11 itself has not been solved due to the architecture problem. The serial structure and all core functions need to pass through the Raid card 11, which makes the Raid card 11 a single point of failure in the entire Raid system. This problem cannot be changed in the traditional physical structure serial architecture.

[0066] Current system-level soft Raid technology and performance analysis:

[0067] The application mode of implementing Raid at the system level (i.e., soft Raid, hereinafter referred to as the same) is derived from two aspects of technology and demand changes. First, the core processing and computing power in the computer system has been greatly improved. At the same time, the improvement of equipment / system performance is much faster than the improvement of demand, resulting in partial or large idleness of core processing and computing resources, so other tasks can be planned based on these idle resources; second, for some cost-sensitive / performance-insensitive users, some general overall performance can be sacrificed in exchange for the required performance and reliability of the primary storage component 2 that meets the requirements.

[0068] There is a huge difference between the physical connection architecture of soft Raid and the traditional hard Raid card 11 in terms of technical implementation. If the system-level soft Raid is simplified and virtualized into a Raid card 11, its connection architecture is completely different from the cascade serial type of the traditional Raid card 11 and is more inclined to the mode of parallel Raid card 11. However, unlike the parallel Raid card 11 (or parallel hard Raid), soft Raid uses system-level resources for Raid management and uses core computing resources such as CPU4 / memory to complete the XOR calculation required for Raid. At the same time, the management and use of the traditional Raid card 11 based on hardware components must be completed by the system layer. This leads to a significant difference in the use of soft Raid and traditional Raid cards 11 (i.e. hard Raid); in addition, some high-level functions that can be achieved by traditional hard Raid cards 11, such as data protection for abnormal power failure, real-time data compression with greater resource consumption, and transparent data encryption, cannot be achieved or are difficult to achieve.

[0069] In addition, the change in the soft Raid architecture brings some other advantages besides its obvious cost-effectiveness, such as better support for NVMe SSD hard drives. However, while giving full play to the performance of the NVMe SSD interface, the increase in data volume will lead to excessive occupation and consumption of the host's core processing and computing resources. In extreme cases, the host's core computing resources will be exhausted and unable to carry out other work except Raid-related work. Based on the problem of excessive consumption of host core resources in a soft Raid environment with large amounts of data, new manufacturers in the industry have installed computing power cards in the host to unload the Raid data XOR processing, which consumes the most computing power resources in the soft Raid environment, from the host CPU4 and memory, and redirect it to a dedicated GPU card, so as to achieve an innovative solution with basically no / little host resource occupation under high-throughput and large data volumes. Although this innovative solution / mode is also equipped with an independent card for Raid calculations, which solves the problem of Raid computing power, it is architecturally the same as other soft Raids and cannot work at the device layer without the support of the system layer. At the same time, since it still requires partial processing at the system layer, it cannot avoid the loss of current data caused by accidental power outages or failures of the entire system (except for devices with overall data protection for accidental power outages and failure conditions at the device layer, such as core mainframes).

[0070] Through the above analysis of the system-level soft Raid mode, it can be seen that the application of soft Raid is more suitable for enterprises or individuals who are sensitive to cost, have basic data reliability requirements but not very high requirements, and have no fixed requirements for the entire system management classification. For traditional enterprise-level users, its characteristics cannot fully meet the enterprise's requirements for Raid performance and functionality; at the same time, even if hardware-accelerated soft Raid is used to avoid most of the host core resource consumption, the usage mode is still basically the same as the soft Raid mode, and the use and management mode of hard Raid cannot be restored. Therefore, soft Raid (including hardware-accelerated soft Raid) can only be used as a transitional solution in some occasions and specific systems with specific needs.

[0071] Through simplified problem analysis and finding the core cause of the problem, we have been able to locate the shortcomings and deficiencies of traditional hard / soft Raid solutions, but the Raid card 11 in the actual system environment does not simply work in the theoretical interface bandwidth and pure read mode. The actual architecture and technical evaluation, as well as the process of innovating new products based on the shortcomings of current traditional products / solutions, still need to fully consider the characteristics of the upstream and downstream interfaces under the traditional mode and the innovative mode, the functional characteristics of each Raid level of the Raid card 11, the task characteristics of the Raid card 11 in the system (the actual situation of the read-write ratio, etc.), the needs and usage habits of users and other factors. At the same time, based on the current technology and industry standards and combined with the direction of new technologies and new standards, plan the innovation points of this innovative technology.

[0072] Based on the above problems of traditional Raid technology, such as Figure 4 As shown, the present invention proposes a parallel structured Raid system, comprising a Raid module 1 and a storage component 2, wherein the Raid module 1 and the storage component 2 are respectively communicatively connected to a bus 3, and the Raid module 1 and the storage component 2 form a parallel structure.

[0073] In the above structure, the Raid module 1 is composed of at least one Raid card 11, each Raid card 11 is respectively connected to the bus 3 for communication, and when the number of the Raid cards 11 is greater than one, a parallel structure is formed between the Raid cards 11.

[0074] In the above structure, at least one storage component 2 is included, and each Raid card 11 is respectively connected to the bus 3 for communication. When the number of storage components 2 is greater than one, a parallel structure is formed between the storage components 2 .

[0075] In the above structure, the Raid card 11 in the Raid module 1 is communicatively connected with each storage component 2 respectively.

[0076] The role of the above parallel structure is:

[0077] 1. The storage component 2 is directly connected to the bus 3 for communication, which can facilitate the high-performance storage component 2 to avoid the formation of performance bottleneck problems.

[0078] 2. When the Raid module 1 uses more than two Raid cards 11, since it adopts a parallel structure, if any one of the Raid cards 11 is damaged, the other Raid card 11 can take over its work task to avoid the problem of single point failure.

[0079] 3. When the Raid module 1 uses more than two Raid cards 11, hot plugging can be performed to achieve the effect of online maintenance of the Raid card 11.

[0080] 4. The number of Raid cards 11 of the Raid module 1 can be expanded, and computing resources can be dynamically allocated according to task conditions to improve the working performance of the Raid card 11.

[0081] At the same time, since it adopts a parallel structure directly connected to bus 3, when packaging the Raid system of this parallel structure, it can be packaged according to the type of required interface. For example, adopting the NVMe standard form packaging (FormFact) can simplify the host device type and reduce the overall complexity of the host. At the same time, with the help of hot-swap function, the convenience of maintenance can be further greatly improved.

[0082] like Figure 4 As shown, different from the traditional serial Raid card 11 (based on HBA structure), the parallel structure Raid system proposed in the present invention adopts a parallel structure in physical connection. In the present system, the Raid card 11 and the storage component 2 managed by it are no longer a serial architecture in which the storage component 2 is connected to the Raid card 11, and the Raid card 11 is further connected to the device bus 3. Instead, the Raid card 11 and the storage component 2 managed by it (mainly SSD hard disks with NVMe interface, and can also cover storage components 2 with traditional SATA / SAS interface connected to the PCIe bus 3 through traditional HBA, such as SATA / SAS SSD, SATA / SASHDD and other traditional storage components 2) are all connected in parallel to the device bus 3 (if it is a new NVMe storage component 2, the Raid card 11 and the NVMe storage component 2 are both connected to the PCIe bus 3 of the device; if it is a SATA / SAS interface storage component 2, the SATA / SAS interface storage device is uniformly connected to the HBA for signaling and interface rate conversion, and then connected to the PCIe bus 3 through the HBA (the main control chip on the motherboard of some devices has an integrated HBA and related SATA / SAS interface, and its structure is similar to that of an independent HBA card, but the structure is simpler)).

[0083] like Figure 5As shown, in order to adapt to and meet the usage habits of the traditional Raid card 11 and improve the compatibility with previous modes in usage and management, the parallel architecture of the Raid system hardware connection based on the parallel structure proposed in the present invention distinguishes the processing of control commands / data in the system from the processing of data transmission in the system, so that the data transmission processing part can make full use of the parallel architecture to obtain a significant performance improvement. At the same time, the control command / management work still uses the same series logic as the traditional Raid card 11, so that the entire Raid system can be restored to the traditional usage and management mode, while improving the performance / function without changing the original usage and management habits, so as to reduce the difficulty of accepting new technologies and reduce the cost of usage and management.

[0084] Serial processing under the serial structure of control commands / data will multiply the overall control command / data processing task volume and time, but because the control command / data processing task volume is between 1% and 1 / 10,000 of the total control command / data processing plus data transmission processing task volume, the method of distinguishing control command / data processing will not have a significant impact on the overall task completion efficiency.

[0085] The present invention proposes a management method for a parallel-structured Raid system, including: managing a Raid card 11; and managing a storage component 2.

[0086] Wherein, when the number of Raid cards 11 is greater than two, including the allocation of the main Raid card 11 and the slave Raid card 11, the steps are as follows:

[0087] Get the running status of all Raid cards 11;

[0088] One of the Raid cards 11 in normal operation is allocated as the master Raid card 11 , and the other Raid cards 11 are allocated as slave Raid cards 11 .

[0089] In the parallel Raid system, all workloads on the card are divided into management roles and workload roles. The management role is divided into a master-slave role mode (Master / Slave, one master and one slave or one master and multiple slaves), and the workload role is formed according to the function allocated by the master Raid card 11. The workload role includes the master-slave mode (Active / Backup) and the active / active plus load balancing mode (Active / Active and Load Balance).

[0090] The purpose of distinguishing the types of management roles is mainly to achieve the uniqueness of the management tasks and logic of the parallel-structured Raid system. The status of the master and slave cards is dynamically monitored through the load of the management role on the parallel-structured Raid system, and the unique master manager (Master) is determined as the manager of the entire parallel-structured Raid system (master Raid card 11) by conditional arbitration in the initial designation / failure, so as to obtain the uniqueness of the management logic and avoid conflicts in management commands. At the same time, the load of the management role task itself only involves the management and arbitration of the status of multiple cards, the corresponding hard disk and the allocation of the workload tasks from the Raid card 11, and its load is relatively low.

[0091] The purpose of distinguishing the types of workload roles is to reasonably utilize the resources of each functional module on each Raid card 11 of the parallel-structured Raid system when a specific type of load / function is required by the system, so as to provide higher system capabilities / functions to the outside world.

[0092] Among them, the active-standby workload mode is mainly used to solve the reliability problem of single point failure, and its read and write performance is equivalent to that of a single card.

[0093] The workload mode of dual-active / multi-active and load balancing is mainly to significantly improve the write performance in the normal working mode and the read and write performance in the case of Raid system degradation while solving the reliability problem of single point failure. Because no matter whether it is writing in normal working conditions or reading and writing in the case of Raid system degradation, data needs to be sent to Raid card 11 for Raid calculation. At this time, the interface channel width and Raid calculation acceleration capability will become the main bottleneck of the entire system. Through multi-card and load balancing scheduling, the interface channel width and Raid calculation acceleration capability of multiple cards can be fully utilized to greatly expand the performance under the corresponding task.

[0094] The multi-Raid card 11 mode of the parallel-structured Raid system is mainly used to solve the single-point failure problem that the traditional Raid card 11 cannot solve, and to solve the write performance under normal working and the read and write performance problems under Raid degradation. At the same time, in the dual-card / multi-card mode, the hot-swap function can further solve the problem of hot maintenance (in the multi-card mode, the automatic master-slave management can easily complete the hot maintenance function that the traditional Raid card 11 cannot support, so as to realize the hot maintenance of equipment components under fault and non-fault planning conditions).

[0095] During normal read tasks, only the master card (Master) with a management role participates in the allocation and management of tasks. The data reading task in the workload is directly transferred from the corresponding hard disk to the corresponding data area in the memory. The Raid card 11 itself does not participate in the work related to the workload.

[0096] The main Raid card 11 is also monitored, and when a failure is detected, one of the slave Raid cards 11 is assigned as the main Raid card 11, wherein the reallocation of the main Raid card 11 is performed based on two dimensions: the fastest completion of the allocation or the least impact on normal tasks.

[0097] Due to the change in the architecture, the parallel-structured Raid system can cope with the single-point failure problem of the traditional Raid card 11 when the number of Raid cards 11 in the installed and working Raid module 1 is more than one. Through the heartbeat information exchange and priority arbitration between the management roles on each Raid card 11, the master-slave relationship of the management roles on each Raid card 11 can be dynamically switched in the event of a failure, thereby preventing the main management role from stopping working due to a failure in the Raid card 11 or a failure in the operation of the main management role, thereby causing the entire Raid system to crash.

[0098] The heartbeat information includes key information such as time information, device and role status and load information, Raid information, task completion information (task completion status table, task data information table, etc.).

[0099] The master Raid card 11 manages each slave Raid card 11 , and the master Raid card 11 dynamically allocates the function of each slave Raid card 11 , so that the functions of the slave Raid cards 11 are the same or different.

[0100] The storage unit 2 is managed from the Raid card 11 according to the assigned functions.

[0101] The slave Raid card 11 forms different working modes according to the assigned functions.

[0102] Exemplary: When reading and writing data to the storage component 2, the master Raid card 11 can allocate functions to the slave Raid card 11 according to the amount of read and write data, and multiple Raid cards 11 can be allocated to the same function to process the data to enhance the data processing capability. Multiple Raid cards 11 can also be allocated to the same function to process the data according to other tasks, such as encryption or decryption tasks, to enhance the data processing capability.

[0103] The above examples are only part of the working modes, including but not limited to the above examples. The remaining working modes that can be obtained according to the technical solution of function allocation will not be repeated here.

[0104] When a new Raid card 11 is added, it is allocated as a slave Raid card 11 , and the master Raid card 11 dynamically allocates the functions of the newly added slave Raid card 11 .

[0105] When the Raid card 11 is deleted, the master Raid card 11 reallocates or does not reallocate the functions of other slave Raid cards 11 according to the function of the deleted Raid card 11.

[0106] The single card mode of the parallel Raid system is mainly to solve the performance bottleneck problem compared to the traditional Raid card 11. Under a single parallel Raid card 11 plus the parallel storage component 2 (4-24 U.2 / E1.s FF NVMeSSD) it manages, its read bandwidth performance can be close to the sum of the interface performance of all effective parallel storage components 2, and its write performance can be close to the performance of the single parallel Raid card 11 and the bus 3 interface. Its overall read and write performance will be 2-10 times that of the traditional Raid card 11 environment).

[0107] For example: a single parallel Raid card 11 (Gen4 PCIe x8) is used to manage 16 parallel NVMe SSD Gen4PCIe x4 U.2 / E1.s. All 16 SSDs are used as Raid5. The read and write performance under normal working conditions are:

[0108] Read performance: (16-1)*7GB / s=105GB / s

[0109] "16" is the total number of hard disks / storage components 2; "-1" is a check bit / disk in the Raid5 case, which does not participate in the work under normal working conditions; "7GB / s" is the estimated value of a single disk (for Gen4 PCIe x4U.2 disk, the theoretical value is 8GB / s. Usually, the read performance of each NVMe SSD disk manufacturer after tuning is between 6.5-7.5GB / s, and the approximate value of 7GB / s is taken as reference data.

[0110] Write performance: 12GB / s (theoretical value is 13.33GB / s)

[0111] This value is the performance of the parallel Raid card 11 interface. Since it uses the Gen4 PCIe x8 interface, the theoretical value is 16GB / s. The theoretical value of the storage component 2 NVMe SSD Gen4 PCIe x4 U.2 is 8GB / s. Excluding the consumption of control commands / data processing and the loss of resources occupied by 1 / 16 of the XOR data transmission, and considering the loss between the theoretical transmission rate and the normal data transmission of the interface, the actual value is about 12GB / s (here, for all Raid5 calculations (ie, XOR calculations) for write data, the loss can be ignored because the Raid5 hard acceleration circuit is used on the Raid card 11).

[0112] The calculation formula of the theoretical value is y=(x-1) / ((x-1) / 2+1.5)*B, where y is the calculated theoretical value, x is the number of disks, and B is the theoretical value of the bus 3 interface of the Raid card 11 (Raid PCIe x8 with PCIe x4 NVMe U.2 / E1SSD).

[0113] It should be understood that the specific order or hierarchy of steps in the disclosed process is an example of an exemplary method. Based on design preferences, it should be understood that the specific order or hierarchy of steps in the process can be rearranged without departing from the scope of protection of the present disclosure. The attached method claims present the elements of the various steps in an exemplary order and are not intended to be limited to the specific order or hierarchy described.

[0114] In the above detailed description, various features are grouped together in a single embodiment to simplify the disclosure. This method of disclosure should not be interpreted as reflecting an intention that the embodiments of the claimed subject matter require more features than are clearly stated in each claim. On the contrary, as reflected in the appended claims, the invention is in a state of having less than all the features of the disclosed individual embodiments. Therefore, the appended claims are hereby expressly incorporated into the detailed description, with each claim standing on its own as a separate preferred embodiment of the invention.

[0115] Those skilled in the art will also appreciate that the various illustrative logic blocks, modules, circuits, and algorithmic steps described in conjunction with the embodiments herein can all be implemented as electronic hardware, computer software, or a combination thereof. In order to clearly illustrate the interchangeability between hardware and software, various illustrative components, blocks, modules, circuits, and steps are generally described above around their functions. Whether such functions are implemented as hardware or software depends on specific applications and the design constraints imposed on the entire system. A skilled person can implement the described functions in an alternative manner for each specific application, but such implementation decisions should not be interpreted as departing from the scope of protection of the present disclosure.

[0116] The steps of the method or algorithm described in conjunction with the embodiments herein can be directly embodied as hardware, a software module executed by a processor, or a combination thereof. The software module can be located in a RAM memory 5, a flash memory, a ROM memory 5, an EPROM memory 5, an EEPROM memory 5, a register, a storage component 2, a mobile disk, a CD-ROM, or any other form of storage medium well known in the art. An exemplary storage medium is connected to the processor so that the processor can read information from the storage medium and can write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an ASIC. The ASIC can be located in a user terminal. Of course, the processor and the storage medium can also be present in a user terminal as discrete components.

[0117] For software implementation, the technology described in this application can be implemented with modules (e.g., procedures, functions, etc.) that perform the functions described in this application. These software codes can be stored in the memory 5 unit and executed by the processor. The memory 5 unit can be implemented in the processor or outside the processor. In the latter case, it is coupled to the processor in a communication manner via various means, which are well known in the art.

[0118] The above description includes examples of one or more embodiments. Of course, it is impossible to describe all possible combinations of components or methods for the purpose of describing the above embodiments, but it should be recognized by those skilled in the art that the various embodiments may be further combined and arranged. Therefore, the embodiments described herein are intended to cover all such changes, modifications and variations that fall within the scope of protection of the appended claims. In addition, with respect to the term "comprising" used in the specification or claims, the word is encompassed in a manner similar to the term "including," as explained by "including" used as a transitional word in the claims. In addition, any term "or" used in the specification of the claims is intended to mean "a non-exclusive or."

Claims

1. A parallel Raid system, characterized in that: include, A Raid module, which has at least one Raid card, and the Raid card is communicatively connected with the bus; and, at least one storage component communicatively coupled to the bus; The Raid module storage component is communicatively connected, and the Raid module is configured to manage the storage component.

2. The system according to claim 1, characterized in that It can be encapsulated according to the interface type to make it suitable for various types of interfaces.

3. A management method for a parallel Raid system, characterized in that: include: Manage Raid cards; And, manage the storage components.

4. The method according to claim 3, characterized in that When the number of Raid cards is greater than two, including the allocation of the master Raid card and the slave Raid card, the steps are as follows: Get the running status of all Raid cards; Assign one of the Raid cards that is in normal operation as the master Raid card, and the other Raid cards as slave Raid cards.

5. The method according to claim 4, characterized in that It also includes monitoring the main Raid card, and when it is identified that it fails, allocating one of the slave Raid cards as the main Raid card, wherein the reallocation of the main Raid card is performed based on two dimensions: the fastest completion of the allocation or the least impact on normal tasks.

6. The method according to claim 4, characterized in that The master RAID card manages each slave RAID card.

7. The method according to claim 6, characterized in that The master RAID card dynamically allocates the function of each slave RAID card so that the functions of the slave RAID cards are the same or different.

8. The method according to claim 7, characterized in that Manage the storage components from the Raid card according to the assigned functions.

9. The method according to claim 6, characterized in that When a new Raid card is added, it is assigned as a slave Raid card, and the master Raid card dynamically assigns the functions of the newly added slave Raid card.

10. The method according to claim 6, characterized in that When a Raid card is deleted, the master Raid card reallocates or does not reallocate the functions of other slave Raid cards according to the function of the deleted Raid card.

Images