Raid system and method under parallel structure

Abstract

The application discloses a Raid system and a method thereof under a parallel structure, relates to the technical field of Raid cards, and comprises a Raid module and a storage component which are respectively connected in communication with a bus, and the Raid module and the storage component form a parallel structure. The novel parallel structure can be used for directly connecting the storage component with the bus, avoiding performance bottleneck, effectively improving the performance of the Raid card, dynamically expanding the number of Raid cards, dynamically allocating computing resources according to a task condition, realizing hot plug-in to improve the maintenance convenience, avoiding the problem of single-point failure, improving the working performance of the Raid card, and realizing the encapsulation of the types of interfaces and the operation logic of traditional Raid cards to improve the compatibility.

Description

Technical Field

[0001] This invention relates to the field of RAID card technology, and specifically to a RAID system and method with a parallel structure. Background Technology

[0002] RAID (Redundant Array of Independent Disks) is a storage technology developed on top of HBA (Host Bus Adapter) to improve the reliability of storage components. It combines multiple independent storage components to provide higher redundancy, data performance, and capacity. RAID technology offers various RAID levels (RAID 0 / 1 / 3 / 5 / 6, with different combinations of storage components and data distribution) to support requirements for data reliability, performance, and space scalability, thus enabling multifaceted expansion of storage component performance.

[0003] Meanwhile, since traditional RAID technology is added to and developed on top of HBA, it inevitably retains the most basic structural characteristics of HBA (the purpose of HBA is to adapt to and match high-speed buses and low-speed storage components, so one end of the HBA structure is connected to the high-speed internal bus (currently the PCI bus (including PCIe / PCIx / PCI, and previously VESA, EISA, ISA, etc.), and the other end is connected to the relatively low-speed external bus interface (such as SAS, SATA, ATA, etc.) to provide matching, connection and management between devices of different speeds).

[0004] With the increasing data processing speed of storage media (even drastically, such as the shift from HDDs to SSDs), an increase in interface speed has become inevitable. Current interface types and related speeds have rapidly transitioned from the original SATA / SAS to NVMe, with speeds evolving from 6Gb / s for SATA 3.0 and 12Gb / s for SAS 3.0 to 8GB / s for NVMe 1.4 PCIe Gen4 U.2FF. Due to this change in interface type, the original external bus (external interface) has become completely on par with the internal bus in terms of speed and can directly interface with it (NVMe U.2 / E1 / E3 / AIC / M.2 actually interface directly to the PCIe bus via the PCIe protocol). The primary functional requirement of the original HBA card in coordinating the speed difference between the internal and external interfaces has disappeared. While the demand for redundancy, performance, and capacity expansion capabilities provided by RAID for storage components still exists, the performance bottleneck inherent in RAID technology using the traditional internal and external bus connection architecture is inevitable.

[0005] Traditional RAID technologies include two types: hardware RAID and software RAID. Hardware RAID is implemented through hardware devices independent of the system. Because it still uses HBA-based technology, its structure consists of a RAID card and several storage components connected to it. This leads to the following problems during use: 1. When the RAID card fails, the data in the connected storage components cannot be read normally, resulting in a single point of failure; 2. The RAID card cannot be hot-maintained (maintenance can not be performed by online unplugging and plugging in the RAID card); 3. Performance cannot be scaled, and performance is insufficient when the amount of data in the task surges; 4. The interface type is limited and cannot be encapsulated according to the interface.

[0006] Software RAID can be simplified as a virtual RAID card generated using system resources. However, it will consume system resources during use. For example, as the amount of data increases, it will excessively occupy and consume the host's core processing and computing resources. In extreme cases, it will lead to the exhaustion of the host's core computing resources, making it impossible to carry out other tasks besides RAID-related tasks. At the same time, unexpected power outages or failures will result in the loss of current data.

[0007] To keep pace with the technological development of current external storage components, and to meet the market demand for the features provided by RAID technology, changing the implementation mode of RAID technology developed on top of HBA has become a new point of innovation. Summary of the Invention

[0008] This invention provides a RAID system and method with a parallel structure, which adopts a novel parallel structure and solves the problem that traditional RAID technology cannot meet the requirements.

[0009] A RAID system with a parallel structure, comprising, A RAID module having at least one RAID card, which is connected to the bus for communication. And at least one storage component that is connected to the bus for communication; The RAID module is connected to the storage components via communication, and the RAID module is configured to manage the storage components.

[0010] Furthermore, it can be encapsulated according to the interface type to make it suitable for various types of interfaces.

[0011] A management method for a RAID system with a parallel structure includes: managing the RAID card; and managing the storage components.

[0012] Furthermore, when there are more than two RAID cards, the process includes assigning master and slave RAID cards, and the steps are as follows: Get the running status of all RAID cards; Assign one of the RAID cards that is running normally as the master RAID card, and assign the other RAID cards as slave RAID cards.

[0013] Furthermore, it also includes monitoring the primary RAID card, and when a failure is detected, reassigning one of the secondary RAID cards as the primary RAID card. The reassignment of the primary RAID card is carried out with reference to two dimensions: the fastest completion of the reassignment or the least impact on normal tasks.

[0014] Furthermore, the master RAID card manages each slave RAID card.

[0015] Furthermore, the master RAID card dynamically allocates functions to each slave RAID card, allowing the slave RAID cards to have the same or different functions.

[0016] Furthermore, the RAID card manages the storage components according to the assigned functions.

[0017] Furthermore, when a new RAID card is added, it is assigned as a slave RAID card, and the master RAID card dynamically allocates the functions of the newly added slave RAID card.

[0018] Furthermore, when a RAID card is deleted, the primary RAID card may or may not reallocate the functions of the other secondary RAID cards, depending on the functions of the deleted RAID card.

[0019] The beneficial effects of the above-described technical solutions provided in the embodiments of the present invention include at least the following: 1. The parallel structure adopted allows the storage components to be directly connected to the bus, avoiding performance bottlenecks and effectively improving the performance of the RAID card.

[0020] 2. The number of RAID cards can be expanded, and computing resources can be dynamically allocated according to the task situation to improve the working performance of RAID cards.

[0021] 3. It can be hot-swapped, enabling online maintenance of RAID cards.

[0022] 4. If one of the RAID cards fails, other RAID cards can take over the tasks of the failed RAID card, thus avoiding a single point of failure.

[0023] 5. The RAID system in this parallel structure can be encapsulated according to the type of interface. For example, adopting the NVMe standard form factor can simplify the host device types and reduce the overall complexity of the host. At the same time, the hot-swappable function can further improve the convenience of maintenance.

[0024] 6. This parallel RAID system adopts the same serial management structure as traditional RAID cards, which can realize the operation logic of traditional RAID cards, improve the compatibility of use, and can directly replace traditional RAID cards without making specific adjustments.

[0025] Other features and advantages of the invention will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures particularly pointed out in the written description, claims, and drawings.

[0026] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0027] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1 This describes the physical and logical connection structure of HBA cards in existing technologies. Figure 2 This refers to the traditional hardware RAID card, the storage components it manages, and the connection architecture between it and the host in existing technologies. Figure 3 This is a schematic diagram of the traditional RAID card mode and related interface speeds; Figure 4 This is a schematic diagram of the structure of a RAID system with a parallel structure provided in an embodiment of the present invention; Figure 5 This is a logical structure diagram of the control command processing and data transmission processing of the RAID card and related components in a parallel structure RAID system provided in an embodiment of the present invention.

[0028] Figure label: 1. RAID module; 11. RAID card; 2. Storage components; 3. Bus; 4. CPU; 5. Memory. Detailed Implementation

[0029] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0030] Figure 1The diagram illustrates the physical and logical connection structure of an HBA card in the technology, which combines multiple independent hard drives together to provide greater redundancy, data performance, and capacity.

[0031] The core logic of RAID technology is to rationally slice (strip) the source data, perform an XOR algorithm on the data slices to generate XOR data, and then associate the source slice data and XOR data and store them on different hard drives. When a hard drive fails and cannot read the source data segment, the remaining source data segments and XOR data can be used to generate the lost data through an inverse XOR operation (also known as XOR estimation), and the segment data can be combined back into the original data and provided to the upper-layer system / device (this function is the most common function of RAID 5 / RAID 3, which can handle the situation of data loss / damage of one data area / hard drive; the function of RAID 6 is to generate two levels / two sets of XOR data on top of RAID 5 to improve fault tolerance and provide support for the situation of simultaneous data loss / damage of two data areas / hard drives). At the same time, the slice data is rationally distributed across the hard drives to fully utilize the read / write / storage space capabilities of each hard drive, thereby greatly expanding the overall performance and capacity.

[0032] like Figure 2 As shown, current traditional RAID technologies are all based on the HBA architecture. Even for NVMe SSDs, which are now being used on a large scale (their typical interface speeds have reached 4GB / s (PCIe x2) – 16GB / s (PCIe x8), and the most common U.2 single port is the PCIe x4 interface with a speed of 8GB / s), the same connection architecture as SATA / SAS HDD / SSD is used.

[0033] from Figure 2 It's clear that in the traditional HBA connection architecture, the interface between the HBA card / RAID card and bus 3 becomes an inevitable bottleneck when supporting NVMe SSDs. (The traditional HBA card / RAID card's bus 3 interface is only PCIe x8 (Broadcom's LSI Raid card has begun to push PCIe x16 interfaces, but is limited by the current PCIe interface specification, which only allows for a maximum of PCIe x16, meaning PCIe x16 is the highest number of lanes per interface in the current bus 3 interface specification)).

[0034] RAID cards based on the HBA architecture remain mainstream in the current market. Enterprise-level products include Brocade LSI's RAID 11 series, which has a leading global user base; Microchip's Adaptec SmartRaid series; the emerging Highpoint RocketAIC series; and Suzhou Guoxin's CCore-Raid series in China.

[0035] Analysis of RAID technology reveals that although it evolved from HBA (High-Speed ​​RAID), its technical principles and implementation do not necessarily rely on the core functions provided by HBA—namely, the matching and connection of fast and slow devices. This is further evidenced by current industry applications. These include system-level software RAID technologies already deployed in enterprise environments (such as the mdadm software RAID module in Linux environments), mass-produced consumer-grade software RAID (such as Synology's built-in software RAID), and emerging GPU-accelerated software RAID technologies in enterprise environments (such as Grad's SupremeRAID SR-1000 series NVGPU-accelerated software RAID).

[0036] The current storage component 2 (mainly hard drives) interface has begun to accelerate its transition from SATA / SAS to NVMe. At the same time, with the improvement of the performance of SSD media itself, high-speed interfaces with performance on par with bus 3 have become inevitable for future SSD components. Under these circumstances, the traditional RAID card 11 will also be replaced by the new RAID card 11.

[0037] Analysis of the technology, performance, and functions of current traditional hardware RAID 11 cards: Traditional HBA-based RAID 11 mode, limited by the functionality and overall structural logic of the HBA, inevitably becomes a bottleneck for performance improvement and greatly hinders the widespread application of high-speed interface hard drives, given that current hard drive interface characteristics are now on par with Bus 3. Furthermore, traditional RAID technology, since its inception, has evolved from HBA cards. While this architecture provides the performance, capacity scalability, and operational reliability of HBA cards, its structural characteristics also introduce single-point-of-failure issues and the inability to perform hot maintenance.

[0038] 1. Technical Analysis: Traditional RAID functionality evolved from HBA (Hardware-Based Array of Independent Drives). It was developed to flexibly expand hard drive capacity and partially improve performance at a low cost, when hard drive capacity, performance, and interface speeds were limited. Later, with technological advancements, some reliability issues were resolved. However, based on current technology, some of RAID's basic functions and underlying architecture are no longer suitable for the current state of equipment.

[0039] 2. Performance issues: like Figure 3 As shown, the traditional RAID 11 card is developed based on HBA and matched with relatively low-speed interfaces (SATA / SAS). To simplify the overall analysis, the impact of communication and data processing losses on performance can be removed, the latency characteristics of each protocol can be ignored, and the structural model can be simplified. That is, the RAID / HBA information processing module is regarded as a central point for the aggregation of upper and lower layer data, and the processing capacity and latency of the central point are ignored. After the model simplification, the downlink data channel of the central point is the bandwidth of the connection channels between the central point and each hard drive, which is summed to form the total bandwidth. The uplink data channel is the interface between the RAID / HBA card and bus 3, and its interface bandwidth is also the interface bandwidth of the card.

[0040] For downstream bandwidth, the communication and data processing losses of the interface are simplified and latency characteristics are ignored. Based on the current SATA and SAS interface protocol versions SATA 3.0, SAS 3.0, and NVMe 1.4 Gen4, if a tri-mode interface controller card (i.e., supporting SATA / SAS / NVMe) is used, its interface speeds are SATA 6Gb / s, SAS 12Gb / s, and NVMe Gen4 PCIe x4 64Gb / s, respectively. Considering that the number of downstream hard drives connected to a RAID / HBA card in current traditional applications is usually 4-32, using hard drives of the same type and interface (in the current more traditional host hard drive support structure, a single HBA / RAID card typically connects no more than 24 hard drives, with more common numbers at 24 / 16 / 8 levels), the highest aggregated total interface speed in SATA interface mode is 32. 6Gb / s = 192Gb / s (or 24Gb / s) 6 = 144, 16 6=96、8 6=48), or 32 in SAS interface mode. 12Gb / s = 384Gb / s (or 24Gb / s) 12 = 288, 16 12 = 192, 8 12 = 96).

[0041] For the uplink channel, the analysis is simplified as well. Its uplink bandwidth is 128Gb / s for PCIe x8 (with the current mainstream PCIe Gen4 bus version 3, the peak bandwidth of the PCIe x8 interface is 16GB / s, or 128Gb / s). If some manufacturers upgrade to PCIe Gen4 x16 cards, the peak bandwidth is 32GB / s, or 256Gb / s; if the PCIe version is upgraded to PCIe Gen5 bus version 3 while retaining the PCIe x8 channel, it is still 32GB / s / 256Gb / s; if it is further expanded to PCIe x16 on the basis of Gen5 version, it is 64GB / s / 512Gb / s).

[0042] The following table shows the uplink and downlink rates and matching analysis: Table 1

[0043] Table 2

[0044] Based on the analysis in Tables 1 and 2, under simplified analysis, the traditional RAID 11 mode exhibits fundamental bottlenecks due to its traditional structure. These bottlenecks have become insurmountable obstacles for supporting new high-speed NVMe storage components, and even impose limitations on traditional SATA / SAS interface hard drives supported by the traditional RAID 11. This is why, in some configurations of the current traditional RAID 11 mode, the hard drive performance cannot be fully utilized (see the gray text in the figure, indicating parts incompatible with the current traditional RAID 11 mode). Of course, this simplified analysis only considers the theoretical uplink and downlink bandwidth, ignoring the inevitable communication losses and basic latency of the actual interface. Furthermore, due to the simplification, the impact on RAID task processing capabilities is not analyzed, and only data read tasks under normal operation of the RAID 11 and connected hard drives are considered as the simplified analysis point. However, the most basic conclusion can be drawn from the simplified architecture analysis: the traditional RAID 11 mode, developed based on the HBA model, is no longer suitable for the needs of the new high-speed NVMe storage components.

[0045] 3. Functional issues: Since its inception, RAID technology has continued to use the HBA (Hardware Builder) architecture. While RAID technology has solved the single point of failure problem for the managed storage component 2 in a RAID system, the single point of failure for the RAID card itself remains unresolved due to architectural limitations. The serial structure and the fact that all core functions pass through the RAID card makes the RAID card itself a single point of failure in the entire RAID system. This problem cannot be changed in traditional serial physical architectures.

[0046] Current system-level soft RAID technology and performance analysis: The application mode of implementing RAID at the system layer (i.e., software RAID, hereinafter referred to as the same) stems from two aspects of technological and demand changes. First, the core processing and computing power within computer systems has been greatly improved. At the same time, the improvement in device / system performance is much faster than the increase in demand, resulting in some or most of the core processing and computing resources being idle. Therefore, other task planning can be carried out based on these idle resources. Second, for some cost-sensitive / performance-insensitive users, some general overall performance can be sacrificed in exchange for the performance and reliability of the primary storage component 2 that meets the requirements.

[0047] The technical implementation of software RAID differs significantly from the physical connection architecture of traditional hardware RAID cards. If the system-level software RAID is simplified and virtualized into a single RAID card, its connection architecture is entirely different from the cascaded serial arrangement of traditional RAID cards, leaning more towards a parallel RAID card configuration. However, unlike parallel RAID cards (or parallel hardware RAID), software RAID utilizes system-level resources for RAID management and leverages core computing resources such as CPU4 and memory to perform the necessary XOR calculations. Meanwhile, the management and use of traditional hardware RAID cards are handled by the system level. This leads to significant differences in usage between software RAID and traditional RAID cards (i.e., hardware RAID). Furthermore, some advanced functions achievable with traditional hardware RAID cards, such as data protection during power outages, resource-intensive real-time data compression, and transparent data encryption, are either impossible or very difficult to implement.

[0048] The change in software RAID architecture brings advantages beyond its obvious cost reduction, such as better support for NVMe SSDs. However, while maximizing the performance of NVMe SSD interfaces, it can also lead to excessive consumption of the host's core processing and computing resources as data volume increases. In extreme cases, this can exhaust the host's core computing resources, preventing the execution of tasks other than RAID-related work. Addressing this issue of excessive host core resource consumption in large-volume software RAID environments, new vendors have developed innovative solutions. By adding computing cards to the host, they offload the most resource-intensive RAID data XOR processing from the host's CPU and memory, redirecting it to a dedicated GPU card. This achieves a solution where host resources are minimally or completely consumed under high-throughput, large-volume data conditions. Although this innovative solution / mode also has a separate card for RAID calculation, solving the problem of RAID computing power, its architecture is the same as other software RAID, and it cannot work at the device layer without the support of the system layer. At the same time, since it still needs to rely on some processing at the system layer, it cannot avoid the loss of current data caused by unexpected power failure or failure of the entire system (except for devices with overall unexpected power failure and failure data protection at the device layer, such as core mainframes).

[0049] The above analysis of system-level software RAID shows that software RAID is more suitable for cost-sensitive enterprises or individuals with basic but not very high data reliability requirements, and no fixed requirements for the overall system management. For traditional enterprise users, its characteristics cannot fully meet their requirements for RAID performance and functionality. Furthermore, even with hardware-accelerated software RAID that avoids consuming most of the host core resources, the usage pattern is still basically the same as hardware RAID, and the usage and management pattern of hardware RAID cannot be replicated. Therefore, software RAID (including hardware-accelerated software RAID) can only be used as a transitional solution in certain situations and specific systems with specific needs.

[0050] By simplifying the problem analysis and identifying the core causes, we have been able to pinpoint the shortcomings and deficiencies of traditional hardware / software RAID solutions. However, in actual system environments, RAID cards do not simply operate at their theoretical interface bandwidth or in a purely read mode. The actual architecture and technical evaluation, as well as the process of innovating new products based on the shortcomings of current traditional products / solutions, still require full consideration of the characteristics of the uplink and downlink interfaces in both traditional and innovative modes, the functional characteristics of each RAID level of the RAID card, the task characteristics of the RAID card in the system (the actual read / write ratio, etc.), user needs and usage habits, and other factors. Furthermore, based on current technology and industry standards, and combined with the direction of new technologies and standards, we need to plan the innovative aspects of this technology.

[0051] Based on the problems existing in traditional RAID technology, such as Figure 4 As shown, the present invention proposes a RAID system with a parallel structure, including a RAID module 1 and a storage component 2, wherein the RAID module 1 and the storage component 2 are respectively connected to a bus 3 for communication, and the RAID module 1 and the storage component 2 form a parallel structure.

[0052] In the above structure, the RAID module 1 consists of at least one RAID card 11, and each RAID card 11 is connected to the bus 3 for communication. When the number of RAID cards 11 is greater than one, the RAID cards 11 form a parallel structure.

[0053] In the above structure, at least one storage component 2 is included, and each Raid card 11 is communicatively connected to the bus 3. When the number of storage components 2 is greater than one, the storage components 2 form a parallel structure.

[0054] In the above structure, the RAID card 11 in the RAID module 1 is communicatively connected to each of the storage components 2.

[0055] The function of the above parallel structure is: 1. The storage component 2 is directly connected to the bus 3, which facilitates the high performance of the storage component 2 and avoids the formation of performance bottlenecks.

[0056] 2. When the RAID module 1 uses more than two RAID cards 11, since it adopts a parallel structure, if any one RAID card 11 fails, the other RAID card 11 can take over its work, thus avoiding the problem of single point of failure.

[0057] 3. When more than two RAID cards 11 are used in RAID module 1, hot-swapping is possible, which can achieve the effect of online maintenance of RAID cards 11.

[0058] 4. The number of RAID cards 11 in RAID module 1 can be expanded, and computing resources can be dynamically allocated according to the task situation to improve the working performance of RAID card 11.

[0059] Meanwhile, since it adopts a parallel structure that is directly connected to bus 3, when encapsulating the RAID system of this parallel structure, it can be encapsulated according to the required interface type. For example, adopting the NVMe specification formFact can simplify the host device type and reduce the overall complexity of the host. At the same time, with the addition of hot-swappable function, the convenience of maintenance can be further greatly improved.

[0060] like Figure 4As shown, unlike the traditional serial RAID card 11 (based on HBA structure), the parallel RAID system proposed in this invention adopts a parallel structure in physical connection. In this system, the hardware connection structure of the RAID card 11 and the storage components 2 it manages is no longer a serial architecture where the storage components 2 are connected to the RAID card 11 and the RAID card 11 is then connected to the device bus 3. Instead, the RAID card 11 and the storage components 2 it manages (mainly NVMe interface SSD hard drives, but also including traditional SATA / SAS interface storage components 2 connected to the PCIe bus 3 via traditional HBA, such as SATA / SAS...) SSDs, SATA / SASHDDs and other traditional storage components 2) are all connected in parallel to the device bus 3. (If it is a new NVMe storage component 2, then the Raid card 11 and the NVMe storage component 2 are both connected to the device's PCIe bus 3. If it is a SATA / SAS interface storage component 2, then the SATA / SAS interface storage devices are uniformly connected to the HBA for signaling and interface rate conversion, and then connected to the PCIe bus 3 through the HBA. (Some devices have an integrated HBA and related SATA / SAS interfaces on the motherboard controller chip, which is similar in structure to an independent HBA card, but simpler in structure.)

[0061] like Figure 5 As shown, in order to adapt to and meet the usage habits of traditional RAID cards 11 and improve the compatibility of existing modes in use and management, the parallel architecture of the RAID system hardware connection proposed in this invention distinguishes between the processing of control commands / data and the processing of data transmission in the system. This allows the data transmission processing part to make full use of the parallel architecture to achieve a significant performance improvement, while the control command / management work still adopts the serial logic as in traditional RAID cards 11. This allows the entire RAID system to be restored to the traditional usage and management mode, improving performance / functions without changing the original usage and management habits, thereby reducing the difficulty of accepting new technologies and reducing usage and management costs.

[0062] Serial processing under a serial structure for control commands / data will multiply the overall workload and time of control command / data processing. However, since the workload of control command / data processing is only one percent to one ten-thousandth of the total workload of control command / data processing plus data transmission processing, distinguishing the method of control command / data processing will not have a significant impact on the overall task completion efficiency.

[0063] This invention proposes a management method for a RAID system in a parallel structure, including: managing the RAID card 11; and managing the storage component 2.

[0064] When there are more than two RAID cards 11, the process includes allocating primary and secondary RAID cards 11, and the steps are as follows: Get the running status of all RAID 11 cards; One of the RAID cards 11 that is running normally is assigned as the master RAID card 11, and the other RAID cards 11 are assigned as slave RAID cards 11.

[0065] In this parallel RAID system, all workloads on the cards are distinguished into two types: management roles and workload roles. Management roles are divided into master / slave modes (one master and one slave or one master and multiple slaves). Workload roles are formed according to the functions allocated to the master RAID card 11, and workload roles include master / backup modes and dual-active / multi-active plus load balancing modes.

[0066] The purpose of differentiating management roles is primarily to ensure the uniqueness of management tasks and logic within this parallel RAID system. By dynamically monitoring the status of the master and slave cards through the load of the management roles on this parallel RAID system, and initially designating / arbitrating a unique master manager (master RAID card 11) based on conditions in case of failure, this ensures the uniqueness of management logic and avoids conflicts in management commands. Meanwhile, the workload of the management roles themselves only involves managing and arbitrating the status of multiple cards, and allocating workload tasks to the corresponding hard drives and slave RAID card 11; its workload is relatively low.

[0067] The purpose of differentiating the types of workload roles is to make reasonable use of the resources of each functional module on each RAID card 11 of this parallel RAID system when specific types of load / functions are required by the system, so as to provide higher system capabilities / functions to the outside world.

[0068] Among them, the primary and backup workload mode is mainly designed to solve the reliability problem of single point of failure, and its read and write performance is comparable to that of a single card.

[0069] The dual-active / multi-active load-balanced workload mode primarily aims to significantly improve write performance under normal operating conditions and read / write performance under RAID system degradation while addressing single-point-of-failure reliability issues. This is because both write operations under normal operating conditions and read / write operations under RAID system degradation require data to be sent to RAID card 11 for RAID calculations. In this case, interface channel width and RAID calculation acceleration capabilities become the main bottlenecks of the entire system. By using multi-card parallel load balancing scheduling, the interface channel width and RAID calculation acceleration capabilities of multiple cards can be fully utilized to significantly expand the performance of corresponding tasks.

[0070] This parallel RAID system in multi-RAID 11 mode primarily addresses single-point-of-failure issues that traditional RAID 11 cards cannot solve, as well as write performance under normal operation and read / write performance under RAID degradation. Furthermore, the hot-swap functionality in dual / multi-SIM card mode further resolves the issue of hot maintenance (automatic master / slave management in multi-SIM card mode can easily perform hot maintenance functions that traditional RAID 11 cards cannot support, enabling hot maintenance of equipment components under both fault and non-fault planning conditions).

[0071] During normal read tasks, only the master card, acting as the manager, participates in task allocation and management. In the workload, data read tasks are directly loaded from the corresponding hard drive into the corresponding data area in memory. The RAID card 11 itself does not participate in the work related to the workload.

[0072] It also includes monitoring the primary RAID card 11, and when a failure is detected, reassigning one of the secondary RAID cards 11 to the primary RAID card 11. The reallocation of the primary RAID card 11 is carried out with reference to two dimensions: the fastest completion of the allocation or the least impact on normal tasks.

[0073] Due to the architectural change, this parallel RAID system can address the single point of failure issue inherent in traditional RAID cards when the number of RAID cards 11 in the installed and operational RAID module 1 exceeds one. Through heartbeat information exchange and priority arbitration among the management roles on each RAID card 11, the master-slave relationship of the management roles on each RAID card 11 can be dynamically switched in the event of a failure. This prevents the master management role from ceasing operation due to a failure of its own RAID card 11 or a failure in the operation of the master management role itself, thus avoiding the collapse of the entire RAID system.

[0074] The heartbeat information includes key information such as time information, device and role status and load information, Raid information, and task completion information (task completion status table, task data information table, etc.).

[0075] The master RAID card 11 manages each slave RAID card 11 and dynamically allocates the functions of each slave RAID card 11, so that the functions of the slave RAID cards 11 are the same or different.

[0076] The RAID card 11 manages the storage component 2 according to its assigned functions.

[0077] The Raid card 11 forms different working modes based on the assigned functions.

[0078] For example, when reading and writing data to storage component 2, the main RAID card 11 can allocate functions to the slave RAID card 11 according to the amount of data to be read and written, and assign multiple RAID cards 11 to the same function to process data, thereby enhancing the data processing capability. It is also possible to assign multiple RAID cards 11 to the same function to process data according to other tasks, such as encryption or decryption tasks, thereby enhancing the data processing capability.

[0079] The above examples are only a part of the working modes, including but not limited to the examples above. Other working modes that can be obtained by allocating this technical solution according to functions will not be elaborated here.

[0080] When a new RAID card 11 is added, it is assigned as a slave RAID card 11, and the master RAID card 11 dynamically allocates the functions of the newly added slave RAID card 11.

[0081] When deleting RAID 11, the primary RAID 11 may or may not reallocate the functions of the other secondary RAID 11, depending on the functions of the primary RAID 11.

[0082] The single-card mode of a parallel RAID system primarily addresses the performance bottleneck compared to traditional RAID 11 cards. With a single parallel RAID 11 card and its managed parallel storage components 2 (4-24 U.2 / E1.s FF NVMe SSDs), its read bandwidth performance can approach the sum of the performance of all effective parallel storage component 2 interfaces, and its write performance can approach the performance of a single parallel RAID 11 card with the bus 3 interface. Its overall read and write performance will be 2-10 times that of a traditional RAID 11 environment.

[0083] For example: Using a single parallel RAID 11 card (Gen4 PCIe x8) to manage 16 parallel NVMe SSDs (Gen4 PCIe x4 U.2 / E1.s), with all 16 SSDs configured in RAID 5, the read and write performance under normal operating conditions is as follows: Read performance: (16-1) 7GB / s = 105GB / s Where “16” represents the total number of hard drives / storage components 2; “-1” represents one parity bit / disk in the case of RAID5, which does not participate in operation under normal working conditions; “7GB / s” is an estimated value for a single disk (for Gen4 PCIe x4 U.2 disks, the theoretical value is 8GB / s, and the read performance of NVMe SSD disks after tuning is usually between 6.5-7.5GB / s, so we take the approximate value of 7GB / s) as reference data.

[0084] Write performance: 12GB / s (theoretical value is 13.33GB / s)

[0085] This value represents the performance of the parallel RAID 11 interface. Due to the use of the Gen4 PCIe x8 interface, the theoretical value is 16GB / s. The theoretical value of the two NVMe SSDs Gen4 PCIe x4 U.2 is 8GB / s. After deducting the consumption of control commands / data processing and the resource loss occupied by 1 / 16 of the XOR data transmission, and considering the loss between the interface's normal data transmission rate and the theoretical transmission rate, the actual value is around 12GB / s. (Here, the RAID 5 calculation (i.e., XOR calculation) for all write data is used. Since the RAID 11 uses RAID 5 hardware acceleration circuitry, its loss can be ignored.)

[0086] The theoretical value is calculated using the formula y = (x - 1) / ((x - 1) / 2 + 1.5). B, where y is the calculated theoretical value, x is the number of disks, and B is the theoretical value of the RAID 11 bus 3 interface (RAID PCIe x8 with PCIe x4 NVMe U.2 / E1SSD).

[0087] It should be understood that the specific order or hierarchy of steps in the disclosed process is an example of an exemplary method. Based on design preferences, it should be understood that the specific order or hierarchy of steps in the process may be rearranged without departing from the scope of this disclosure. The appended method claims provide elements of various steps in an exemplary order and are not intended to limit the scope to the specific order or hierarchy described.

[0088] In the detailed description above, various features are combined together in a single embodiment to simplify this disclosure. This approach to disclosure should not be construed as reflecting an intention that embodiments of the claimed subject matter require more features than are explicitly stated in each claim. Rather, as reflected in the appended claims, the invention is presented with fewer features than all of the features in a single disclosed embodiment. Therefore, the appended claims are hereby explicitly incorporated into the detailed description, with each claim representing a separate preferred embodiment of the invention.

[0089] Those skilled in the art will also understand that the various illustrative logic blocks, modules, circuits, and algorithm steps described in conjunction with the embodiments herein can be implemented as electronic hardware, computer software, or a combination thereof. To clearly illustrate the interchangeability between hardware and software, the various illustrative components, blocks, modules, circuits, and steps described above are generally described in terms of their functionality. Whether such functionality is implemented as hardware or software depends on the specific application and the design constraints imposed on the overall system. Those skilled in the art can implement the described functionality in alternative ways for each specific application; however, such implementation decisions should not be construed as departing from the scope of this disclosure.

[0090] The steps of the methods or algorithms described in conjunction with the embodiments herein can be directly embodied in hardware, software modules executed by a processor, or a combination thereof. The software module can reside in RAM memory 5, flash memory, ROM memory 5, EPROM memory 5, EEPROM memory 5, registers, storage component 2, removable disk, CD-ROM, or any other form of storage medium well known in the art. An exemplary storage medium is connected to the processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and storage medium can reside in an ASIC. The ASIC can reside in a user terminal. Alternatively, the processor and storage medium can exist as discrete components in the user terminal.

[0091] For software implementation, the techniques described in this application can be implemented using modules (e.g., procedures, functions, etc.) that perform the functions described in this application. This software code can be stored in memory cell 5 and executed by a processor. The memory cell 5 can be implemented within the processor or outside the processor; in the latter case, it is communicatively coupled to the processor via various means, as is well known in the art.

[0092] The foregoing description includes examples of one or more embodiments. It is certainly impossible to describe all possible combinations of components or methods in order to describe the above embodiments, but those skilled in the art will recognize that the various embodiments can be further combined and arranged. Therefore, the embodiments described herein are intended to cover all such changes, modifications, and variations that fall within the scope of the appended claims. Furthermore, the term "comprising" as used in the specification or claims is interpreted in a manner similar to the term "including," as interpreted when used as a conjunction in the claims. Additionally, the use of any term "or" in the specification of the claims is intended to mean "non-exclusive or."

Claims

1. A RAID system with a parallel structure, characterized in that, include, A RAID module has at least one RAID card, which is connected to the bus for communication. When there is more than one RAID, the RAID cards are connected in parallel. When the RAID module uses more than two RAID cards, hot-swapping is allowed. And at least one storage component that is communicatively connected to the bus; wherein, the RAID module is communicatively connected to the storage component and is configured to manage the storage component; The management method for a parallel RAID system includes: managing the RAID cards; and managing the storage components. When there are more than two RAID cards, the method involves allocating master and slave RAID cards, with the following steps: obtaining the operating status of all RAID cards; assigning one RAID card with a normal operating status as the master RAID card, and the others as slave RAID cards; the master RAID card manages each slave RAID card, dynamically allocating functions to each slave RAID card, making the functions of the slave RAID cards the same or different; the slave RAID cards manage the storage components according to the allocated functions; and the slave RAID cards form different operating modes according to the allocated functions. It also includes the main RAID card dynamically allocating computing resources according to the task situation, and the main RAID card dynamically allocating the functions of each slave RAID card. According to the encryption or decryption task, multiple RAID cards are assigned to the same function to process data. The RAID system described above with a parallel structure is encapsulated using the NVMe specification.

2. The system as described in claim 1, characterized in that, It also includes monitoring the primary RAID card, and when a failure is detected, reassigning one of the secondary RAID cards as the primary RAID card. The reassignment of the primary RAID card is carried out with reference to two dimensions: the fastest reassignment and the least impact on normal tasks.

3. The system as described in claim 1, characterized in that, When a new RAID card is added, it is assigned as a slave RAID card, and the master RAID card dynamically allocates the functions of the newly added slave RAID card.

4. The system as described in claim 1, characterized in that, When a RAID card is deleted, the primary RAID card may or may not reallocate the functions of the other secondary RAID cards, depending on the functions of the deleted RAID card.

Images