Storage cluster, system, data processing method, device, medium and product

Abstract

The application discloses a storage cluster, a system, a data processing method, equipment, a medium and a product, relates to the technical field of data storage, and solves the technical problem of poor high availability of a storage system caused by poor scalability of the number of controllers and the number of data copies by controlling the number of devices in the controller unit, the switch unit and the extension memory unit, respectively, and respectively corresponding redundant devices existing in each unit, a plurality of switches of the switch unit being connected to a plurality of extension memories of the extension memory unit, and each of the plurality of extension memories storing copy data of a plurality of controllers. The memory cache data of the controller is decoupled from the DRAM memory inside the controller to the external extension memory, the deployment operation is simplified, and the data access processing can be completed in the case that N-1 controllers fail simultaneously, M-1 switches fail simultaneously and / or P-1 extension memories corresponding to the data copies fail, thereby improving the technical effect of the high availability of the storage system.

Description

Technical Field

[0001] The present application relates to the field of data storage technology, and in particular to a storage cluster, system, data processing method, device, medium and product. Background Art

[0002] Storage write caching technology typically uses a four-controller per chassis standard, expanding the controller's dynamic random access memory (DRAM). 32 controllers are implemented using eight four-controller chassis, interconnected by an external network. Scaling the number of controllers and data replicas requires complex software deployment modifications. Increasing the number of data replicas increases DRAM memory space and costs, and only one controller per chassis can fail simultaneously. Inter-chassis connectivity is poorer than internal connectivity within a chassis, and increasing the number of replicas increases data transmission between chassis and external locations, reducing the storage system's high availability.

[0003] Therefore, how to improve the high availability of the storage system while expanding the number of controllers and the number of replicas is an urgent problem that needs to be solved by those skilled in the art. Summary of the Invention

[0004] The present application provides a storage cluster, system, data processing method, device, medium and product to at least solve the problem of reduced high availability of storage systems caused by poor scalability of the number of controllers and replicas in related technologies.

[0005] The present application provides a storage cluster, including a controller unit, a switch unit, and an extended memory unit. The controller unit, the switch unit, and the extended memory unit have multiple devices, and each unit has corresponding redundant devices.

[0006] The multiple controllers of the controller unit are all connected to the host, and are respectively connected to the multiple switches of the switch unit;

[0007] The multiple switches of the switch unit are connected to the multiple extended memories of the extended memory unit, and the multiple extended memories respectively store copy data of the multiple controllers;

[0008] When a controller, switch and / or extended memory fails and the host sends a data access request, the redundant devices corresponding to the controller, switch and / or extended memory in each unit are controlled to process the controller's copy data according to the data access request.

[0009] The present application also provides a storage system, comprising a host and at least one storage cluster as described above;

[0010] The host is connected to at least one storage cluster and is used to send a data access request to the storage cluster to facilitate copy data processing by a controller in the storage cluster.

[0011] The present application also provides a data processing method, which is applied to a storage cluster, including a controller unit, a switch unit, and an extended memory unit. The controller unit, the switch unit, and the extended memory unit have multiple devices, and each unit has corresponding redundant devices. The multiple controllers of the controller unit are connected to a host and are respectively connected to multiple switches of the switch unit. The multiple switches of the switch unit are correspondingly connected to multiple extended memories of the extended memory unit, and duplicate data of the multiple controllers is stored in each of the multiple extended memories. The method includes:

[0012] Receive data access requests issued by the host;

[0013] When a controller, a switch and / or an extended memory fails, redundant devices corresponding to the controller, the switch and / or the extended memory in each unit are controlled to process the copy data of the controller according to the data access request.

[0014] The present application also provides an electronic device, comprising: a memory for storing a computer program; and a processor for implementing the steps of any of the above-mentioned data processing methods when executing the computer program.

[0015] The present application also provides a non-volatile storage medium, in which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above-mentioned data processing methods are implemented.

[0016] The present application also provides a computer program product, comprising a computer program, which implements the steps of any of the above-mentioned data processing methods when executed by a processor.

[0017] Through this application, firstly, by utilizing the multiple components within the controller unit, switch unit, and extended memory unit, each unit has its own corresponding redundant component, breaking the conventional expansion scheme of one frame and four controllers. By utilizing components such as the switch unit and extended memory unit, the memory cache data of the controller in the conventional one-controller-four scheme is decoupled from the DRAM memory inside the controller to the external extended memory. The hardware latency of the extended memory is smaller than that of DRAM memory, avoiding the hardware latency corresponding to the interaction between frames and the interaction between the DRAM memories within the frame in the conventional scheme, thereby improving data processing efficiency. The cost of the extended memory is lower than that of DRAM memory, saving device costs while increasing memory space usage. Secondly, the multiple controllers of the controller unit are all connected to the host and are respectively connected to the multiple switches of the switch unit. The horizontal expansion of the multiple controllers of the controller unit does not require considering complex data distribution structures or modifying complex software programs, and can be directly expanded in the form of hardware devices. At the same time, there is no need to be limited by the size of memory space, and the controller cache data can be stored in the external extended memory through the switch. Thirdly, the multiple switches of the switch unit correspond to the multiple extended memories of the extended memory unit, and the multiple extended memories each store duplicate data of multiple controllers, so that when the controller, switch and / or extended memory fails and the host issues a data access request, the redundant devices corresponding to the controller, switch and / or extended memory in each unit are controlled to process the controller's duplicate data according to the data access request. On the basis of achieving horizontal expansion of the controller, the number of copies corresponding to the extended memory is also expanded, without the need for complex software program deployment. If the number of copies is increased, only additional extended memory devices need to be added, simplifying the deployment operation while improving deployment efficiency. In the case of controller failure, the present application is not limited by the setting of the number of data copies. Instead, when N-1 controllers fail at the same time, the duplicate data can be stored in the extended memory of other controllers, and when there are multiple extended memories, the duplicate data will not be lost. On the basis of controller failure, when the switch and / or extended memory fails, due to the large number of copies, as long as one copy of the data is left, data access and storage can be completed. In other words, when M-1 switches fail at the same time and P-1 extended memories fail at the same time, duplicate data can be stored in the corresponding redundant devices, thereby improving the high availability of the storage system.

[0018] Therefore, it is possible to solve the technical problems that conventional technical solutions all expand the number of controllers and data copies based on the standard of one frame and four controllers, resulting in increased controller memory space and cost, poor interconnectivity due to the limitations of the machine frame, and poor high availability of the storage system. The standard expansion of one frame and four controllers is broken, and the controller's memory cache data is decoupled from the controller's internal DRAM memory to external extended memory. While saving device costs, it increases the memory occupied space, realizes the horizontal expansion of controllers and data copies, simplifies deployment operations, and allows data access processing to be completed even in the event of N-1 controller failures, M-1 switches failures, and / or P-1 data copies corresponding to extended memory failures, thereby improving the high availability of the storage system. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0020] Figure 1 A schematic diagram of a one-frame, four-controller storage cluster for conventional technical solutions.

[0021] Figure 2 A schematic diagram of the interaction between two controllers provided for a conventional technical solution;

[0022] Figure 3 A schematic diagram of the structure of a storage cluster provided in an embodiment of the present application;

[0023] Figure 4 A schematic diagram of the structure of a storage cluster corresponding to dual-copy data provided in an embodiment of the present application;

[0024] Figure 5 A schematic diagram of the structure of a storage cluster corresponding to three copies of data provided in an embodiment of the present application;

[0025] Figure 6 A schematic diagram of a switch port provided in an embodiment of the present application;

[0026] Figure 7 A schematic diagram of a port for expanding memory provided in an embodiment of the present application;

[0027] Figure 8 A flowchart of a data processing method provided in an embodiment of the present application;

[0028] Figure 9 A schematic diagram of a normal processing procedure of a data access request provided in this embodiment;

[0029] Figure 10 A schematic diagram of writing data to a takeover controller provided in an embodiment of the present application;

[0030] Figure 11 A schematic diagram of memory takeover of a controller provided in an embodiment of the present application;

[0031] Figure 12 A structural diagram of a data processing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0032] The following will be combined with the accompanying drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0033] It should be noted that, in the description of this application, the terms "comprises," "includes," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. The terms "first," "second," etc., in this application are used to distinguish similar objects, and are not used to describe a particular order or sequence.

[0034] In order to enable those skilled in the art to better understand the present application, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0035] Conventional storage write caching technology ensures high availability of the storage system, requiring careful consideration of the number of controllers, the number of data replicas, and the maximum number of controller failures that can be tolerated. The software and hardware design of the write cache is closely related to the high availability of the storage system (cluster). Figure 1 A schematic diagram of a one-frame, four-controller storage cluster provided for conventional technical solutions, such as Figure 1As shown, this storage cluster implements dual data replication, but supports only one controller failure simultaneously or three controller failures in sequence. The support for only one controller failure simultaneously is based on the fact that each data block has two replicas. If one controller fails, the other replica remains available. Any two nodes are still connected using high-speed Remote Direct Memory Access (RDMA). Each data block must be replicated in the other controller using RDMA. The dual data replicas form a loop structure across the four controllers (controllers B1, B2, B3, and B4). In a high-availability scenario, even if any controller fails, data is not lost due to the presence of data replicas, and storage services can continue to function normally. However, if more than two controllers fail simultaneously, data loss occurs due to the existence of only two replicas, resulting in poor system availability.

[0036] Furthermore, in a four-controller chassis configuration, achieving 32 controllers requires eight chassis with four controllers, interconnected by an external network. However, inter-chassis connectivity is poor, and cross-chassis data flow can impact system performance. Designing a loop structure in a 32-controller system with both intra-chassis and inter-chassis interconnections is challenging. The availability of a 32-controller cluster from a typical vendor is the same as that of a four-controller cluster. It can only support the simultaneous failure of one of the four controllers in each chassis, or the sequential failure of three of the four controllers (repairs can be performed as they fail, but simultaneous failure of two controllers is not possible). Figure 2 The interaction diagram between the two controllers provided by the conventional technical solution is as follows: Figure 2 As shown, host A1 and host A2 write to their respective connected controllers B1 and B2. The two controllers use an RDMA link (remote direct memory access link) for transmission. The memory data between the two controllers forms dual copies of data, which requires input and output write synchronization.

[0037] Due to the poor scalability of replicas, typical centralized storage systems use a 2-replica structure with 4 controllers using a circular replication structure. To achieve 3 replicas across 32 controllers, a 2-replica structure is typically used within the 4 controllers, with the third replica located in a separate storage frame. Replicas are generally not increased because each additional data replica doubles the memory usage and memory costs. It also requires additional data transfer within and between frames, impacting performance.

[0038] The structure of the storage system provided in this application can solve the above technical problems.

[0039] Figure 3 A schematic diagram of a storage cluster structure provided in an embodiment of the present application is shown in FIG. Figure 3As shown, it includes a controller unit 1, a switch unit 2 and an extended memory unit 3. The number of devices in the controller unit 1, the switch unit 2 and the extended memory unit 3 is multiple, and each unit has its own corresponding redundant device;

[0040] The multiple controllers of the controller unit 1 are all connected to the host, and are also connected to the multiple switches of the switch unit 2 respectively;

[0041] The multiple switches of the switch unit 2 are connected to the multiple extended memories of the extended memory unit 3, and the multiple extended memories respectively store the copy data of the multiple controllers;

[0042] When a controller, switch and / or extended memory fails and the host sends a data access request, the redundant devices corresponding to the controller, switch and / or extended memory in each unit are controlled to process the controller's copy data according to the data access request.

[0043] Specifically, there are multiple devices in each of the controller unit, switch unit, and extended memory unit. That is, one controller unit includes multiple controllers, one switch unit includes multiple switches, and one extended memory unit includes multiple extended memories. Each unit has corresponding redundant devices. That is, the multiple devices in each unit will correspond to each other's redundant devices. Figure 3 In the example, controllers B1 to B8 are used to represent the expansion. More controllers, switches D1 to D2, and extended memory E1 to E2 can be expanded horizontally, allowing for more switches and extended memory. Inside the switch, there are different ports. Taking switch D1 as an example, ports P9-P16 are used to connect to each controller, and ports P1-P8 are used to connect to ports P1-P8 in extended memory E1. Within extended memory E1, there are memory controllers F1 and F2, which write cached data to corresponding memories G1-G4, respectively. Furthermore, there may be multiple memory controllers and multiple memories in the extended memory. The number can be set based on the specific situation and is not limited here.

[0044] Multiple controllers are connected to the host computer and transmit data corresponding to input / output (IO) requests from the host to the controller. The controller then controls the data transmission to the extended memory for caching and ultimately writes it to the hard drive. It should be noted that during this process, the host transfers and flushes data between the controller and the hard drive. The controller manages data access and transmission, while the switch provides network connectivity between these devices, ensuring efficient and stable data transmission. The extended memory acts as a buffer, temporarily storing IO data from the host before flushing it to the hard drive. This improves data write performance and reduces direct write operations to the hard drive.

[0045] Multiple controllers are connected to multiple switches, each of which must be connected to all switches to facilitate the transfer of replica data from each controller to the corresponding extended memory via the switch connections. Multiple switches correspond to multiple extended memories connected to the extended memory unit. Here, one port of a switch corresponds to one port of the extended memory. Replica data for all controllers is stored in one extended memory. For example, if the switch unit includes two switches and the extended memory unit includes two extended memories, the eight ports of one switch correspond to the eight ports of the extended memory.

[0046] It's worth noting that switch units can be arranged in parallel or in cascade. Parallel means multiple switches are laid out in parallel. Cascade means one switch is connected to the next. Multiple physical switches are connected together to facilitate scaling to connect more controllers. In Compute Express Link (CXL) 3.0, this can be expanded to thousands of controllers.

[0047] With such a storage cluster connection, if any device in any unit fails and the host issues a data access request, the redundant device in each unit can be controlled to process the controller's copy of the failed device. The execution entities here are the controllers in the control unit. The controllers are responsible for handling cluster metadata management, data distribution, fault recovery, and load balancing, and may be distributed to each node in the cluster. There will be a master controller among multiple controllers. If the master controller fails, the cluster will re-determine a new master controller as the execution entity based on the other controllers through distributed storage.

[0048] If a controller fails, the controller's corresponding redundant device will be needed to process the duplicate data handled by that controller. If a switch fails, the controller will need to transfer duplicate data to the extended memory through the failed switch's redundant device for processing. If the extended memory fails, the controller will need to process the duplicate data stored by the failed extended memory's redundant device. It should be noted that even if all three fail, duplicate data processing can still be achieved through redundant devices. A maximum of N-1 controllers, M-1 switches, and P-1 extended memories can fail simultaneously, and duplicate data processing can still be achieved through their respective redundant devices.

[0049] Through the embodiments of the present application, firstly, by using the number of devices in the controller unit, switch unit, and extended memory unit, each unit has its own corresponding redundant device, breaking the conventional expansion scheme of one frame and four controllers. Through devices such as the switch unit and the extended memory unit, the memory cache data of the controller of the conventional one-controller-four scheme is decoupled from the DRAM memory inside the controller to the external extended memory. The hardware delay of the extended memory is smaller than that of the DRAM memory, avoiding the hardware delay corresponding to the interaction between frames and the interaction of each DRAM memory within the frame in the conventional scheme, thereby improving data processing efficiency. The cost of the extended memory is lower than that of the DRAM memory, saving device costs while increasing the memory space occupied. Secondly, the multiple controllers of the controller unit are all connected to the host and are respectively connected to the multiple switches of the switch unit. The horizontal expansion of the multiple controllers of the controller unit does not need to consider the complex data distribution structure and modify the complex software program. It can be expanded directly in the form of hardware devices. At the same time, it is not limited by the memory space size issue. The controller cache data is stored in the external extended memory through the switch. Thirdly, the multiple switches of the switch unit correspond to the multiple extended memories of the extended memory unit, and the multiple extended memories each store duplicate data of multiple controllers, so that when the controller, switch and / or extended memory fails and the host issues a data access request, the redundant devices corresponding to the controller, switch and / or extended memory in each unit are controlled to process the controller's duplicate data according to the data access request. On the basis of achieving horizontal expansion of the controller, the number of copies corresponding to the extended memory is also expanded, without the need for complex software program deployment. If the number of copies is increased, only additional extended memory devices need to be added, simplifying the deployment operation while improving deployment efficiency. In the case of controller failure, the present application is not limited by the setting of the number of data copies. Instead, when N-1 controllers fail at the same time, the duplicate data can be stored in the extended memory of other controllers, and when there are multiple extended memories, the duplicate data will not be lost. On the basis of controller failure, when the switch and / or extended memory fails, due to the large number of copies, as long as one copy of the data is left, data access and storage can be completed. In other words, when M-1 switches fail at the same time and P-1 extended memories fail at the same time, duplicate data can be stored in the corresponding redundant devices, thereby improving the high availability of the storage system.

[0050] Therefore, it is possible to solve the technical problems that conventional technical solutions all expand the number of controllers and data copies based on the standard of one frame and four controllers, resulting in increased controller memory space and cost, poor interconnectivity due to the limitations of the machine frame, and poor high availability of the storage system. The standard expansion of one frame and four controllers is broken, and the controller's memory cache data is decoupled from the controller's internal DRAM memory to external extended memory. While saving device costs, it increases the memory occupied space, realizes the horizontal expansion of controllers and data copies, simplifies deployment operations, and allows data access processing to be completed even in the event of N-1 controller failures, M-1 switches failures, and / or P-1 data copies corresponding to extended memory failures, thereby improving the high availability of the storage system.

[0051] In some embodiments, the multiple switches of the switch unit are in a parallel relationship, each controller is connected to the multiple switches, and the multiple switches are connected to the multiple extended memories in a one-to-one relationship.

[0052] It should be noted that in the CXL2.0 version, multiple switches are in a flat mode, each controller is connected to multiple switches, and there is a one-to-one connection relationship between multiple switches and multiple extended memories. Figure 4 A schematic diagram of a storage cluster structure corresponding to a dual copy of data provided in an embodiment of the present application is shown as follows: Figure 4 As shown in Figure 1, there are 8 controllers and 2 data copies. Each controller is connected to two switches, and each data copy of each controller is stored in the extended memory. Taking controller B1 as an example, Figure 4 In the example, the replica data are stored in the extended memory E1 and E2 respectively, and correspond to the data replicas H1 and H2.

[0053] Figure 5 A schematic diagram of a storage cluster structure corresponding to three copies of data provided in an embodiment of the present application is shown as follows: Figure 5 As shown, there are 8 controllers and 3 data copies. Each controller is connected to three switches. Each data copy of each controller is stored in the corresponding extended memory. That is, one extended memory stores a data copy corresponding to 8 controllers. Taking controller B1 as an example, Figure 5 In the example, the replica data are stored in the extended memory E1, E2 and E3 respectively, and correspond to the data replicas H1, H2 and H3.

[0054] Figure 6 A schematic diagram of a switch port provided in an embodiment of the present application is shown in FIG. Figure 6As shown, the configuration is 16 uplinks and 16 downlinks. This means two switches, with uplink port 4 on each switch connected to eight controllers and a downlink port connected to one expansion memory unit, are used. The switch hardware specifications include a 19-inch 2U standard chassis with a depth of 700mm, a width of 435mm, and a height of 87mm. A single CXL switch has 256 lanes and 16 x16 ports, configurable for both uplink and downlink. The entire system supports two CXLSwitches, each configurable for 16 uplinks and 16 downlinks. The system provides 32 x16 Common Data Format for Performance (CDFP) ports and supports CXL 2.0. External cables are connected using CDFP Direct Attach Cables (DAC).

[0055] Figure 7 A schematic diagram of a port for extending memory provided in an embodiment of the present application is shown in FIG. Figure 7 As shown, it includes two expansion memories, and the ports shown are all connected to switches, with one switch port connected to port 5 of the expansion memory. Its hardware specifications are: 19-inch 2U standard chassis, depth: 700mm; width: 435mm; height: 87mm; support for 64 fifth-generation double data rate registered dual in-line memory modules (DDR5 RDIMMs) (4800MT / s) expansion, 256GB / DIMM, with a single port supporting 1TB of memory expansion, for a maximum single-machine expanded memory capacity of 16TB; external cables: CDFP DAC cable interconnection.

[0056] This embodiment provides that each switch is directly connected to the extended memory, which can reduce data transmission delays and improve data read and write speeds. This one-to-one connection relationship simplifies the network topology and effectively utilizes network resources. Each switch can specifically handle the data traffic of the corresponding extended memory.

[0057] In some embodiments, one extended memory stores the replica data of all controllers, and the number of extended memories, the number of switches, and the number of data replicas of the controllers are the same.

[0058] Specifically, in the embodiment of the switch parallel relationship, a copy of the data of all controllers is stored in an extended memory, combined with Figure 4 and Figure 5As shown in Figure 2, each extended memory stores a copy of the data for eight controllers. The extended memory, switches, and controllers all have the same number of data copies. In other words, one extended memory contains a copy of the data for all controllers. To achieve a larger number of data copies, simply expand the switch and extended memory.

[0059] The expansion of the number of controller data copies provided in this embodiment actually corresponds to the expansion of the number of switches and extended memory. There is no need for complex software and hardware design. Switches and extended memory can be directly added, which simplifies the expansion operation and improves the expansion efficiency.

[0060] In other embodiments, considering that in the CXL3.0 version, the switches are connected in a cascade relationship, multiple switches of the switch unit are in a cascade relationship, each controller is connected to the first cascade switch, some ports of the first cascade switch are connected to the second cascade switch and part of the extended memory; the second cascade switch is connected to the remaining extended memory to store the copy data of each controller in multiple extended memories.

[0061] Specifically, multiple switches are in a cascade relationship, and each controller is connected to the first cascade switch. The first cascade switch here is for transmitting IO data that needs to be transmitted through the controller. At this time, some ports of the first cascade switch are connected to the second cascade switch and part of the extended memory. It should be noted here that, considering that in the storage system, the transmission path between the controller and the extended memory is relatively long, if you want to avoid interference during the transmission path, you need to use a cascade method of multiple switches for transmission. Therefore, due to the different transmission paths between the controller and the extended memory, some ports of the first cascade switch are connected to part of the extended memory. The remaining extended memory is connected to the second cascade switch. Regarding the second cascade relationship, there is one sub-cascade relationship or multiple sub-cascade relationships, which is not limited here and can be set according to actual conditions.

[0062] No matter how many cascades there are, the ports of the corresponding switches correspond one-to-one with the ports of the extended memory, so that each controller can implement multiple copies of data.

[0063] This embodiment provides a cascaded connection between switches and extended memory, centrally managing extended memory resources to efficiently allocate and utilize cache resources and avoid resource waste. As business grows, more switches or extended memory can be dynamically added to expand controllers without requiring software modifications. This also improves interference resistance by accommodating varying transmission paths.

[0064] Furthermore, the present application also provides a storage system, comprising a host and at least one storage cluster;

[0065] The host is connected to at least one storage cluster and is used to send a data access request to the storage cluster to facilitate copy data processing by a controller in the storage cluster.

[0066] When a host issues a data access request, the storage cluster provides the necessary interfaces and protocols to enable efficient data storage. The storage cluster manages the data lifecycle: creation, modification, deletion, and archiving. By issuing a data access request, the storage cluster identifies the corresponding controller within the storage cluster, enabling that controller to perform data transmission and caching corresponding to the data access request. Simultaneously, data replicas assigned to the controller are processed within the storage cluster.

[0067] For an introduction to a storage system provided by this application, please refer to the above method embodiment, which will not be described in detail in this application. It has the same beneficial effects as the above storage cluster.

[0068] Furthermore, the present application also provides a data processing method, which is applied to a storage cluster, including a controller unit, a switch unit, and an extended memory unit. The controller unit, the switch unit, and the extended memory unit have multiple devices, and each unit has corresponding redundant devices. The multiple controllers of the controller unit are all connected to the host and are respectively connected to the multiple switches of the switch unit. The multiple switches of the switch unit are correspondingly connected to the multiple extended memories of the extended memory unit, and the multiple extended memories each store copy data of the multiple controllers. Figure 8 A flowchart of a data processing method provided in an embodiment of the present application is shown in FIG. Figure 8 As shown, the method includes:

[0069] S11: receiving a data access request sent by the host;

[0070] S12: When a controller, a switch, and / or an extended memory fails, the redundant devices corresponding to the controller, the switch, and / or the extended memory in each unit are controlled to process the copy data of the controller according to the data access request.

[0071] Specifically, the data access request issued by the host is based on the storage of I / O data. Here, the logical address information of the corresponding controller can be determined based on the request to determine the controller to which it belongs. If a controller, switch, and / or extended memory fails during the data access request, the redundant device within each unit will be controlled to process the duplicate data according to the data access request.

[0072] There is no strict order between step S11 and step S12. The two can be in a sequential order or in parallel order. The main purpose of the storage cluster is to write the data corresponding to the data access request under the control of the controller based on the data access request, and transmit it to the corresponding extended memory through the switch in a normal data access processing mode. However, when the controller, switch and / or extended memory in the storage cluster fails, it is necessary to process the copy data of the controller through their respective redundant devices.

[0073] If the controller fails, the controller's corresponding redundant device will take over the data transfer and process the data from the failed controller's copy stored in the same extended memory. If the switch fails, based on the connection relationship discussed above, the failed switch's redundant device will be selected for data transfer to the corresponding extended memory. This means that the copy of the extended memory connected to the failed switch cannot be used, so the other copy of the data in the failed switch's redundant device will be selected. If the extended memory fails, based on the connection relationship discussed above, the other copy of the data in the failed extended memory's redundant device will be selected for processing.

[0074] It should be noted that the copy data processing here requires the redundant device of the failed controller to take over for processing in the event of a controller failure. It can be flushed to the hard disk, or continued to be stored in the extended memory, or other takeover processing, etc., which is not limited here.

[0075] For an introduction to a data processing method provided by this application, please refer to the above method embodiment, which will not be described in detail in this application. It has the same beneficial effects as the above storage cluster.

[0076] In some embodiments, the processing process corresponding to the data access request specifically includes:

[0077] Determine the corresponding target controller, target cache data, and number of data copies according to the data access request;

[0078] Controlling the target controller to write the target cache data according to the number of data copies, so as to write the data into the corresponding target extended memory;

[0079] When writing to the target extended memory is successful, responding to the host's write success instruction;

[0080] The target cache data is asynchronously flushed according to the write success instruction to write it to the hard disk.

[0081] Specifically, the corresponding target controller, target cache data, and number of data replicas are determined based on the data access request. During the data access request issuance process, the CXL protocol does not support memory sharing with central processing unit (CPU) cache coherence. When multiple controllers or servers access the same CXL extended memory, data consistency cannot be guaranteed between the caches within each CPU and the CXL extended memory. A node updating a CXL memory data page cannot invalidate the same page in another node's CPU cache. Furthermore, a node's update may remain in the CPU cache and be asynchronously flushed to CXL memory, preventing other nodes from immediately seeing the change. Specifically, when controller 1's CPU writes data to CXL, the data may be stored in controller 1's CPU cache. If other controllers also have caches of the same data, data inconsistency will occur. To address this issue, host-side data sharding is used. The host calculates the controller identification number (ID) based on the logical block address (LBA) of each I / O. This calculation can be performed using a hash function.

[0082] The value obtained by performing a modulo or remainder calculation on the hash function serves as the controller ID, thereby determining the controller (the target controller) to which the data access request is sent. The target cache data is the I / O data corresponding to the data access request. The number of data replicas corresponds to the number of possible copies.

[0083] The target controller is controlled to write the target cache data according to the number of data copies and write it to the corresponding target extended memory. The write here corresponds to the write of multiple data copies. Figure 3 For example, with dual-copy data, the target controller is controller B1, and the target cache data is written to two extended memories. After writing, a data write success instruction is required from the corresponding host. Since there is no interaction between the target controller and the remaining controllers during the entire process, compared with the conventional one-box four-controller solution, the circular mirroring between controllers and the modification of complex data distribution structures are simplified, improving the interaction efficiency of the storage cluster.

[0084] Asynchronous flushing of the target cache data based on the successful write instruction is performed because the data is flushed to the hard disk using Direct Memory Access (DMA), which does not support the CXL protocol. Therefore, asynchronous flushing is required. Asynchronous operations are not performed immediately but in the background. Unlike synchronous operations, which return results only after the data is written to the hard disk, asynchronous operations return immediately, completing the data writing in the background. It should be noted that during this process, the CXL data needs to be copied back to DRAM. The overhead of this copy step is much lower than that of remote RDMA and much less than the overhead of writing to disk, and does not affect flushing performance.

[0085] The normal processing process of the data access request provided by this embodiment writes the target cache data to the hard disk using an asynchronous flushing method, which saves data transmission overhead while improving the response speed and performance of the system.

[0086] In some embodiments, asynchronously flushing the target cache data according to the write success instruction to write it to the hard disk includes:

[0087] Read the target cache data into the target controller's own memory;

[0088] Direct memory access is used to flush the target cache data to the hard disk.

[0089] Specifically, given the complexity of the CXL protocol, DMA allows devices to directly access system memory, typically using non-coherent data transfer. CXL devices allow direct data exchange between devices without host intervention. Because DMA does not support CXL, data from extended memory cannot be directly flushed to the hard drive. Therefore, during asynchronous operations, data is first returned to the controller's internal memory and then flushed to the hard drive via DMA.

[0090] It should be noted that the target cache data is read into the target controller's own memory. This step adopts the replication method, and its replication overhead is much lower than the remote RDMA method and much less than the overhead of disk expansion.

[0091] The asynchronous flushing process provided in this embodiment falls back to the target controller's own memory, and uses DMA to access the target cache data, directly flushing the target cache data to the hard disk, avoiding normal writing to the hard disk when the DMA method does not support CXL, thereby ensuring the performance of writing to the hard disk.

[0092] In some embodiments, after asynchronously flushing the target cache data according to the write success instruction to write the target cache data to the hard disk, the method further includes:

[0093] Respond to the host's instruction of writing to the hard disk successfully;

[0094] Delete the target cache data in the target extended memory to clear the memory space of the target extended memory occupied by the target cache data.

[0095] Specifically, after writing to the hard disk, the extended memory, as the medium of the cache memory, has completed its caching task. In order to facilitate the storage of other buffered data for subsequent data access requests and reduce the memory space occupied by the extended memory, after successfully writing to the back-end hard disk, it is necessary to respond to the asynchronous flush success instruction of the write to the hard disk, and it is necessary to delete the target cache data of the extended memory. The deletion process here is to delete all copies of the target cache data to clear the memory space occupied by the target cache data.

[0096] This embodiment provides that after the target cache data is successfully written to the hard disk, all copies of the target cache data in the extended memory are deleted to completely release the cache space occupied by these copies, avoid space waste, eliminate redundancy, and ensure that the target cache data is only persistently stored in the hard disk, reducing unnecessary storage overhead.

[0097] In some embodiments, after responding to the host's instruction of writing to the hard disk successfully, the method further includes:

[0098] Determine the access status of the target cache data;

[0099] When the access state is a read access state, determining that the target cache data is read cache data;

[0100] Modify the data attribute of the target cache data from the dirty data attribute to the non-dirty data attribute;

[0101] The target cache data is not deleted and is saved in the target extended memory.

[0102] Specifically, after writing to the hard drive, cached data is typically immediately discarded from extended memory. However, if the same data is read again by the host or application very shortly after writing, the cached data will be retained in extended memory and temporarily used as read cache data. This is typically triggered by several scenarios: immediate read after write, repeated metadata access, checksum replication and snapshot synchronization, and rapid sharing of the same data by multiple processes or threads.

[0103] In this embodiment, the access status of the target cache data is determined. If it is in a read access state, it is retained to identify the target cache data as read cache data. Its data attribute is changed from dirty data to non-dirty data to distinguish it from other cache data that needs to be deleted. This cache data is not deleted and continues to be stored in the target extended memory.

[0104] Figure 9 A schematic diagram of a normal processing process of a data access request provided in this embodiment is shown as follows: Figure 9 As shown in the figure, 1) the host calculates the hash of the data shards and knows that the I / O should be sent to controller B1. 2) The write cache module writes the I / O data to extended memory E1, implementing this by calling the memcpy function. 3) The write cache module writes the I / O data to extended memory E2, implementing this by calling the memcpy function. If there are more replicas, an additional JBOM replica is written here. 4) Controller B1 responds to the host: "Data write successful." During steps 1-4, controller B1 does not interact with other controllers, which differs from traditional circular mirroring technology. 5) Asynchronous flushing of the write cache data to the disk begins. The system uses DMA to write data to the disk, but DMA does not support CXL. Therefore, the CXL data must first be copied back to DRAM. The overhead of this copy step is much lower than that of remote RDMA and much less than the overhead of disk writes, so it does not affect performance. 6) The data in DRAM is written to the backend hard drive, typically a solid-state drive (SSD). 7) The asynchronous flush succeeds, and the data in memory is written to the hard disk. 8) Delete the data in extended memory E1. If the data in the write cache is also used as read cache, there is no need to delete the data; simply change its status from "dirty data" to "not dirty data." 9) Delete the data in extended memory E2. If the data in the write cache is also used as read cache, there is no need to delete the data; simply change its status from "dirty data" to "not dirty data."

[0105] As can be seen from the above process, controller B1 does not interact with any other controllers. The software implementation of writing to CXL memory simply executes a memcpy function, resulting in minimal system overhead. This process eliminates inter-node RDMA communication and the associated software processing logic, and avoids traversing slow inter-chassis networks.

[0106] After the target cache data provided in this embodiment is written to the hard disk, it is not deleted in the extended memory based on the read access status, thereby improving the response read speed while avoiding secondary disk input and output.

[0107] In some embodiments, when a target controller fails, controlling a redundant device corresponding to the target controller to process the copy data of the target controller according to a data access request includes:

[0108] Get the physical memory address corresponding to the target controller in the target extended memory;

[0109] Establish a mapping relationship between the physical memory addresses of each controller and the target controller in advance;

[0110] When a target controller fails, a takeover controller is determined from the remaining controllers except the target controller;

[0111] Determine the logical address of the takeover controller; wherein the logical address of each controller is different and points to the physical memory address of the same target controller;

[0112] The physical memory address of the target controller is taken over according to the logical address of the takeover controller to complete the storage processing of the copy data of the target controller.

[0113] Specifically, in the above embodiment, host-side data sharding is used to circumvent the issue of cache-coherent memory sharing in CXL. In the event of a target controller failure, redundant components process the target controller's duplicate data, allowing other controllers to access the physical memory addresses corresponding to the target controller's extended memory, effectively taking over the target controller's memory. It should be noted that, regardless of whether the takeover occurs before or after the takeover, two or even multiple controllers will not simultaneously access the same physical memory address in the extended memory; rather, they will access the same physical memory address in sequence.

[0114] To obtain the physical memory address corresponding to the target controller within the target extended memory, you need to configure the CXL switch to map the same CXL physical memory to all controllers. This means that all controllers can see this segment of CXL physical memory and establish a mapping relationship between each controller and the physical memory address of the target controller. Each controller uses the numa_alloc_onnode function or the mmap system call to request the entire 128GB of CXL memory. From a software perspective, after the request, the storage software obtains a pointer to the CXL memory. Each controller uses memset or memcpy to write to the CXL memory. This causes the Linux kernel to allocate physical address space for the requested CXL memory space, thereby obtaining the physical memory address of the target controller.

[0115] In the event of a target controller failure, a takeover controller capable of taking over the target controller is found among the remaining controllers. The logical address of the takeover controller must be determined. It should be noted that this logical address differs from the logical address corresponding to the host-side data shard in the aforementioned embodiment. It is an initial logical address with an offset specific to the takeover controller added to ensure that it points to the same physical memory address of the target controller.

[0116] The physical memory address of the target controller is taken over according to the logical address of the takeover controller, and the target controller continues to complete the subsequent storage processing of the copy data, that is, the cache data.

[0117] During the memory takeover process provided in this embodiment, the logical addresses of different controllers are used to point to the physical memory addresses of the same controller, eliminating the need for each controller to maintain an independent copy of memory, which would prevent true controller device redundancy. Data exchange within the controller's internal memory is eliminated, reducing data transmission overhead and interaction costs, and improving the overall performance of the storage system.

[0118] In some embodiments, taking over the physical memory address of the target controller according to the logical address of the takeover controller includes:

[0119] Pre-control each controller to apply for the physical memory address corresponding to the target extended memory;

[0120] Aggregate the logical addresses and physical memory addresses of each controller to the main controller;

[0121] Based on the physical memory address of the main controller, the physical memory address, extended memory size, starting address of the physical memory address, and starting address of the logical address of each controller are realigned so that each controller points to the physical memory address corresponding to the extended memory of the same controller.

[0122] Sending the aligned physical memory address, the extended memory size, the starting address position of the physical memory address, and the starting address position of the logical address to the remaining controllers except the main controller;

[0123] Control the remaining controllers to modify their own logical addresses so that the physical memory address of the target controller can be taken over according to the logical address of the takeover controller.

[0124] Specifically, each controller is pre-controlled to apply for the physical memory address corresponding to the target extended memory. Each controller reads the physical memory address and logical address of the target extended memory. All controllers aggregate their respective physical memory addresses and logical addresses to a master controller via the User Datagram Protocol (UDP) / Transmission Control Protocol (TCP). The master controller uses its own physical memory address (i.e., the public physical memory address) as a reference to realign the physical memory addresses corresponding to each controller, the target extended memory size, the starting address position of the physical memory address, and the starting position of the logical address. Alignment within the physical memory address space is required. The starting address position of the physical memory address and the starting position of the logical address can be used to determine the offset corresponding to each controller, so that the subsequent address information can be sent to the remaining controllers. Each remaining controller modifies its own logical address so that the logical address of the determined takeover controller can take over the physical memory address of the target controller.

[0125] Regarding the offset setting, each controller modifies its own CXL memory logical address and size. Thus, although the CXL logical address of each controller is different, they all point to the same CXL physical address.

[0126] It should be noted that when using this CXL memory segment, data values ​​are read and written using memcpy and memset. When reading and writing CXL memory pointers, writes must be written to the CXL memory using an offset from the respective starting logical address. After reading the pointer value, add the offset to the respective starting logical address before accessing the value corresponding to the pointer.

[0127] In some embodiments, controlling other controllers to modify their own logical addresses includes:

[0128] Obtaining the first logical addresses pre-allocated by other controllers;

[0129] The first logical address and the offset determined after alignment are added together to obtain a modified logical address.

[0130] Specifically, the first logical address pre-allocated by the remaining controllers is obtained, which is the LBA logical address corresponding to the host-side data segmentation. The first logical address and the aligned offset are added to obtain the modified logical address. It should be noted that the offset is the byte distance from the starting address of the segment to the target cache data. It is a relative address, which indicates the position of the target cache data within the segment. The determination of the offset may depend on the program instructions and the layout of the data, that is, the offset may be directly specified, or the offset may be determined by pointer calculation, or may be obtained by base address and index calculation, such as the logical address is composed of a segment selector and an offset.

[0131] This embodiment sets the offset so that the logical addresses of the remaining controllers are modified to point to the physical memory address corresponding to the same target controller, which can help locate the target cache data corresponding to the target controller within the segment of the extended memory, quickly find the corresponding position, and improve the memory takeover processing efficiency.

[0132] Figure 10 A schematic diagram of writing data to a controller provided in an embodiment of the present application is shown as follows: Figure 10 As shown in the figure, the pointers of each controller point to the same CXL memory. However, during normal operation, only one controller writes data to it. Although the other controllers hold pointers to this memory, they do not write data to it.

[0133] Figure 11 A memory takeover diagram of a controller that takes over memory provided in an embodiment of the present application is shown in FIG. Figure 11 As shown in the figure, when controller B1 fails, the cluster elects controller B2, currently the lightest loaded, to continue processing incoming I / O. Data originally written by controller B1 in extended memory E1 is not lost; controller B2 continues to access this data using pointer Q2. This illustrates the process for determining the takeover controller: determining the current load of each controller; and then selecting the controller with the lowest current load among the remaining controllers, excluding the target controller.

[0134] The entire process of taking over the controller provided in this embodiment aligns the physical memory addresses and the starting positions of the logical addresses of each controller through the main controller, so that each controller can obtain a pointer to the same physical memory address, avoiding memory data interaction and transmission within the controller of one frame and four controllers, and improving data interactivity.

[0135] In some embodiments, after the target controller recovers from a failure, the method further includes:

[0136] Rejoin the restored target controller to the storage cluster;

[0137] The control takeover controller stops accessing the physical memory address of the target controller;

[0138] The host responds to a new data access request of the target controller and performs data processing on the new data access request received by the target controller.

[0139] Specifically, if controller B1 fails and the cluster detects it, the cluster's master controller (if the failed controller B1 is the master, the cluster automatically elects another controller as the master) collects the CPU loads of all controllers and selects the one with the lowest load, such as controller B2, to take over controller 1's data. The master controller notifies controller B2 via UDP / TCP to take over controller B1's CXL write cache, allowing controller B2 to read and write to controller B1's CXL memory. The master controller notifies the host's multipathing software to redirect I / O requests destined for controller B1 to controller B2. I / O proceeds normally: the multipathing software sends controller B1's I / O requests to controller B2, which writes the user I / O requests to CXL memory and asynchronously writes the data in CXL memory to the disk. As can be seen, the above process only involves the transmission of control messages between cluster controllers and between the master controller and the host, and does not generate a large amount of I / O. This is an efficient controller failover and memory data takeover process.

[0140] After the target controller recovers from the failure and rejoins the cluster, the takeover controller stops accessing the target controller's physical memory addresses and responds to new data access requests from hosts corresponding to the target controller, processing the data. The master controller notifies all hosts' multipathing software to resume I / O to controller B1. I / O continues normally: the multipathing software sends I / O to controller B1 and no longer sends I / O to controller B2. During the switchover between controllers B2 and B1, the CXL memory is external and does not need to be changed. No data transfer is required between controllers B1 and B2.

[0141] The target controller failure recovery process provided in this embodiment requires reconnecting to the storage cluster, taking over the stopped services, and then allowing the target controller to receive new data access requests. During this process, no interaction or data transmission occurs between the takeover controller and the target controller, thus avoiding transmission performance issues caused by this interaction and further improving storage system performance.

[0142] In some embodiments, when a target extended memory and / or a target switch fails, controlling a redundant device corresponding to the target extended memory and / or the target switch to process duplicate data of the target controller according to a data access request includes:

[0143] Performing interrupt processing on the hardware mechanism corresponding to the target extended memory and / or the target switch;

[0144] Report to the operating system so that the operating system can issue a hardware notification indicating that the target extended memory and / or target switch is unavailable.

[0145] Modify the device status corresponding to the target extended memory and / or the target switch to stop the processing mechanism of the target controller accessing the target extended memory and / or the target switch;

[0146] Acquire target redundant devices excluding target expansion memory and / or target switch;

[0147] The data stored in the control target redundant device is processed.

[0148] Specifically, in the event of a failure in the target extended memory and / or target switch, the corresponding redundant devices process the copy data of the target controller, and the controller hardware needs to sense that the target extended memory and / or target switch has failed. The hardware mechanism for the failure of the target extended memory and / or target switch needs to perform interrupt processing, that is, generate an interrupt to inform the operating system of the failure, so that the operating system is unavailable for the target extended memory and / or target switch. The device status of the target extended memory and / or target switch needs to be modified so that the controller no longer accesses the target extended memory and / or target switch. It should be noted here that if only the target extended memory fails, the target extended memory will not be accessed and data cannot be transmitted to the corresponding switch. If only the target switch fails, the extended memory corresponding to the target switch cannot be accessed.

[0149] At this time, it is necessary to obtain the target redundant device except the target extended memory and / or the target switch, and control the data stored in the target redundant device to perform data processing.

[0150] In the event of a failure of the target extended memory and / or target switch provided in this embodiment, it is necessary to interrupt the failed device, modify the device status of the failed device to stop the processing mechanism, and process it through the redundant device of the failed device to ensure the horizontal expansion of data copies while improving the high availability of the storage system.

[0151] In some embodiments, the target expansion memory and / or target switch failure recovery process includes:

[0152] After processing the hardware mechanism interrupt of the target extended memory and / or target switch failure, the interrupt is fed back to the target controller;

[0153] Receiving a user's recovery instruction through the target controller to trigger a recovery processing mechanism for a target extended memory and / or target switch failure;

[0154] After the target extended memory and / or target switch is recovered from the fault, the step of obtaining the physical memory address corresponding to the target controller in the target extended memory is performed to initialize the target extended memory and / or target switch;

[0155] Modify the device status corresponding to the target extended memory and / or the target switch to restore the processing mechanism for the target controller to access the target extended memory and / or the target switch.

[0156] Specifically, during the target extended memory and / or target switch fault recovery process, feedback needs to be sent to the target controller to receive the user's recovery instructions and trigger the recovery processing mechanism. The user recovery instruction here is that the user triggers the cluster's target extended memory and / or target switch device recovery process through the management software interface or command line, and performs memory initialization, that is, it is necessary to make the logical addresses of each controller point to the physical memory address of the same target controller in the target extended memory based on the pointer pointing situation of the target extended memory target controller. At this time, it is necessary to modify the device status of the target extended memory and / or target switch, write normal IO data into the target extended memory, and restore to normal.

[0157] The target extended memory and / or target switch fault recovery process provided in this embodiment can ensure that the processing process is independent of the number of controllers and various connection relationship modes, simplifying the fault handling and recovery processing, and eliminating the need for complex copy pairing of circular mirrors.

[0158] In some embodiments, the allowable range of the number of failures of the target controller is less than or equal to the total number of controllers minus 1; the allowable range of the number of failures of the target switch is less than or equal to the total number of switches minus 1; the allowable range of the number of failures of the target extended memory is less than or equal to the total number of extended memories minus 1; when the number of simultaneous failures of the target controller, target switch and / or target extended memory reaches the maximum range of the corresponding allowable range of the number of failures, the number of redundant devices corresponding to each of the target controller, target switch and target extended memory is 1.

[0159] Specifically, it can support the simultaneous failure of N-1 controllers out of N controllers, the simultaneous failure of M-1 switches out of M switches, and the simultaneous failure of P-1 extended memories out of P extended memories. In such cases, only one corresponding redundant device is retained, and the storage system can operate normally.

[0160] If N-1 controllers fail simultaneously, cache data is decoupled to external extended memory, preventing data loss. If P-1 extended memory devices fail simultaneously, as long as one remains, cache data can be preserved. Furthermore, extended memory is a hardware system with a much lower failure rate than storage controllers using complex storage software.

[0161] It should be noted that the maximum limit for simultaneous failures here is that, because the switch and the extended memory are connected in a corresponding relationship, if the target switch fails, the corresponding extended memory cannot be used even if it is functioning normally. Therefore, in this case, it is also considered that the extended memory connected to the target switch has failed. Similarly, if the target extended memory fails, the corresponding switch cannot transmit even if it is functioning normally. In this case, it is considered that the switch connected to the target extended memory has failed.

[0162] The controller unit, switch unit, and extended memory unit provided in this embodiment can ensure that when the maximum number of simultaneous failures occurs, only one device remains in normal operation, and the system can provide storage services normally to the outside world, thereby improving the high availability of the storage system.

[0163] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus the necessary general hardware platform, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method.

[0164] The embodiment of the present application further provides a data processing device, Figure 12 A structural diagram of a data processing device provided in an embodiment of the present application, such as Figure 12 As shown, the device includes:

[0165] Receiving module 11, used for receiving a data access request sent by a host;

[0166] The processing module 12 is used to control the redundant devices corresponding to the controller, switch and / or extended memory in each unit to process the controller's copy data according to the data access request when the controller, switch and / or extended memory fails.

[0167] For the description of the features in the embodiments corresponding to the data processing device, reference can be made to the relevant description of the embodiments corresponding to the data processing method, and no further details will be given here.

[0168] An embodiment of the present application further provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any of the above data processing method embodiments.

[0169] An embodiment of the present application further provides a non-volatile storage medium, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above-mentioned data processing method embodiments when running.

[0170] In an exemplary embodiment, the non-volatile storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk, or an optical disk.

[0171] An embodiment of the present application further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the steps in any one of the above data processing method embodiments are implemented.

[0172] An embodiment of the present application further provides another computer program product, including a non-volatile storage medium, wherein the non-volatile storage medium stores a computer program, and when the computer program is executed by a processor, the steps in any of the above-mentioned data processing method embodiments are implemented.

[0173] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the components and steps of each example according to their functions. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0174] The above is a detailed introduction to a storage cluster, system, data processing method, device, medium, and product provided by the present application. Specific examples are used herein to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only intended to help understand the method and core ideas of the present application. It should be noted that for ordinary technicians in this technical field, without departing from the principles of the present application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the scope of protection of the present application.

Claims

1. A storage cluster, characterized in that: The controller unit, the switch unit, and the extended memory unit are included. The controller unit, the switch unit, and the extended memory unit have multiple devices, and each unit has corresponding redundant devices. The multiple controllers of the controller unit are all connected to the host, and are respectively connected to the multiple switches of the switch unit; The multiple switches of the switch unit are connected to the multiple extended memories of the extended memory unit, and the multiple extended memories respectively store copy data of the multiple controllers; When a controller, switch, and / or extended memory fails and the host issues a data access request, the redundant devices corresponding to the controller, switch, and / or extended memory in each unit are controlled to process the controller's copy data according to the data access request; Correspondingly, the multiple switches of the switch unit are in a parallel relationship, each controller is connected to multiple switches, and the multiple switches are connected to the multiple extended memories in a one-to-one relationship; Correspondingly, one extended memory stores the copy data of all controllers, and the number of extended memories and switches is the same as the number of data copies of the controllers.

2. A storage system, characterized in that: comprising a host and at least one storage cluster according to claim 1; The host is connected to at least one storage cluster and is used to send a data access request to the storage cluster to facilitate copy data processing by a controller in the storage cluster.

3. A data processing method, characterized in that: The invention is applied to a storage cluster, comprising a controller unit, a switch unit and an extended memory unit, wherein the controller unit, the switch unit and the extended memory unit have multiple devices, and each unit has corresponding redundant devices; the multiple controllers of the controller unit are connected to a host and are respectively connected to the multiple switches of the switch unit; the multiple switches of the switch unit are correspondingly connected to the multiple extended memories of the extended memory unit, and the multiple extended memories respectively store duplicate data of the multiple controllers; the multiple switches of the switch unit are in a parallel relationship, each controller is connected to the multiple switches, and the multiple switches are in a one-to-one connection relationship with the multiple extended memories; one extended memory stores duplicate data of all controllers, and the number of extended memories, the number of switches and the number of data copies of the controllers are the same; the method comprises: Receive data access requests issued by the host; When a controller, a switch and / or an extended memory fails, redundant devices corresponding to the controller, the switch and / or the extended memory in each unit are controlled to process the copy data of the controller according to the data access request.

4. The data processing method according to claim 3, wherein: The processing process corresponding to the data access request specifically includes: Determine the corresponding target controller, target cache data and number of data copies according to the data access request; Controlling the target controller to write the target cache data according to the number of data copies, so as to write the data into the corresponding target extended memory; When writing into the target extended memory is successful, responding to a write success instruction from the host; Asynchronous flushing processing is performed on the target cache data according to the write success instruction to write it to the hard disk.

5. The data processing method according to claim 4, characterized in that: Performing asynchronous flushing processing on the target cache data according to the write success instruction to write the target cache data to the hard disk, including: Reading the target cache data into the target controller's own memory; The target cache data is written to the hard disk using a direct memory access method.

6. The data processing method according to claim 4 or 5, characterized in that: After performing asynchronous flushing processing on the target cache data according to the write success instruction to write the target cache data to the hard disk, the method further includes: Respond to the host's instruction of writing to the hard disk successfully; The target cache data in the target extended memory is deleted to clear the memory space of the target extended memory occupied by the target cache data.

7. The data processing method according to claim 6, characterized in that: After responding to the host's instruction to write to the hard disk successfully, it also includes: Determining an access status of the target cache data; When the access state is a read access state, determining that the target cache data is read cache data; Modifying the data attribute of the target cache data from a dirty data attribute to a non-dirty data attribute; The target cache data is not deleted and is saved in the target extended memory.

8. The data processing method according to claim 4, characterized in that: When the target controller fails, controlling the redundant device corresponding to the target controller to process the copy data of the target controller according to the data access request includes: Obtaining a physical memory address corresponding to the target controller in the target extended memory; Establishing a mapping relationship between each controller and the physical memory address of the target controller in advance; When the target controller fails, determining a takeover controller from the remaining controllers except the target controller; Determine the logical address of the takeover controller; wherein the logical address of each controller is different and points to the physical memory address of the same target controller; The physical memory address of the target controller is taken over according to the logical address of the takeover controller to complete the storage processing of the copy data of the target controller.

9. The data processing method according to claim 8, characterized in that: Performing takeover processing on the physical memory address of the target controller according to the logical address of the takeover controller includes: Controlling each controller to apply for the physical memory address corresponding to the target extended memory in advance; Aggregate the logical addresses and physical memory addresses of each controller to the main controller; Based on the physical memory address of the main controller, realign the physical memory address, extended memory size, starting address position of the physical memory address, and starting address position of the logical address corresponding to each controller so that each controller points to the physical memory address corresponding to the extended memory of the same controller; Sending the aligned physical memory address, the extended memory size, the starting address position of the physical memory address, and the starting address position of the logical address to the remaining controllers except the main controller; The remaining controllers are controlled to modify their own logical addresses so as to perform takeover processing on the physical memory address of the target controller according to the logical address of the takeover controller.

10. The data processing method according to claim 9, characterized in that: Control other controllers to modify their own logical addresses, including: Obtaining the first logical addresses pre-allocated by other controllers; The first logical address and the offset determined after alignment are added together to obtain a modified logical address.

11. The data processing method according to any one of claims 8 to 10, characterized in that: After the target controller fails over, it also includes: Rejoin the restored target controller to the storage cluster; Controlling the takeover controller to stop accessing the physical memory address of the target controller; The host responds to a new data access request corresponding to the target controller, and performs data processing on the new data access request received by the target controller.

12. The data processing method according to claim 8, characterized in that: When the target extended memory and / or the target switch fails, controlling the redundant device corresponding to the target extended memory and / or the target switch to process the copy data of the target controller according to the data access request includes: Performing interrupt processing on the hardware mechanism corresponding to the target extended memory and / or the target switch; Reporting to the operating system so that the operating system can issue a hardware mechanism indicating that the target extended memory and / or target switch is unavailable; Modifying the device status corresponding to the target extended memory and / or the target switch to stop the processing mechanism of the target controller accessing the target extended memory and / or the target switch; Acquire a target redundant device other than the target extended memory and / or target switch; The data stored in the control target redundant device is processed.

13. The data processing method according to claim 12, characterized in that: The target extended memory and / or target switch failure recovery process includes: After processing the hardware mechanism interrupt of the target extended memory and / or target switch failure, the interrupt is fed back to the target controller; Receiving a user's recovery instruction through the target controller to trigger a recovery processing mechanism for the target extended memory and / or target switch failure; After the target extended memory and / or target switch recovers from a fault, the step of obtaining the physical memory address corresponding to the target controller in the target extended memory is performed to initialize the target extended memory and / or target switch. The device status corresponding to the target extended memory and / or the target switch is modified to restore the processing mechanism for the target controller to access the target extended memory and / or the target switch.

14. The data processing method according to claim 12, wherein: The allowed range of the number of failures of the target controller is less than or equal to the total number of controllers minus 1; the allowed range of the number of failures of the target switch is less than or equal to the total number of switches minus 1; the allowed range of the number of failures of the target extended memory is less than or equal to the total number of extended memories minus 1; When the number of simultaneous failures of the target controller, the target switch, and / or the target extended memory reaches the maximum range of the corresponding allowed number of failures, the number of redundant devices corresponding to each of the target controller, the target switch, and the target extended memory is one.

15. An electronic device, characterized in that: include: memory for storing computer programs; A processor, configured to implement the steps of the data processing method according to any one of claims 3 to 14 when executing the computer program.

16. A non-volatile storage medium, characterized in that: The non-volatile storage medium stores a computer program, wherein the computer program implements the steps of the data processing method according to any one of claims 3 to 14 when executed by a processor.

17. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the data processing method according to any one of claims 3 to 14 are implemented.

Images