Data exchange equipment of multi-node storage equipment and multi-node storage system

Abstract

The invention provides a data exchange device of a multi-node storage device and a multi-node storage system, each of a plurality of storage nodes of the multi-node storage device comprises a plurality of node controllers and a plurality of storage spaces which are in one-to-one correspondence, and the plurality of node controllers comprise a target node controller. The data exchange equipment comprises a data controller and a plurality of equipment interfaces, and the equipment interfaces are used for being connected with target node controllers on corresponding storage nodes; the reference device interface in the plurality of device interfaces is used for acquiring the mirror image data from the connected reference node controller and transmitting the mirror image data to the data controller; and the data controller is used for synchronizing the mirror image data to storage spaces corresponding to other node controllers through other equipment interfaces. By means of the data synchronization method and device, the problem that the data synchronization efficiency among the multiple storage nodes is low is solved, and then the effect of improving the data synchronization efficiency among the multiple storage nodes is achieved.

Description

Technical Field

[0001] The embodiments of the present application relate to the computer field, and in particular, to a data exchange device of a multi-node storage device and a multi-node storage system. Background Art

[0002] Centralized array storage devices, i.e., multi-node storage devices, are usually used in scenarios with high reliability requirements. Multiple storage nodes are usually set up in a multi-node storage device. These storage nodes jointly provide storage services to the server host, and they perform both load balancing and redundant protection. In order to improve the performance and efficiency of multi-node storage devices, multiple node controllers in each storage node are usually allowed to access different parts of the storage space at the same time. In the case where the mirroring interconnection function is integrated in a node controller and the node controller can provide limited external interfaces, if the number of storage nodes set up in the multi-node storage device is too large, the current solution of connecting each storage node to each other through external interfaces to achieve mirroring interconnection will result in the mirroring data synchronization process between each storage node involving synchronous data transmission across NUMA (Non-Uniform Memory Access), which affects the data synchronization efficiency between multiple storage nodes and the data synchronization efficiency between multiple storage nodes is low. Summary of the invention

[0003] The embodiments of the present application provide a data exchange device of a multi-node storage device and a multi-node storage system, so as to at least solve the problem of low data synchronization efficiency among multiple storage nodes in the related art.

[0004] According to an embodiment of the present application, a data exchange device of a multi-node storage device is provided, wherein the multi-node storage device comprises a plurality of storage nodes, each of the plurality of storage nodes comprises a plurality of node controllers and a plurality of storage spaces corresponding to each other, the plurality of node controllers comprises a target node controller, and the data exchange device comprises: a data controller and a plurality of device interfaces, the data controller is connected to the plurality of device interfaces, the plurality of device interfaces correspond to each other to each other, and the plurality of device interfaces are used to connect to the target node controller on the corresponding storage node; the plurality of storage nodes comprises a reference storage node having mirror data to be synchronized; a reference device interface among the plurality of device interfaces is used to obtain the mirror data from the connected reference node controller and transmit the mirror data to the data controller, wherein the reference node controller is the target node controller on the reference storage node; the data controller is used to synchronize the mirror data to the storage space corresponding to the other node controller through other device interfaces, wherein the other device interfaces are device interfaces among the plurality of device interfaces except the reference device interface, and the other node controller is the target node controller connected to the other device interfaces.

[0005] In an exemplary embodiment, each storage node also includes a node interface, which is connected to the target node controller, and the device interface corresponding to the target node controller among the multiple device interfaces is used to connect to the node interface; the node interface includes: a bridge interface and / or a network interface; the reference device interface is used to receive the mirror data sent by the connected reference node interface; the data controller is used to send the mirror data to other node interfaces through the other device interfaces, wherein the other node interface is the node interface to which the other device interface is connected, and the other node interface is used to synchronize the received mirror data to the storage space corresponding to the other node controller.

[0006] In an exemplary embodiment, when the node interface includes the bridge interface, the bridge interface is used to extract the mirror data from the storage space corresponding to the reference node controller; and send the mirror data to the reference device interface; when the node interface includes a network interface, the reference node controller is used to extract the mirror data from the storage space corresponding to the reference node controller; and send the mirror data to the reference device interface through the network interface.

[0007] In an exemplary embodiment, the data controller is used to: identify the interface type of the node interface; when the device type of the node interface is identified as the bridge interface, send a first synchronization instruction to the bridge interface through the reference device interface, wherein the first synchronization instruction is used to instruct the bridge interface to synchronize data on the reference node controller; when the device type of the node interface is identified as the network interface, send a second synchronization instruction to the network interface through the reference device interface, wherein the second synchronization instruction is used to instruct the network interface to forward the second synchronization instruction to the reference node controller, and the second synchronization instruction is used to instruct the reference node controller to synchronize data.

[0008] In an exemplary embodiment, the data controller is used to read data as the mirror data from the storage space corresponding to the reference node controller through the reference device interface; and write the mirror data into the storage space corresponding to the other node controller connected to the other device interface through the other device interface.

[0009] In an exemplary embodiment, the data exchange device includes multiple switches, each of which includes a sub-controller and multiple sub-interfaces, the sub-controller in each of the switches is connected to the multiple sub-interfaces, and the multiple sub-interfaces are used to connect to the target node controller on the corresponding storage node, the mirror data is divided into multiple first sub-data, the data volume difference between any two of the multiple first sub-data is less than or equal to the target threshold, and the multiple first sub-data correspond to the multiple switches one by one; each reference sub-interface is used to obtain the first sub-data corresponding to the reference sub-interface and transmit the corresponding first sub-data to the sub-controller in the switch where the reference sub-interface is located, wherein the reference sub-interface is a sub-interface connected to the reference node controller among the multiple sub-interfaces of each of the switches; each sub-controller is used to synchronize the first sub-data to the storage space corresponding to the other node controller through other sub-interfaces on the switch where the sub-controller is located, wherein the other sub-interfaces are sub-interfaces other than the reference sub-interface among the multiple sub-interfaces in each of the switches.

[0010] In an exemplary embodiment, multiple target node controllers are interconnected; the reference node controller is used to: detect the operating status of sub-interfaces on each switch connected to the reference node controller to obtain multiple interface operating statuses, wherein the interface operating statuses are used to indicate whether the corresponding switch can normally exchange data for the reference node controller; and notify the other node controllers to stop using the reference switch to transmit data when it is detected that the reference operating status among the multiple interface operating statuses indicates that the corresponding reference switch cannot normally exchange data for the reference node controller.

[0011] In an exemplary embodiment, a plurality of the sub-controllers are connected to each other; each of the sub-controllers is used to: when synchronizing the first sub-data to the storage space corresponding to the other node controllers, send a first health signal to one or more other sub-controllers connected to the sub-controller, and receive one or more second health signals, wherein the one or more second health signals correspond to the one or more other sub-controllers one-to-one, and the health signal is used to indicate that the operation status of the switch where the corresponding sub-controller is located is normal, and the health signal includes the first health signal and the second health signal; when at least one of the one or more second health signals is not received by the target time of the target time length after sending the first health signal, send an alarm message to the connected plurality of target node controllers through the plurality of sub-interfaces on the switch where the sub-controller is located, wherein the alarm message is used to indicate that the operation status of the faulty switch where the sub-controller corresponding to the unreceived second health signal is located is abnormal; each of the target node controllers is used to stop using the faulty switch to synchronize data when receiving the alarm message; the reference node controller is also used to distribute the first sub-data corresponding to the faulty switch to switches other than the faulty switch among the plurality of switches.

[0012] According to another embodiment of the present application, a multi-node storage system is provided, comprising: a plurality of storage nodes and a data exchange device, each of the plurality of storage nodes comprising a plurality of node controllers and a plurality of storage spaces corresponding to each other, the plurality of node controllers comprising a target node controller, and the data exchange device being connected to each of the target node controllers; a reference storage node among the plurality of storage nodes, for sending the mirror data to be synchronized to the data exchange device via the reference node controller when the mirror data to be synchronized is generated, wherein the reference node controller is the target node controller on the reference storage node; the data exchange device, for obtaining the mirror data from the reference node controller, and synchronizing the mirror data to the storage space corresponding to other node controllers, wherein the other node controllers are the target node controllers among the target node controllers except the reference node controller.

[0013] In an exemplary embodiment, the data exchange device includes: a data controller and multiple device interfaces, the data controller is connected to the multiple device interfaces, the multiple device interfaces correspond one-to-one to the multiple storage nodes, and the multiple device interfaces are connected to the target node controllers on the corresponding storage nodes; a reference device interface among the multiple device interfaces is used to obtain the mirror data from the connected reference node controller and transmit the mirror data to the data controller; the data controller is used to synchronize the mirror data to the storage space corresponding to the other node controllers through other device interfaces, wherein the other device interfaces are device interfaces among the multiple device interfaces except the reference device interface.

[0014] In an exemplary embodiment, each storage node also includes a node interface, which is connected to the target node controller, and the device interface corresponding to the target node controller among the multiple device interfaces is used to connect to the node interface; the node interface includes: a bridge interface and / or a network interface; the reference device interface is used to receive the mirror data sent by the connected reference node interface; the data controller is used to send the mirror data to other node interfaces through the other device interfaces, wherein the other node interface is the node interface to which the other device interface is connected, and the other node interface is used to synchronize the received mirror data to the storage space corresponding to the other node controller.

[0015] In an exemplary embodiment, when the node interface includes the bridge interface, the bridge interface is used to extract the mirror data from the storage space corresponding to the reference node controller; and send the mirror data to the reference device interface; when the node interface includes a network interface, the reference node controller is used to extract the mirror data from the storage space corresponding to the reference node controller; and send the mirror data to the reference device interface through the network interface.

[0016] In an exemplary embodiment, the data exchange device includes multiple switches, each of which is used to connect to the target node controller on each of the storage nodes, the mirror data is divided into multiple first sub-data, the data volume difference between any two of the multiple first sub-data is less than or equal to the target threshold, and the multiple first sub-data correspond one-to-one to the multiple switches; each of the switches is used to obtain the corresponding first sub-data and synchronize the corresponding first sub-data to the storage space corresponding to the other node controllers.

[0017] In an exemplary embodiment, multiple target node controllers are interconnected; the reference node controller is used to: detect the operating status of sub-interfaces on each switch connected to the reference node controller to obtain multiple interface operating statuses, wherein the interface operating statuses are used to indicate whether the corresponding switch can normally exchange data for the reference node controller; and notify the other node controllers to stop using the reference switch to transmit data when it is detected that the reference operating status among the multiple interface operating statuses indicates that the corresponding reference switch cannot normally exchange data for the reference node controller.

[0018] In an exemplary embodiment, each of the switches includes a sub-controller and multiple sub-interfaces, the sub-controller in each of the switches is connected to the multiple sub-interfaces, the multiple sub-interfaces are used to connect to the target node controller on the corresponding storage node, and the multiple sub-controllers are connected to each other; each of the sub-controllers is used to: when synchronizing the first sub-data to the storage space corresponding to the other node controllers, send a first health signal to one or more other sub-controllers connected to the sub-controller, and receive one or more second health signals, wherein the one or more second health signals correspond one-to-one to the one or more other sub-controllers, and the health signal is used to indicate that the operating status of the switch where the corresponding sub-controller is located is normal, and the health signal The health signal includes the first health signal and the second health signal; when at least one of the one or more second health signals is not received by the target time of the target time length after the first health signal is sent, an alarm message is sent to the multiple connected target node controllers through the multiple sub-interfaces on the switch where the sub-controller is located, wherein the alarm message is used to indicate that the operating state of the faulty switch where the sub-controller corresponding to the second health signal that has not been received is abnormal; each of the target node controllers is used to stop using the faulty switch to synchronize data when receiving the alarm message; the reference node controller is also used to distribute the first sub-data corresponding to the faulty switch to switches other than the faulty switch among the multiple switches.

[0019] Through the present application, since the data controller in the data exchange device is connected to multiple device interfaces, the multiple device interfaces are used to connect to the target node controller on the corresponding storage node, and the reference device interface among the multiple device interfaces connected to the reference storage node with the mirror data to be synchronized is used to obtain the mirror data from the connected reference node controller and transmit the mirror data to the data controller, and the data controller is used to synchronize the mirror data through other device interfaces among the multiple device interfaces except the reference device interface to the storage space corresponding to the target node controller connected to the other device interfaces, so that each storage node only needs to be connected to one device interface to achieve data synchronization between multiple storage nodes, reduce the requirements on the number of external interfaces of the storage node, and avoid cross-NUMA synchronous data transmission during the data synchronization process between multiple storage nodes. Therefore, the problem of low data synchronization efficiency between multiple storage nodes can be solved, and the effect of improving the data synchronization efficiency between multiple storage nodes can be achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 is a schematic diagram of a data exchange device of a multi-node storage device according to an embodiment of the present application;

[0021] Figure 2 is a structural block diagram of an internal central processing unit according to an embodiment of the present application;

[0022] Figure 3 is a schematic diagram of an optimized four-node cache mirror interconnection according to an embodiment of the present application;

[0023] Figure 4 This is a schematic diagram of a four-node mirror interconnection of an NTB switch card according to an embodiment of the present application;

[0024] Figure 5 This is a schematic diagram of a four-node mirror interconnection of an Ethernet switch card according to an embodiment of the present application;

[0025] Figure 6 It is a schematic diagram of a multi-node storage system according to an embodiment of the present application. DETAILED DESCRIPTION

[0026] The embodiments of the present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0027] It should be noted that the terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0028] In an embodiment of the present application, a data exchange device for a multi-node storage device is provided. Figure 1 is a schematic diagram of a data exchange device of a multi-node storage device according to an embodiment of the present application, such as Figure 1 As shown, the multi-node storage device includes multiple storage nodes, each of the multiple storage nodes includes multiple node controllers and multiple storage spaces corresponding to each other, the multiple node controllers include a target node controller, the data exchange device includes: a data controller and multiple device interfaces, the data controller is connected to the multiple device interfaces, the multiple device interfaces are corresponding to the multiple storage nodes one by one, and the multiple device interfaces are used to connect to the target node controllers on the corresponding storage nodes; the multiple storage nodes include a reference storage node with mirror data to be synchronized;

[0029] A reference device interface among the plurality of device interfaces, used to obtain mirror data from a connected reference node controller and transmit the mirror data to a data controller, wherein the reference node controller is a target node controller on a reference storage node;

[0030] The data controller is used to synchronize the mirror data to the storage space corresponding to other node controllers through other device interfaces, wherein the other device interfaces are device interfaces other than the reference device interface among multiple device interfaces, and the other node controllers are target node controllers connected to the other device interfaces.

[0031] Optional, such as Figure 1 As shown, the multi-node storage device may include but is not limited to Q storage nodes, each storage node includes a plurality of node controllers and a plurality of storage spaces corresponding to each other, the plurality of node controllers include a target node controller, for example, storage node 1 includes M node controllers and M storage spaces corresponding to each other, storage node 2 includes N node controllers and N storage spaces corresponding to each other, ..., storage node Q includes P node controllers and P storage spaces corresponding to each other, storage node 1, storage node 2 and more storage nodes each include a target node controller, the data exchange device includes Q device interfaces and a data controller connected to the Q device interfaces, and the Q device interfaces are also respectively connected to the target node controllers in the corresponding storage nodes.

[0032] According to the above content, since the data controller in the data exchange device is connected to multiple device interfaces, the multiple device interfaces are used to connect to the target node controller on the corresponding storage node, the reference device interface among the multiple device interfaces connected to the reference storage node with the mirror data to be synchronized is used to obtain the mirror data from the connected reference node controller and transmit the mirror data to the data controller, and the data controller is used to synchronize the mirror data through other device interfaces among the multiple device interfaces except the reference device interface to the storage space corresponding to the target node controller connected to the other device interfaces, so that each storage node only needs to be connected to one device interface to realize data synchronization between multiple storage nodes, reduce the requirements on the number of external interfaces of the storage node, and avoid cross-NUMA synchronous data transmission during the data synchronization process between multiple storage nodes. Therefore, the problem of low data synchronization efficiency between multiple storage nodes can be solved, and the effect of improving the data synchronization efficiency between multiple storage nodes can be achieved.

[0033] Figure 2 is a structural block diagram of a central processing unit according to an embodiment of the present application. Figure 2As shown, in order to achieve the purpose of stacking cores, multiple modules (module 0-module 3) are set inside the CPU (Central Processing Unit). Different modules correspond to different storage spaces. Generally, one module supports 2 PCIe (Peripheral Component Interconnect Express) x16 roots with the best affinity and 3 memory channels (corresponding to 3 memory controllers). When such a CPU is applied to a multi-node storage device as each node controller of each storage node in the multi-node storage device, the control unit of the mirror interconnection function / storage node synchronization function is generally integrated in the cores of a module. When the interface capability provided by each module is limited, the mirror interconnection / data synchronization method of interconnecting storage nodes is adopted, which is prone to the situation that the mirror interconnection / data synchronization process between storage nodes involves cross-module transmission, that is, cross-NUMA transmission. For example, when the multi-node storage device includes 4 storage nodes, each storage node needs to provide 3 PCIe x16 interfaces. When each module in the node controller of the storage node can only provide 2 PCIe x16 roots with the best affinity, at least two modules must provide PCIe interfaces to the outside. After the mirror data / synchronization data is transmitted to the storage node, there must be cross-module and cross-NUMA data transmission, and the forwarding of the data to be transmitted must span 2 or even 3 modules. Due to the cross-module NUMA affinity, the control unit will have to carry data back and forth across different memory channels and CPU cores, resulting in low bandwidth efficiency and high latency, which affects the data synchronization efficiency between storage nodes. By utilizing the data exchange device of the multi-node storage device proposed in this application, each storage node only needs to provide one PCIe x16 interface to the outside, and the module where the control unit with integrated mirror interconnection function / storage node synchronization function is located can provide one PCIe x16 root, thereby avoiding cross-module and cross-NUMA data transmission and improving the data synchronization efficiency between storage nodes.

[0034] Optionally, in the embodiment of the present application, the multi-node storage device may be, but is not limited to, a storage device including multiple storage nodes, and the storage device including multiple storage nodes can enable other non-failed storage nodes to maintain the operation of the multi-node storage device in the event of a partial storage node failure, and the multi-node storage device can still access, read and write data. For example, it may be a multi-node storage device including 4 storage nodes or a multi-node storage device including 6 storage nodes, etc.

[0035] Optionally, in an embodiment of the present application, a storage node may include, but is not limited to, multiple node controllers and multiple storage spaces that correspond one to one. Each node controller is designed to allow simultaneous access to its corresponding storage space, and each node controller has a higher access rate to the storage space corresponding to itself. If each node controller needs to access data stored in the storage space corresponding to other node controllers, it involves the transmission of data across node controllers, i.e., cross-NUMA transmission, and the efficiency of such data transmission is relatively low.

[0036] Optionally, in the embodiment of the present application, the node controller may be, but is not limited to, a processing device with a control function, such as a processor, or a processor core, etc. This includes but is not limited to allocating one or more different services to multiple node controllers, such as node controller A is responsible for storing service data of service a, node controller B is responsible for service data of service b and service c, etc.

[0037] Optionally, in an embodiment of the present application, the target node controller includes but is not limited to a node controller in each storage node that is responsible for mirroring interconnection / data synchronization services between multiple storage nodes, and the data to be synchronized are all stored in the storage space corresponding to the target node controller.

[0038] Optionally, in an embodiment of the present application, the space sizes of each storage space corresponding to each node controller may be, but not limited to, the same or different, and the space sizes of each storage space may be, but not limited to, determined based on the amount of business data of the business undertaken by each node controller. When the amount of business data of the business undertaken by the node controller is larger, the control size of the storage space corresponding to the node controller is also larger.

[0039] Optionally, in an embodiment of the present application, the data controller may be, but is not limited to, a processing device with a control function, such as a CPU, or an FPGA (Field-Programmable Gate Array), a coprocessor, and the like.

[0040] Optionally, in an embodiment of the present application, multiple device interfaces may be, but are not limited to, interfaces corresponding to a single data transmission protocol, such as multiple PCIe interfaces or multiple Ethernet interfaces, or a combination of interfaces corresponding to multiple data transmission protocols, such as a combination of a PCIe interface and an Ethernet interface.

[0041] Optionally, in an embodiment of the present application, the mirror data to be synchronized includes but is not limited to data with a higher importance level stored in a multi-node storage device. By synchronizing the data with a higher importance level to other storage nodes in the multi-node storage device except the reference storage node, it can be ensured that in the event of failure of one or more storage nodes in the multi-node storage device, these important data can still be obtained from normal storage nodes.

[0042] Optionally, in an embodiment of the present application, the reference device interface includes but is not limited to obtaining mirror data from a connected reference node controller by receiving mirror data sent by the reference node controller. In this case, the mirror data is actively sent to the reference device interface by the reference storage node.

[0043] Optionally, in an embodiment of the present application, the reference device interface includes but is not limited to obtaining mirror data from a connected reference node controller by accepting control of the data controller and having the data controller directly read the mirror data from a storage space corresponding to the reference node controller via the reference device interface. In this case, the mirror data is actively extracted by the data exchange device.

[0044] Optionally, in an embodiment of the present application, the data controller includes but is not limited to synchronizing the mirror data to other storage nodes by sending the mirror data to other node controllers through other device interfaces. In this case, after the data controller sends the mirror data to other node controllers, the other node controllers store the received mirror data in the storage space corresponding to each other node controller, thereby completing the synchronization of the mirror data among multiple storage nodes.

[0045] Optionally, in an embodiment of the present application, the data controller includes but is not limited to synchronizing the mirror data to other storage nodes by writing the mirror data into storage spaces corresponding to other node controllers through other device interfaces.

[0046] Optionally, in an embodiment of the present application, the mirror data to be synchronized includes, but is not limited to, data to be synchronized to all other storage nodes in the plurality of storage nodes except the reference storage node, and may also be data to be synchronized to one or more target storage nodes in the plurality of storage nodes. In the case where the mirror data to be synchronized is data to be synchronized to the target storage node, it includes, but is not limited to, adding a target identifier to the mirror data to obtain the target mirror data, wherein the target identifier is used to indicate that the mirror data is data to be transmitted to the target storage node, after the reference device interface obtains the target mirror data from the connected reference node controller and transmits the target mirror data to the data controller, the data controller receives the target mirror data, the data controller detects the target mirror data, and in the case where the data controller detects that the target mirror data includes the target identifier, the target mirror data is synchronized to the storage space corresponding to the target node controller through the target device interface, wherein the target device interface is a device interface corresponding to the target storage node, and the target node controller is a target node controller on the target storage node.

[0047] As an optional implementation, each storage node also includes a node interface, which is connected to the target node controller, and the device interface corresponding to the target node controller among multiple device interfaces is used to connect to the node interface; the node interface includes: a bridge interface and / or a network interface; a reference device interface, which is used to receive the mirror data sent by the connected reference node interface; a data controller, which is used to send the mirror data to other node interfaces through other device interfaces, wherein the other node interfaces are node interfaces connected to other device interfaces, and the other node interfaces are used to synchronize the received mirror data to the storage space corresponding to the other node controllers.

[0048] Optionally, in an embodiment of the present application, each storage node may, but is not limited to, include multiple node controllers and multiple storage spaces in one-to-one correspondence, and a node interface connected to a target node controller among the multiple node controllers, and the node interface is used to externally connect to a device interface corresponding to the target node controller among multiple device interfaces of a data exchange device.

[0049] Optionally, in an embodiment of the present application, the number of node interface devices in each storage node includes but is not limited to being determined based on the number of data exchange devices connected to the multi-node storage device. For example, in order to realize disaster recovery and load balancing functions, when two data exchange devices are set up for the multi-node storage device to realize data exchange functions, the number of node interface devices in each storage node is two, and these two node interface devices are used to realize the target node controller in the storage node to be connected to two data exchange devices at the same time.

[0050] Optionally, in an embodiment of the present application, the number of node interface devices of each storage node includes but is not limited to being determined based on the amount of data that needs to be synchronized between the storage nodes. When the amount of data is large, it includes but is not limited to using multiple node interface devices to connect to the data exchange device to obtain a larger connection bandwidth and speed up data synchronization.

[0051] Optionally, the node interface includes but is not limited to a bridge interface, a network interface, and a combination of a bridge interface and a network interface.

[0052] Optionally, in an embodiment of the present application, the bridge interface may be, but is not limited to, an NTB (Non-Transparent Bridging) IC (Integrated Circuit). NTB technology is a technology based on Non-Transparent Bridge, which is mainly used to improve the communication efficiency and throughput between computer systems. NTB technology is usually used in multi-processor systems or cluster systems, and can improve system performance by connecting multiple computing nodes. NTB technology is implemented through special hardware devices, which can directly transmit data between different computing nodes without the need for transit through the host. This can greatly reduce the delay in data transmission and improve the response speed and performance of the system. NTB technology can support a variety of data transmission protocols, including PCIe protocols, etc., and can select appropriate protocols according to actual needs to achieve efficient data transmission. In the case where the node interface is a bridge interface, the data exchange device needs to have a device interface that is compatible with the bridge interface.

[0053] Optionally, in the embodiment of the present application, the network interface device may be, but is not limited to, a network IC. In the case where the node interface device is a network interface device, the data exchange device needs to have a device interface adapted to the network interface device.

[0054] Optionally, in an embodiment of the present application, when the data transmission protocol supported by the interface for outputting data to the target node controller is inconsistent with the data transmission protocol supported by the node interface, including but not limited to, the conversion device on the node interface converts the original data received on the node interface into data supporting the data transmission protocol supported by the node interface. For example, when the data transmission protocol supported by the interface for outputting data to the target node controller is the PCIe protocol and the node interface is a network interface, the conversion device on the node interface converts the received data supporting the PCIe protocol into data supporting the TCP / IP (Transmission Control Protocol / Internet Protocol) protocol.

[0055] Optionally, it includes but is not limited to selecting the type of data exchange device based on the determined type of node interface, for example, when the node interface is a bridge interface, selecting a bridge switch card as the data exchange device, and when the node interface is a network interface, selecting an Ethernet switch card as the data exchange device.

[0056] Optionally, in an embodiment of the present application, the node interface included in each storage node in the same multi-node storage device may include but is not limited to a bridge interface and a network interface, and the multiple device interfaces in the data exchange device include multiple bridge interfaces corresponding to the bridge interface and multiple network interfaces corresponding to the network interface, each bridge interface in the data exchange device is used to connect to the bridge interface of each storage node, and each network interface in the data exchange device is used to connect to the network interface of each storage node; a data controller is used to send a first test message to a target node controller connected to each bridge interface through each bridge interface, and is used to send a second test message to a target node controller connected to each network interface through each network interface, wherein the first test message is used to instruct the target node controller to send a first test completion message to the data controller through the bridge interface, and the second test message is used to instruct the target node controller to send a second test completion message to the data controller through the network interface; the bridge interface is used to send a first test message to the connected target node controller to the data controller through the bridge interface, and the second test message is used to instruct the target node controller to send a second test completion message to the data controller through the network interface; the bridge interface is used to send a first test message to the connected target node controller to the data controller through the bridge interface. The node controller sends a first test message, receives a first test completion message sent by the connected target node controller, and sends the received first test completion message to the data controller; the network interface is used to send a second test message to the connected target node controller, receive a second test completion message sent by the connected target node controller, and send the received second test completion message to the data controller; the data controller is also used to receive the first test completion message within a reference time period after sending the first test message, and to receive the second test completion message within the reference time period after sending the second test message; if the first test completion message is not received within the reference time period, the bridge interface is stopped from being used for data transmission, and the target node controller is notified through the network interface to stop using the bridge interface for data transmission; if the second test completion message is not received within the reference time period, the network interface is stopped from being used for data transmission, and the target node controller is notified through the bridge interface to stop using the network interface for data transmission. The above method can be used to arrange both bridge interface and network interface in each storage node, and make the bridge interface and network interface serve as disaster backup for each other. When the target node controller cannot use the bridge interface, the network interface can be used; when the network interface cannot be used, the bridge interface can be used, thereby avoiding the occurrence of data synchronization failures between nodes and improving the data synchronization efficiency between nodes.

[0057] Through the above content, the reference device interface receives the mirror data sent by the connected reference node interface, and the data controller sends the mirror data to other node interfaces through other device interfaces. Other node interfaces will synchronize the received mirror data to the storage space corresponding to other node controllers. The data exchange device provides a transmission channel for mirror data for the multi-node storage device. The multiple storage nodes in the multi-node storage device no longer need to rely on direct connection with each other to achieve data synchronization, which saves the external interface of the storage node and avoids cross-NUMA transmission within the storage node, thereby improving the data synchronization efficiency between nodes.

[0058] As an optional implementation, when the node interface includes a bridge interface, the bridge interface is used to extract the mirror data from the storage space corresponding to the reference node controller; and send the mirror data to the reference device interface; when the node interface includes a network interface, the reference node controller is used to extract the mirror data from the storage space corresponding to the reference node controller; and send the mirror data to the reference device interface through the network interface.

[0059] Optionally, in an embodiment of the present application, when the node interface is a bridge interface, the synchronization delay of the mirrored data is lower, and when the node interface is a network interface, the bandwidth of the mirrored data transmission channel is relatively larger.

[0060] Optionally, in an embodiment of the present application, when the node interface includes a bridge interface, it includes but is not limited to controlling the timing of the bridge interface extracting mirror data by the target node controller.

[0061] Through the above content, it is possible to select a node interface according to demand. For example, a bridge interface can be used when the synchronization delay requirement is high, and a network interface can be used when the amount of synchronization data is large. In this way, different inter-node data synchronization methods can be matched according to different specific situations, so that the synchronization of inter-node data can better meet user needs.

[0062] As an optional implementation, the data controller is used to: identify the interface type of the node interface; when the device type of the node interface is identified as a bridge interface, send a first synchronization instruction to the bridge interface through a reference device interface, wherein the first synchronization instruction is used to instruct the bridge interface to synchronize data on the reference node controller; when the device type of the node interface is identified as a network interface, send a second synchronization instruction to the network interface through the reference device interface, wherein the second synchronization instruction is used to instruct the network interface to forward the second synchronization instruction to the reference node controller, and the second synchronization instruction is used to instruct the reference node controller to synchronize data.

[0063] Through the above content, the data controller sends corresponding synchronization instructions to node interface devices of different interface types to trigger data synchronization between nodes when the interface type of the node interface device is identified. This can ensure that data synchronization is carried out when the data exchange device and the multi-node storage device are both ready, avoid the occurrence of data synchronization failure between nodes, and improve the data synchronization efficiency between nodes.

[0064] As an optional implementation, this application also provides an optimized design method for high-end storage cache mirror interconnection, and designs a method for high-speed multi-controller cache mirror interconnection that does not cross NUMA. From the hardware design, it avoids the storage device performance jitter introduced by the mirror interconnection problem across NUMA domains, and eliminates the additional software development costs caused by low bandwidth and high latency across NUMA domains. It saves the cost of interconnection materials and computing resources such as expensive PCIE channels on the CPU. It also provides a flexible and changeable cache mirror interconnection solution selection. Figure 3 FIG. 1 is a schematic diagram of an optimized four-node cache mirror interconnection according to an embodiment of the present application. After using the optimized design method of high-end storage cache mirror interconnection, the internal cache mirror interconnection of the high-end four-controller storage device is schematically shown as follows: Figure 3 As shown. The storage device includes four storage nodes (i.e., node 1 to node 4), each storage node includes multiple node controllers (i.e., different parts of the CPU) and multiple storage spaces (i.e., different parts of the memory) corresponding to each other, and the data exchange device (i.e., mirror exchange card) is connected to the target node controller among the multiple node controllers through the mirror card (i.e., node interface). Different parts of the CPU provide multiple connection interfaces to the outside, in addition to the mirror card for data transmission between nodes, it also includes an interface card that can be used to connect more storage devices or connect to server devices and other devices using the storage device.

[0065] When designing the storage device hardware, the backplane cache mirror interconnection is designed independently and replaced with a hot-swappable mirror switch card. A cache mirror switch card solution is adopted. The NTB interconnection solution is called the NTB switch card, and the Ethernet interconnection solution is called the Ethernet switch card. The NTB switch card and the Ethernet switch card are collectively referred to as the mirror switch card. Two mirror switch cards need to be designed to achieve multi-path redundancy, support hot-swappable replacement, and ensure system reliability. When a mirror switch card fails, it does not affect the normal operation of the high-end storage.

[0066] Because the mirror switch card is used in a storage device with four storage nodes in this embodiment, it is required that the number of hosts that can be supported by the port of the selected mirror switch card is greater than or equal to four.

[0067] When the mirror switch card supports the NTB solution, a PCIE NTB switch card is required; when the mirror switch card supports the Ethernet solution, an Ethernet switch card is required. When the NTB interconnection solution is adopted, an NTB IC that supports the NTB port is used, and when the Ethernet solution is adopted for interconnection, a network IC that supports the network port is used. When the mirror switch card includes the mirror switch card 1 and the mirror switch card 2, the mirror card needs to provide two ports to connect to the two cache mirror switch cards respectively. You can flexibly select the NTB solution or the Ethernet mirror interconnection solution as needed.

[0068] The CPU in the storage node uses Figure 2 In the case of the internal structure of the central processing unit shown in the figure, when the mirror card on the storage node is connected to the CPU, it is necessary to ensure that the connected PCIE roots are in the same module to avoid crossing the NUMA domain inside the CPU.

[0069] In practical applications, such as Figure 3 The storage device shown in the figure has only one frame, and one frame has four controller nodes, namely storage nodes. The storage device itself does not have a hard disk, and the data storage space used by the storage device is realized through an external disk frame. A backplane is designed in the frame of the storage device, and all storage nodes are connected to the backplane through connectors, which is convenient for hot plugging of storage nodes. The mirror switch card is not mounted on the mainboard of any storage node, but belongs to the entire storage device for four storage nodes to share, and is plugged into the common backplane of the storage device connecting the four storage nodes.

[0070] As an optional implementation, the data controller is used to: read data as mirror data from the storage space corresponding to the reference node controller through the reference device interface; and write the mirror data into the storage space corresponding to other node controllers connected to other device interfaces through other device interfaces.

[0071] Optionally, in an embodiment of the present application, including but not limited to recording in the data controller a mapping relationship between a first physical address space in the data controller and a second physical address space of a storage space corresponding to a reference node controller, and recording a mapping relationship between the first physical address space and a third physical address space of storage spaces corresponding to other node controllers, the data controller can read data from the second physical address space of the reference node controller according to its own first physical address space, and can also write mirror data to the third physical address space of other node controllers according to its own first physical address space.

[0072] In the above manner, by having the data controller in the data exchange device directly perform data read and write operations in the storage space corresponding to each target node controller, the operating pressure of the storage node can be reduced and the synchronous transmission rate of the data can also be improved.

[0073] As an optional implementation, the data exchange device includes multiple switches, each of which includes a sub-controller and multiple sub-interfaces, the sub-controller in each switch is connected to the multiple sub-interfaces, and the multiple sub-interfaces are used to connect to the target node controller on the corresponding storage node, the mirror data is divided into multiple first sub-data, the data volume difference between any two first sub-data in the multiple first sub-data is less than or equal to the target threshold, and the multiple first sub-data correspond one-to-one to the multiple switches; each reference sub-interface is used to obtain the first sub-data corresponding to the reference sub-interface, and transmit the corresponding first sub-data to the sub-controller in the switch where the reference sub-interface is located, wherein the reference sub-interface is a sub-interface connected to the reference node controller among the multiple sub-interfaces of each switch; each sub-controller is used to synchronize the first sub-data to the storage space corresponding to other node controllers through other sub-interfaces on the switch where the sub-controller is located, wherein the other sub-interfaces are sub-interfaces other than the reference sub-interface among the multiple sub-interfaces in each switch.

[0074] Optionally, in an embodiment of the present application, the sub-controller included in the switch is equivalent to the data controller included in the data exchange device, and the multiple sub-interfaces included in the switch are equivalent to the multiple device interfaces included in the data exchange device.

[0075] Optionally, in an embodiment of the present application, including but not limited to dividing the mirrored data into multiple data portions with equal data volume, the number of the multiple data portions is greater than or equal to the number of switches, and the multiple data portions can be allocated to multiple switches in a round-robin manner. For example, the mirrored data can be divided into 5 portions, namely data portion 1 to data portion 5, data portion 1 is allocated to switch 1, data portion 2 is allocated to switch 2, data portion 3 is allocated to switch 1, data portion 4 is allocated to switch 2, and data portion 5 is allocated to switch 1.

[0076] Through the above content, multiple switches are configured for the multi-node storage device, and a balanced working mechanism for multiple switches is set up, which can fully utilize the working capacity of each switch, improve the operating efficiency of the data exchange device, and improve the data synchronization efficiency between storage nodes.

[0077] As an optional implementation, multiple target node controllers are interconnected; the reference node controller is used to: detect the operating status of sub-interfaces on each switch connected to the reference node controller to obtain multiple interface operating statuses, wherein the interface operating status is used to indicate whether the corresponding switch can normally exchange data for the reference node controller; when it is detected that the reference operating status among the multiple interface operating statuses is used to indicate that the corresponding reference switch cannot normally exchange data for the reference node controller, notify other node controllers to stop using the reference switch to transmit data.

[0078] Through the above content, each target node controller will detect the operating status of the sub-interfaces of each switch connected to itself. If an abnormal status is detected, it will notify other storage nodes to no longer use the switch where the sub-interface with the abnormal operating status is located, ensuring that the data to be synchronized will not be sent normally but cannot be received normally, thereby improving synchronization efficiency.

[0079] Optional, Figure 4 : is a schematic diagram of a four-node mirror interconnection of an NTB switch card according to an embodiment of the present application, such as Figure 4 As shown in the figure, when the NTB solution is adopted, the card on the node is called an NTB mirror card (i.e., a node interface). The node mirror card needs to use an NTB card (i.e., a bridge interface), with one x16 uplink port connected to the CPU (i.e., the target node controller), and the two NTB ports of the NTB card are connected to two NTB switch cards (i.e., switches) respectively. When the mirror card on the node is connected to the CPU, it is necessary to ensure that the PCIE root is in the same module to avoid crossing the NUMA domain inside the CPU. NTB switch card 1 and NTB switch card 2 cooperate with each other to achieve load balancing and disaster preparedness. After the software of each storage node on the storage device is started, two cache mirror channels will be identified. Multipath redundancy and load balancing must be performed at the software level to give full play to the reliability and high performance of the hardware link.

[0080] Optional, Figure 5 Schematic diagram of four-node mirror interconnection of Ethernet switch cards according to an embodiment of the present application. Figure 5As shown in the figure, when the Ethernet solution is adopted, the one on the node is called an Ethernet mirror card (i.e., a node interface). The node mirror card needs to use a network card (i.e., a network interface), with one x16 uplink port connected to the CPU (i.e., the target node controller), and the two Ethernet ports of the network card are connected to two Ethernet switch cards (i.e., switches) respectively. When the mirror card on the node is connected to the CPU, it is necessary to ensure that the PCIE root is in the same module to avoid crossing the NUMA domain inside the CPU. Similar to the NTB solution, Ethernet switch card 1 and Ethernet switch card 2 work together to achieve load balancing and disaster preparedness. After the software of each storage node on the storage device is started, two cache mirror channels will be identified. Multipath redundancy and load balancing must be performed at the software level to give full play to the reliability and high performance of the hardware link.

[0081] Optionally, in the embodiments of the present application, the above contents are all based on a single CPU (i.e., the target node controller) as an example. The dual CPU design needs to evaluate whether mirror switching interconnection can be performed across CPUs based on software capabilities. The number of NUMA domains for cache mirroring interconnection can still be reduced based on the design ideas of the switching card to improve the efficiency of mirroring interconnection.

[0082] As an optional implementation, multiple sub-controllers are connected to each other; each sub-controller is used to: when synchronizing the first sub-data to the storage space corresponding to other node controllers, send a first health signal to one or more other sub-controllers connected to the sub-controller, and receive one or more second health signals, wherein the one or more second health signals correspond one-to-one to the one or more other sub-controllers, and the health signal is used to indicate that the operation status of the switch where the corresponding sub-controller is located is normal, and the health signal includes the first health signal and the second health signal; when at least one of the one or more second health signals is not received by the target time of the target time after the first health signal is sent, send an alarm message to the multiple connected target node controllers through multiple sub-interfaces on the switch where the sub-controller is located, wherein the alarm message is used to indicate that the operation status of the faulty switch where the sub-controller corresponding to the unreceived second health signal is located is abnormal; each target node controller is used to stop using the faulty switch to synchronize data when receiving the alarm message; the reference node controller is also used to distribute the first sub-data corresponding to the faulty switch to switches other than the faulty switch among the multiple switches.

[0083] Optionally, in an embodiment of the present application, the reference node controller distributes the first sub-data corresponding to the faulty switch to the switches other than the faulty switch among the multiple switches, which may be, but is not limited to, the switch with the idlest operating status other than the faulty switch, or is evenly distributed to multiple switches other than the faulty switch.

[0084] Through the above content, multiple switches monitor each other's operating status and provide their own operating status to the multi-node storage device, instead of the multi-node storage device checking the switch operating status, reducing the operating pressure of the target node controller in the multi-node storage device.

[0085] In an embodiment of the present application, a multi-node storage system is also provided. Figure 6 is a schematic diagram of a multi-node storage system according to an embodiment of the present application, such as Figure 6 As shown, the multi-node storage system includes: multiple storage nodes and a data exchange device, each of the multiple storage nodes includes multiple node controllers and multiple storage spaces corresponding to each other, the multiple node controllers include a target node controller, and the data exchange device is connected to each target node controller;

[0086] A reference storage node among the plurality of storage nodes, used for sending the mirror data to be synchronized to the data exchange device through a reference node controller when the mirror data to be synchronized is generated, wherein the reference node controller is a target node controller on the reference storage node;

[0087] The data exchange device is used to obtain the mirror data from the reference node controller and synchronize the mirror data to the storage space corresponding to other node controllers, wherein the other node controllers are target node controllers other than the reference node controller.

[0088] Through the above content, since the data exchange device is connected to each target node controller, the reference storage node among the multiple storage nodes is used to send the mirror data to the data exchange device through the reference node controller on the reference storage node when the mirror data to be synchronized is generated. The data exchange device is used to obtain the mirror data from the reference node controller and synchronize the mirror data to the storage space corresponding to other node controllers in the multiple target node controllers except the reference node controller, so that each storage node only needs to be connected to the data exchange device to realize data synchronization between multiple storage nodes, reduce the number of external interfaces required for the storage node, and avoid cross-NUMA synchronous data transmission during the data synchronization process between multiple storage nodes. Therefore, the problem of low data synchronization efficiency between multiple storage nodes can be solved, and the effect of improving the data synchronization efficiency between multiple storage nodes can be achieved.

[0089] As an optional implementation, the data exchange device includes: a data controller and multiple device interfaces, the data controller is connected to the multiple device interfaces, the multiple device interfaces correspond one-to-one to multiple storage nodes, and the multiple device interfaces are connected to target node controllers on corresponding storage nodes; a reference device interface among the multiple device interfaces is used to obtain mirror data from the connected reference node controller and transmit the mirror data to the data controller; the data controller is used to synchronize the mirror data to the storage space corresponding to other node controllers through other device interfaces, wherein the other device interfaces are device interfaces among the multiple device interfaces except the reference device interface.

[0090] As an optional implementation, each storage node also includes a node interface, which is connected to the target node controller, and the device interface corresponding to the target node controller among multiple device interfaces is used to connect to the node interface; the node interface includes: a bridge interface and / or a network interface; a reference device interface, which is used to receive the mirror data sent by the connected reference node interface; a data controller, which is used to send the mirror data to other node interfaces through other device interfaces, wherein the other node interfaces are node interfaces connected to other device interfaces, and the other node interfaces are used to synchronize the received mirror data to the storage space corresponding to the other node controllers.

[0091] As an optional implementation, when the node interface includes a bridge interface, the bridge interface is used to extract the mirror data from the storage space corresponding to the reference node controller; and send the mirror data to the reference device interface; when the node interface includes a network interface, the reference node controller is used to extract the mirror data from the storage space corresponding to the reference node controller; and send the mirror data to the reference device interface through the network interface.

[0092] Optionally, in an embodiment of the present application, the data controller is used to: identify the interface type of the node interface; when it is identified that the device type of the node interface is a bridge interface, send a first synchronization instruction to the bridge interface through the reference device interface, wherein the first synchronization instruction is used to instruct the bridge interface to synchronize data on the reference node controller; when it is identified that the device type of the node interface is a network interface, send a second synchronization instruction to the network interface through the reference device interface, wherein the second synchronization instruction is used to instruct the network interface to forward the second synchronization instruction to the reference node controller, and the second synchronization instruction is used to instruct the reference node controller to synchronize data.

[0093] Optionally, in an embodiment of the present application, the data controller is used to: read data as mirror data from the storage space corresponding to the reference node controller through the reference device interface; and write the mirror data into the storage space corresponding to other node controllers connected to other device interfaces through other device interfaces.

[0094] As an optional implementation, the data exchange device includes multiple switches, each of which is used to connect to a target node controller on each storage node, the mirror data is divided into multiple first sub-data, the data volume difference between any two of the multiple first sub-data is less than or equal to a target threshold, and the multiple first sub-data correspond one-to-one to multiple switches; each switch is used to obtain the corresponding first sub-data and synchronize the corresponding first sub-data to the storage space corresponding to other node controllers.

[0095] As an optional implementation, multiple target node controllers are interconnected; the reference node controller is used to: detect the operating status of sub-interfaces on each switch connected to the reference node controller to obtain multiple interface operating statuses, wherein the interface operating status is used to indicate whether the corresponding switch can normally exchange data for the reference node controller; when it is detected that the reference operating status among the multiple interface operating statuses is used to indicate that the corresponding reference switch cannot normally exchange data for the reference node controller, notify other node controllers to stop using the reference switch to transmit data.

[0096] As an optional implementation, each switch includes a sub-controller and multiple sub-interfaces, the sub-controller in each switch is connected to the multiple sub-interfaces, the multiple sub-interfaces are used to connect to the target node controller on the corresponding storage node, and the multiple sub-controllers are connected to each other; each sub-controller is used to: when synchronizing the first sub-data to the storage space corresponding to the other node controllers, send a first health signal to one or more other sub-controllers connected to the sub-controller, and receive one or more second health signals, wherein the one or more second health signals correspond to the one or more other sub-controllers one-to-one, and the health signal is used to indicate that the operation status of the switch where the corresponding sub-controller is located is normal, The health signal includes a first health signal and a second health signal; when at least one of the one or more second health signals is not received by the target time of the target time length after the first health signal is sent, an alarm message is sent to the multiple connected target node controllers through the multiple sub-interfaces on the switch where the sub-controller is located, wherein the alarm message is used to indicate that the operating status of the faulty switch where the sub-controller corresponding to the second health signal that has not been received is abnormal; each target node controller is used to stop using the faulty switch to synchronize data when the alarm message is received; the reference node controller is also used to distribute the first sub-data corresponding to the faulty switch to switches other than the faulty switch among the multiple switches.

[0097] Through the above content, this application provides an optimized design method for high-end storage cache mirror interconnection, which separates the mirror interconnection solution from the onboard point-to-point design interconnection through the mirror card and mirror switch card solution, and can flexibly select different cache mirror interconnection solutions according to needs. Multiple cache mirror switch cards are designed with multi-path redundancy, support hot plug replacement, and ensure system reliability. When one card fails, it does not affect the normal operation of high-end storage. And the use of onboard NTB devices / high-speed Ethernet devices is reduced, saving device costs. The reduction of devices can also make the controller node motherboard design more concise and reduce the power consumption of the equipment. In addition, PCIE channels are saved, and the saved PCIE channels can be used to connect more interface cards. At the same time, a method for high-speed multi-controller cache mirror interconnection that does not cross the CPU internal NUMA is designed. From the hardware design, the storage device performance jitter introduced by the mirror interconnection across NUMA domains is avoided, and the additional development cost of software caused by low bandwidth and high latency across NUMA domains is eliminated, which helps to improve the performance of high-end storage systems. At the same time, a flexible and changeable cache mirror interconnection solution selection is provided to avoid being bound by a single manufacturer's solution.

[0098] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the principles of the present application shall be included in the protection scope of the present application.

Claims

1. A data exchange device for a multi-node storage device, characterized in that: The multi-node storage device comprises a plurality of storage nodes, each of the plurality of storage nodes comprises a plurality of node controllers and a plurality of storage spaces corresponding to each other, the plurality of node controllers comprises a target node controller, the data exchange device comprises: a data controller and a plurality of device interfaces, the data controller is connected to the plurality of device interfaces, the plurality of device interfaces correspond to the plurality of storage nodes to each other, the plurality of device interfaces are used to connect to the target node controller on the corresponding storage node; the plurality of storage nodes comprises a reference storage node having mirror data to be synchronized; A reference device interface among the plurality of device interfaces is used to obtain the mirror data from a connected reference node controller and transmit the mirror data to the data controller, wherein the reference node controller is a target node controller on the reference storage node; The data controller is used to synchronize the mirror data to the storage space corresponding to the other node controller through other device interfaces, wherein the other device interface is a device interface among the multiple device interfaces except the reference device interface, and the other node controller is a target node controller connected to the other device interface.

2. The data exchange device of the multi-node storage device according to claim 1, characterized in that: Each storage node further comprises a node interface, the node interface is connected to the target node controller, and the device interface corresponding to the target node controller among the plurality of device interfaces is used to connect to the node interface; The node interface includes: a bridge interface and / or a network interface; The reference device interface is used to receive the mirror data sent by the connected reference node interface; The data controller is used to send the mirror data to other node interface devices through the other device interface, wherein the other node interface device is a node interface device to which the other device interface is connected, and the other node interface device is used to synchronize the received mirror data to the storage space corresponding to the other node controller.

3. The data exchange device of the multi-node storage device according to claim 2, characterized in that: In the case where the node interface includes the bridge interface, the bridge interface is used to extract the mirror data from the storage space corresponding to the reference node controller; and send the mirror data to the reference device interface; In the case where the node interface includes a network interface, the reference node controller is used to extract the mirror data from the storage space corresponding to the reference node controller; and send the mirror data to the reference device interface through the network interface.

4. The data exchange device of the multi-node storage device according to claim 3, characterized in that: The data controller is responsible for: Identifying an interface type of the node interface; In the case where it is identified that the device type of the node interface is the bridge interface, sending a first synchronization instruction to the bridge interface through the reference device interface, wherein the first synchronization instruction is used to instruct the bridge interface to synchronize data on the reference node controller; When it is identified that the device type of the node interface is the network interface, a second synchronization instruction is sent to the network interface through the reference device interface, wherein the second synchronization instruction is used to instruct the network interface to forward the second synchronization instruction to the reference node controller, and the second synchronization instruction is used to instruct the reference node controller to synchronize data.

5. The data exchange device of the multi-node storage device according to claim 1, characterized in that: The data controller is responsible for: Reading data from a storage space corresponding to the reference node controller through the reference device interface as the mirror data; The mirror data is written into a storage space corresponding to the other node controller connected to the other device interface through the other device interface.

6. The data exchange device of the multi-node storage device according to claim 1, characterized in that: The data exchange device includes a plurality of switches, each of the switches includes a sub-controller and a plurality of sub-interfaces, the sub-controller in each of the switches is connected to the plurality of sub-interfaces, the plurality of sub-interfaces are used to connect to the target node controller on the corresponding storage node, the mirror data is divided into a plurality of first sub-data, the data volume difference between any two of the plurality of first sub-data is less than or equal to the target threshold, and the plurality of first sub-data corresponds to the plurality of switches one by one; Each reference sub-interface is used to obtain the first sub-data corresponding to the reference sub-interface, and transmit the corresponding first sub-data to the sub-controller in the switch where the reference sub-interface is located, wherein the reference sub-interface is a sub-interface connected to the reference node controller among the multiple sub-interfaces of each of the switches; Each of the sub-controllers is used to synchronize the first sub-data to the storage space corresponding to the other node controllers through other sub-interfaces on the switch where the sub-controller is located, wherein the other sub-interfaces are sub-interfaces among the multiple sub-interfaces in each of the switches except the reference sub-interface.

7. The data exchange device of the multi-node storage device according to claim 6, characterized in that: A plurality of the target node controllers are connected to each other; The reference node controller is used to: Detecting the operating status of the sub-interfaces on each switch connected to the reference node controller to obtain a plurality of interface operating statuses, wherein the interface operating statuses are used to indicate whether the corresponding switch can normally perform data exchange for the reference node controller; When it is detected that the reference operating state among the multiple interface operating states is used to indicate that the corresponding reference switch cannot normally exchange data for the reference node controller, the other node controllers are notified to stop using the reference switch to transmit data.

8. The data exchange device of the multi-node storage device according to claim 6, characterized in that: A plurality of said sub-controllers are connected to each other; Each of the sub-controllers is used to: when synchronizing the first sub-data to the storage space corresponding to the other node controllers, send a first health signal to one or more other sub-controllers connected to the sub-controller, and receive one or more second health signals, wherein the one or more second health signals correspond to the one or more other sub-controllers one-to-one, and the health signal is used to indicate that the operation status of the switch where the corresponding sub-controller is located is normal, and the health signal includes the first health signal and the second health signal; when at least one of the one or more second health signals is not received by the target time of the target time length after the first health signal is sent, send alarm information to the multiple target node controllers connected thereto through the multiple sub-interfaces on the switch where the sub-controller is located, wherein the alarm information is used to indicate that the operation status of the faulty switch where the sub-controller corresponding to the second health signal that is not received is abnormal; Each of the target node controllers is used to stop using the faulty switch to synchronize data when receiving the alarm information; The reference node controller is further configured to distribute the first sub-data corresponding to the failed switch to switches other than the failed switch among the plurality of switches.

9. A multi-node storage system, It is characterized in that The method comprises: a plurality of storage nodes and a data exchange device, wherein each storage node in the plurality of storage nodes comprises a plurality of node controllers and a plurality of storage spaces corresponding to each other, the plurality of node controllers comprises a target node controller, and the data exchange device is connected to each of the target node controllers; A reference storage node among the plurality of storage nodes, configured to, when mirror data to be synchronized is generated, send the mirror data to the data exchange device through a reference node controller, wherein the reference node controller is a target node controller on the reference storage node; The data exchange device is used to obtain the mirror data from the reference node controller and synchronize the mirror data to the storage space corresponding to other node controllers, wherein the other node controllers are target node controllers other than the reference node controller in the target node controller.

10. The multi-node storage system according to claim 9, characterized in that: The data exchange device comprises: a data controller and a plurality of device interfaces, the data controller is connected to the plurality of device interfaces, the plurality of device interfaces correspond to the plurality of storage nodes one by one, and the plurality of device interfaces are connected to the target node controllers on the corresponding storage nodes; A reference device interface among the plurality of device interfaces is used to obtain the mirror data from the connected reference node controller and transmit the mirror data to the data controller; The data controller is used to synchronize the mirror data to the storage space corresponding to the other node controller through other device interfaces, wherein the other device interfaces are device interfaces other than the reference device interface among the multiple device interfaces.

11. The multi-node storage system according to claim 10, characterized in that: Each storage node further comprises a node interface, the node interface is connected to the target node controller, and the device interface corresponding to the target node controller among the plurality of device interfaces is used to connect to the node interface; The node interface includes: a bridge interface and / or a network interface; The reference device interface is used to receive the mirror data sent by the connected reference node interface; The data controller is used to send the mirror data to other node interface devices through the other device interface, wherein the other node interface device is a node interface device to which the other device interface is connected, and the other node interface device is used to synchronize the received mirror data to the storage space corresponding to the other node controller.

12. The multi-node storage system according to claim 11, characterized in that: In the case where the node interface includes the bridge interface, the bridge interface is used to extract the mirror data from the storage space corresponding to the reference node controller; and send the mirror data to the reference device interface; In the case where the node interface includes a network interface, the reference node controller is used to extract the mirror data from the storage space corresponding to the reference node controller; and send the mirror data to the reference device interface through the network interface.

13. The multi-node storage system according to claim 9, characterized in that: The data exchange device includes a plurality of switches, each of which is used to connect to the target node controller on each of the storage nodes, the mirror data is divided into a plurality of first sub-data, a data volume difference between any two of the plurality of first sub-data is less than or equal to a target threshold, and the plurality of first sub-data corresponds to the plurality of switches one by one; Each of the switches is used to obtain the corresponding first sub-data and synchronize the corresponding first sub-data to the storage space corresponding to the other node controllers.

14. The multi-node storage system according to claim 13, characterized in that: A plurality of the target node controllers are connected to each other; The reference node controller is used to: Detecting the operating status of the sub-interfaces on each switch connected to the reference node controller to obtain a plurality of interface operating statuses, wherein the interface operating statuses are used to indicate whether the corresponding switch can normally perform data exchange for the reference node controller; When it is detected that the reference operating state among the multiple interface operating states is used to indicate that the corresponding reference switch cannot normally exchange data for the reference node controller, the other node controllers are notified to stop using the reference switch to transmit data.

15. The multi-node storage system according to claim 13, characterized in that: Each of the switches comprises a sub-controller and a plurality of sub-interfaces, the sub-controller in each of the switches is connected to the plurality of sub-interfaces, the plurality of sub-interfaces are used to connect to the target node controller on the corresponding storage node, and the plurality of sub-controllers are connected to each other; Each of the sub-controllers is used to: when synchronizing the first sub-data to the storage space corresponding to the other node controllers, send a first health signal to one or more other sub-controllers connected to the sub-controller, and receive one or more second health signals, wherein the one or more second health signals correspond to the one or more other sub-controllers one-to-one, and the health signal is used to indicate that the operation status of the switch where the corresponding sub-controller is located is normal, and the health signal includes the first health signal and the second health signal; when at least one of the one or more second health signals is not received by the target time of the target time length after the first health signal is sent, send alarm information to the multiple target node controllers connected thereto through the multiple sub-interfaces on the switch where the sub-controller is located, wherein the alarm information is used to indicate that the operation status of the faulty switch where the sub-controller corresponding to the second health signal that is not received is abnormal; Each of the target node controllers is used to stop using the faulty switch to synchronize data when receiving the alarm information; The reference node controller is further configured to distribute the first sub-data corresponding to the failed switch to switches other than the failed switch among the plurality of switches.

Images