Communication boards and storage systems
By introducing communication boards and target integrated circuits into the storage system, the problem of high cost of storage system networking schemes in the prior art is solved, and the effect of reducing hardware costs and improving communication efficiency and reliability is achieved.
Patent Information
- Application Number
- CN202411980983.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-30
- Publication Date
- 2025-05-23
- Estimated Expiration
- 2044-12-30
AI Technical Summary
The existing storage system networking scheme has problems in cost control, resulting in excessive system hardware costs, especially in large-scale storage systems.
A communication board is designed to introduce a controller connection slot and communication network port into the memory head of the storage system, avoid configuring an independent network card for each controller, use target integrated circuits to realize communication connection between controllers, and manage communication network cards and optical modules through I2C channels.
It reduces the system hardware cost, improves the communication efficiency between controllers, enhances the reliability and redundancy of the storage system, and simplifies system architecture and maintenance management.
Smart Images

Figure CN119402304B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of storage systems, and in particular to a communication board and a storage system. Background Art
[0002] With the rapid development of information technology, the demand for data storage has exploded, and storage systems are increasingly used in various fields. In the design and implementation of storage systems, networking solutions are a key consideration, which directly affects the performance, reliability and cost of the system.
[0003] At present, there are many storage system networking solutions on the market, including two controllers, four controllers and other different configuration forms. These networking solutions can realize system networking topology through one frame two controllers, one frame four controllers, two frames four controllers and other methods. For example, the one frame two controllers networking method is to insert two controllers into a chassis / frame, and each controller is equipped with an independent network card. As a network interface device, the cost of the network card accounts for a certain proportion of the entire storage system. Equipping each controller with an independent network card will undoubtedly increase the hardware cost of the system. Especially in large-scale storage systems, the network card requirements of a large number of controllers will lead to a significant increase in costs, which is a disadvantage for storage systems that pursue cost-effectiveness.
[0004] In summary, the existing storage system networking solutions have certain problems in cost control, and a new solution is needed to reduce system costs to meet the growing storage needs and market competition requirements. Summary of the invention
[0005] In view of this, the present invention provides a communication board and a storage system to solve the problem of how to reduce system cost.
[0006] In a first aspect, the present invention provides a communication board, which is applied to a storage system, and includes at least one controller connection slot and at least one communication network port; wherein:
[0007] Each controller in the storage head of the storage system is installed on the communication board through each controller connection slot, and is communicated with the storage cabinet of the storage system based on each communication network port in the communication board.
[0008] The communication board provided in the embodiment of the present application is applied to a storage system, and the communication board includes at least one controller connection slot and at least one communication network port; wherein: each controller in the storage head of the storage system is installed to the communication board through each controller connection slot, and is connected to the storage cabinet of the storage system through each communication network port in the communication board. Therefore, it is not necessary to configure an independent network card for each controller, thereby reducing costs, and does not affect the communication between each controller and the storage cabinet of the storage system.
[0009] In an optional implementation, the communication board also includes a target integrated circuit; wherein:
[0010] Each controller is communicatively connected to the target integrated circuit via each controller connection slot;
[0011] The target integrated circuit is connected to each communication network port, and a communication connection is performed based on the communication network port storage cabinet.
[0012] The communication board provided in the embodiment of the present application also includes a target integrated circuit; wherein: each controller is connected to the target integrated circuit through each controller connection slot; thus, the communication connection between each controller can be realized based on the target integrated circuit. The problem in the prior art that each controller has its own plug-in card and the plug-in cards cannot communicate with each other, which results in that when information synchronization between different controllers is required, it can only be completed by the two controllers themselves, resulting in low efficiency of the information synchronization method, which may affect the overall performance of the storage system. For example, during data storage, if it is necessary to synchronize the relevant information of the storage task between the two controllers, such as the storage location, the size of the data block, etc., since communication can only be performed through the controllers themselves, it may cause information transmission delay, thereby affecting the storage efficiency. Therefore, the above-mentioned communication board improves the efficiency of communication between controllers. The target integrated circuit is connected to each communication network port, and the communication connection is performed based on the communication network port storage cabinet, so that communication between each controller and the storage cabinet can be realized.
[0013] In an optional embodiment, the target integrated circuit is used to detect whether the communication connection between the target integrated circuit and each communication network port is normal; if there is a failure in the communication connection between the target integrated circuit and the communication network port, the target integrated circuit communicates with the storage cabinet based on other normal communication network ports.
[0014] The communication board and target integrated circuit provided in the embodiment of the present application are used to detect whether the communication connection between the target integrated circuit and each communication network port is normal; if there is a failure in the communication connection between the target integrated circuit and the communication network port, the target integrated circuit communicates with the storage cabinet based on other normal communication network ports. Therefore, there is no need to switch the network card and the controller based on the above-mentioned communication network card. This avoids problems such as data loss and long business interruption time during the switching process, which will have a serious impact on the user experience and business operations. In addition, the above-mentioned communication board increases the redundancy of the storage system and reduces the risk of business interruption due to controller failure, thereby improving the reliability of the entire storage system.
[0015] In an optional embodiment, each communication network port is an Ethernet port, and each controller is communicatively connected to the target integrated circuit via a high-speed serial computer expansion bus in each controller connection slot;
[0016] The target integrated circuit converts the high-speed serial computer expansion bus to Ethernet to achieve communication connection between each controller and storage enclosure.
[0017] The communication board provided in the embodiment of the present application has each communication network port as an Ethernet port, and each controller is connected to the target integrated circuit through a high-speed serial computer expansion bus in each controller connection slot. Compared with some traditional communication methods, fast and stable information transmission between controllers is achieved. This helps the controller to synchronize fault information more quickly when the communication network port fails, so as to switch services in time, reduce service interruption time, and improve the communication efficiency of the system. The target integrated circuit converts the high-speed serial computer expansion bus into Ethernet to achieve communication connection between each controller and the storage cabinet. The complexity of the direct connection between the controller and the storage cabinet is reduced. At the same time, it is also convenient for the maintenance and management of the storage system, because only the target integrated circuit and the related connection bus need to be maintained and managed, rather than multiple independent network cards and complex connection lines, thereby simplifying the architecture of the storage system.
[0018] In an optional implementation, the communication board also includes at least one communication network card, the target integrated circuit is connected to each communication network card, and each communication network card is respectively connected to a corresponding communication network port.
[0019] The communication board provided by the embodiment of the present application further includes at least one communication network card. The target integrated circuit is connected to each communication network card, and each communication network card is respectively connected to a corresponding communication network port. Thus, it is possible to flexibly configure network connections according to the actual requirements of the storage system. For example, in different application scenarios, different communication network cards can be selected to connect to different network devices or network segments to meet diverse network access requirements. If it is necessary to connect to Ethernet networks with different speeds (such as 10 Mbps, 100 Mbps, 1 Gbps, etc.), it can be achieved by selecting a suitable communication network card, improving the flexibility of network connections. In addition, the above communication board increases the redundancy of network communication. When one of the communication network cards fails, the target integrated circuit can continue to communicate with the storage disk cabinet through other normal communication network cards to ensure the continuity of data transmission. For example, in a long-running storage system, if a certain communication network card fails to work properly due to hardware aging or accidental damage, the system can still maintain network communication through other communication network cards, reducing the risk of network interruption caused by network card failures and enhancing the reliability of network communication.
[0020] In an alternative embodiment, the target integrated circuit communicates with each communication network card based on the I2C channel and manages the optical modules in each communication network card based on the I2C channel.
[0021] For the communication board provided by the embodiment of the present application, the target integrated circuit communicates with each communication network card based on the I2C channel and manages the optical modules in each communication network card based on the I2C channel. Thereby, the complexity and cost of wiring are reduced. By managing the optical modules in the communication network cards through the I2C channel, precise control of the optical modules can be achieved.
[0022] In an alternative embodiment, the communication board further includes a clock buffer. One end of the clock buffer is connected to the target integrated circuit; the other end is connected to each controller connection slot; where:
[0023] The clock buffer is used to process the clock signals of each controller to ensure that the clock signals corresponding to each controller are of the same source.
[0024] For the communication board provided by the embodiment of the present application, the communication board further includes a clock buffer. One end of the clock buffer is connected to the target integrated circuit; the other end is connected to each controller connection slot; where the clock buffer is used to process the clock signals of each controller to ensure that the clock signals corresponding to each controller are of the same source. Thereby, the stability of the storage system corresponding to the communication board can be improved. In addition, it helps to ensure the accuracy of data transmission between controllers and between the controller and the storage disk cabinet, and makes the cooperation between controllers more efficient.
[0025] In an optional implementation, the clock buffer is further used to divide the clock signal corresponding to each controller into at least two sub-clock signals; and transmit each sub-clock signal to the target integrated circuit;
[0026] The target integrated circuit is further used to determine a target clock signal from the sub-clock signal and transmit the target clock signal to each controller;
[0027] Each controller is used to communicate with a target integrated circuit based on a target clock signal.
[0028] The communication board and clock buffer provided in the embodiment of the present application are also used to divide the clock signal corresponding to each controller into at least two sub-clock signals; and transmit each sub-clock signal to the target integrated circuit. This allows the communication board to flexibly select a suitable clock signal as the target clock signal according to actual needs. For example, in different working modes or application scenarios, the communication board can select different sub-clock signals to optimize the communication between the controller and the target integrated circuit, thereby improving the flexibility and adaptability of the communication board. The target integrated circuit is also used to determine the target clock signal from the sub-clock signal and transmit the target clock signal to each controller, thereby ensuring the accuracy of the determined target clock signal and being able to better adapt to the communication requirements between the controller and the target integrated circuit. Each controller is used to communicate with the target integrated circuit based on the target clock signal, thereby optimizing the communication between each controller and the target integrated circuit.
[0029] In an optional implementation, each controller connection slot is pluggably connected to each controller, wherein the controller connection slot includes a first hot-swap circuit; and each controller is connected to each first hot-swap circuit.
[0030] In the communication card provided in the embodiment of the present application, each controller connection slot is pluggable and connected to each controller, wherein the controller connection slot includes a first hot-swap circuit; each controller is connected to each first hot-swap circuit. This makes the replacement and maintenance of the controller extremely convenient. In addition, the first hot-swap circuit can effectively manage the switching of power supply and signal, and prevent damage to the system due to sudden power failure or signal interruption.
[0031] In an optional implementation, the ground wire in the first hot-swap circuit is connected to the controller based on the first pin;
[0032] The clock line and the power line in the first hot-swap circuit are connected to the controller based on the second pin;
[0033] The presence signal detection line in the first hot plug circuit is connected to the controller based on the third pin; wherein the length of the first pin is greater than the length of the second pin, and the length of the second pin is greater than the length of the third pin.
[0034] In the communication card provided by the embodiment of the present application, the ground wire in the first hot-swap circuit is connected to the controller based on the first pin. In this way, a stable grounding path can be established before other lines such as the clock line and the power line are connected, effectively preventing the controller from being damaged by electrical shocks that may be caused by static electricity accumulation or power supply fluctuations, and ensuring the safe power-on of the controller. For example, when the controller is inserted in an electrostatic environment, the ground wire connected first can safely guide the electrostatic charge that may be generated to avoid its impact on the precision circuit inside the controller, thereby ensuring the safety and stability of the controller. The clock line and the power line in the first hot-swap circuit are connected to the controller based on the second pin. It is ensured that the controller has established a good grounding foundation before obtaining a stable power supply, and the clock signal is connected at the right time. The appropriate clock signal access timing helps the internal circuit of the controller to start and initialize at the correct timing, avoiding abnormal circuit operation caused by the clock signal being connected too early or too late. For example, if the clock signal is connected to the unprepared controller circuit too early, it may cause data transmission errors or internal logic confusion, and this sequential connection method can effectively avoid such problems and improve the reliability of system startup. The in-place signal detection line in the first hot-swap circuit is connected to the controller based on the third pin. This allows the controller to detect the in-place signal only after completing key connections such as power and clock and working stably. This ensures the accuracy of the in-place signal detection and avoids misjudgment caused by in-place detection before the controller is fully ready. For example, if the in-place signal detection line is connected too early, the controller may be judged to be in place before it is fully working properly, and the system may misjudge it later due to reasons such as controller initialization failure. This design can effectively prevent the occurrence of such misjudgments and improve the accuracy of the system's detection of the controller status. Thereby improving the reliability of the storage system as a whole. By ensuring a safe and orderly power-on process, optimized signal timing, and accurate in-place status detection, the risk of system failure caused by electrical connection problems during the plug-in and unplugging of the controller is reduced. Whether in daily system maintenance and controller replacement operations or in application scenarios where the system is frequently started and stopped, it can effectively ensure the stable operation of the system, reduce the probability of system shutdown due to hardware connection problems, and improve the availability and reliability of the system.
[0035] In a second aspect, the present invention provides a storage system, the storage system comprising a storage head and a storage disk cabinet and a communication card of any one of the first aspect or any corresponding embodiments thereof, wherein:
[0036] Each controller in the storage head is installed to the communication board through each controller connection slot in the communication board; and is communicated and connected with the storage cabinet of the storage system based on each communication network port in the communication board.
[0037] In the storage system provided by the embodiment of the present application, each controller in the storage head is installed to the communication board through each controller connection slot in the communication board; and is connected to the storage cabinet of the storage system based on each communication network port in the communication board. Therefore, it is not necessary to configure an independent network card for each controller, thereby reducing costs and not affecting the communication between each controller and the storage cabinet of the storage system.
[0038] In an optional implementation, the communication board also includes a target integrated circuit; wherein:
[0039] Each controller is connected to the target integrated circuit through each controller connection slot; the target integrated circuit is connected to each communication network port, and is connected to the storage cabinet based on the communication network port.
[0040] In the storage system provided by the embodiment of the present application, each controller is connected to the target integrated circuit through each controller connection slot; the target integrated circuit is connected to each communication network port, and is connected to the storage cabinet based on the communication network port. The problem in the prior art that each controller has its own plug-in card and the external cards cannot communicate with each other, which results in that when information synchronization between different controllers is required, it can only be completed by the two controllers themselves, resulting in low efficiency of the information synchronization method, which may affect the overall performance of the storage system. For example, during data storage, if it is necessary to synchronize the relevant information of the storage task between the two controllers, such as storage location, data block size, etc., since communication can only be performed through the controllers themselves, information transmission delay may occur, thereby affecting storage efficiency. Therefore, the above-mentioned communication board improves the efficiency of communication between controllers. The target integrated circuit is connected to each communication network port, and is connected to the storage cabinet based on the communication network port, so that communication between each controller and the storage cabinet can be achieved.
[0041] In an optional implementation, if there are multiple controllers, there are a main controller and a standby controller between the controllers, wherein:
[0042] The main controller is used to detect whether the communication connection between the main controller and each standby controller and the target integrated circuit is normal; if there is an abnormality in the communication connection between the standby controller and the target integrated circuit, the main controller or other normal standby controllers are controlled to complete the task of the abnormal standby controller.
[0043] In the storage system provided by the embodiment of the present application, if the number of controllers is multiple, there are main controllers and backup controllers between the controllers. The main controller is used to detect whether the communication connection between the main controller and each backup controller and the target integrated circuit is normal; if there is an abnormality in the communication connection between the backup controller and the target integrated circuit, the main controller or other normal backup controllers are controlled to complete the tasks of the abnormal backup controller. Thereby, the continuity of the business is greatly guaranteed. In addition, the risk of system paralysis caused by local failures is reduced, and the tolerance and resistance of the system to failures are enhanced. In addition, the reasonable allocation of abnormal backup controller tasks by the main controller helps to optimize system resource utilization. When a backup controller has abnormal communication, its tasks are transferred to other normal controllers, avoiding the waste of resources caused by the idleness of the faulty controller.
[0044] In an optional implementation, the main controller is further configured to determine the main controller from among the standby controllers if an abnormality occurs in the communication connection between the main controller and the target integrated circuit.
[0045] The storage system provided by the embodiment of the present application, the main controller, is also used to determine the main controller from the backup controllers if the communication connection between the main controller and the target integrated circuit is abnormal. This ensures that the entire storage system is always under effective control, enhances the robustness of the storage system, avoids the tedious process of manual intervention for switching, and effectively reduces the cost of manual intervention.
[0046] In an optional implementation, the main controller is further used to send a communication board power-off notification to each standby controller when it is necessary to control the communication board to power off;
[0047] Each standby controller is used to process the target task related to the communication board after receiving the power-off notification of the communication board; and send a target task processing completion notification to the main controller after the target task is processed;
[0048] The main controller is used to control the communication board to power off and generate log information after receiving the notification that the target task has been processed.
[0049] The storage system provided by the embodiment of the present application, the main controller, is also used to send a communication board power-off notification to each standby controller when it is necessary to control the communication board to power off; so that each standby controller can know and process the target tasks related to the communication board in advance. This process can effectively avoid the forced interruption of ongoing tasks such as data transmission and configuration changes related to the communication board when the communication board is suddenly powered off, thereby ensuring the integrity of the data. Each standby controller is used to process the target task related to the communication board after receiving the communication board power-off notification; and after the target task is processed, it sends a target task processing completion notification to the main controller; the main controller is used to control the communication board to power off after receiving the target task processing completion notification, ensuring that the communication board can be safely powered off when all related tasks are properly processed. At the same time, it helps to reasonably release system resources associated with the communication board to avoid problems such as resource occupation or resource conflicts. Then, the main controller generates log information, which provides detailed record basis for subsequent fault tracing and system maintenance.
[0050] In an optional implementation, each controller is used to generate a competition value when the storage system is started, and broadcast the competition value to other controllers through a shared communication channel;
[0051] Each controller is used to compare the competition value received from other controllers with the competition value generated by itself, and determine the controller with the largest competition value as the main controller.
[0052] In the storage system provided in the embodiment of the present application, each controller is used to generate a competition value when the storage system is started, and broadcast the competition value to other controllers through a shared communication channel; each controller is used to compare the competition value received from other controllers with the competition value generated by itself; and the controller with the largest competition value is determined as the main controller, which ensures the accuracy of the determined main controller and enhances the stability and fault tolerance of the storage system.
[0053] In an optional implementation, each controller is used to detect its own CPU usage, memory utilization, I / O throughput, and network delay data; and generate a competition value based on the CPU usage, memory utilization, I / O throughput, and network delay data.
[0054] In the storage system provided in the embodiment of the present application, each controller is used to detect its own CPU utilization, memory utilization, I / O throughput, and network delay data; a competition value is generated based on the CPU utilization, memory utilization, I / O throughput, and network delay data, thereby ensuring the accuracy of the generated competition value and ensuring that the generated competition value can characterize the corresponding performance of each controller.
[0055] In an optional embodiment, the storage system includes at least one communication board, each controller in the storage head is respectively connected to each controller connection slot in each communication board, and each communication board is connected to the storage cabinet in the storage system based on each communication network port; wherein, if there is a failure in the communication connection between the target controller in each controller and the target communication board, the target controller completes the task with the storage cabinet based on the communication connection with other communication boards except the target communication board.
[0056] The storage system provided by the embodiment of the present application includes at least one communication board in the storage system, each controller in the storage head is respectively connected to each controller connection slot in each communication board, and each communication board is connected to the storage cabinet in the storage system based on each communication network port; wherein, if there is a communication connection failure between the target controller in each controller and the target communication board, the target controller completes the task with the storage cabinet based on the communication connection with other communication boards except the target communication board. Therefore, the communication redundancy and reliability of the storage system are enhanced, and the continuity of the business is effectively guaranteed.
[0057] In an optional implementation, each controller includes a second hot-swap circuit; the second hot-swap circuit is hot-swap connected to the first hot-swap circuit in the controller connection slot.
[0058] In the storage system provided by the embodiment of the present application, each controller includes a second hot-swap circuit; the second hot-swap circuit is hot-swapped to the first hot-swap circuit in the controller connection slot. This greatly improves the convenience of controller hardware replacement. When a controller fails or needs to be upgraded, there is no need to shut down the entire storage system. The operation and maintenance personnel can directly perform hot-swap operations on the controller, easily unplug the failed or old controller, and then insert a new or repaired controller. For example, in an enterprise-level storage system, if a controller has performance degradation due to hardware aging, it can be quickly replaced without shutting down the system, which minimizes system downtime, reduces the impact on business operations, and significantly improves the maintainability of the system. In addition, the hot-swap connection method ensures that the system can smoothly transition during the process of plugging and unplugging the controller, and will not affect the normal operation of the system due to problems such as sudden power outages and signal interruptions. The first hot-swap circuit and the second hot-swap circuit work together to properly handle the switching of power, signals, etc. at the moment of plugging and unplugging, avoiding electrical shocks, data loss, and system crashes. For example, when the controller needs to be replaced during data transmission, the hot-swap connection can ensure the continuity of data transmission, so that ongoing read and write operations in the storage system are not disturbed and continue to run stably, ensuring the stability of system operation and business continuity.
[0059] In an optional implementation, the second hot-swap circuit communicates with the first hot-swap circuit based on a high-speed serial computer expansion bus.
[0060] In the storage system provided by the embodiment of the present application, the second hot-swap circuit communicates with the first hot-swap circuit based on a high-speed serial computer expansion bus, thereby ensuring accurate, coordinated, efficient and stable communication between the second hot-swap circuit and the first hot-swap circuit. BRIEF DESCRIPTION OF THE DRAWINGS
[0061] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0062] Figure 1 is a schematic diagram of the structure of an existing networking block diagram according to an embodiment of the present invention;
[0063] Figure 2 is a schematic structural diagram of a first communication board according to an embodiment of the present invention;
[0064] Figure 3 is a schematic diagram of the structure of a second communication board according to an embodiment of the present invention;
[0065] Figure 4 is a schematic structural diagram of a third communication board according to an embodiment of the present invention;
[0066] Figure 5 is a schematic structural diagram of a fourth communication board according to an embodiment of the present invention;
[0067] Figure 6 is a schematic structural diagram of a fifth communication board according to an embodiment of the present invention;
[0068] Figure 7 It is a schematic diagram of hardware results of the homology design of the 100M clock according to an embodiment of the present invention;
[0069] Figure 8 is a schematic diagram of the structure of a first storage system according to an embodiment of the present invention;
[0070] Fig. 9 It is a structural schematic diagram of a main controller according to an embodiment of the present invention notifying a CPLD of a communication board through a BMC to perform power-on and power-off operations;
[0071] Fig.10 is a schematic diagram of the structure of a second storage system according to an embodiment of the present invention;
[0072] Fig.11 It is a schematic diagram of the result of the second hot-swap circuit and the first hot-swap circuit communicating based on a high-speed serial computer expansion bus according to an embodiment of the present invention. DETAILED DESCRIPTION
[0073] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.
[0074] With the rapid development of information technology, the demand for data storage has exploded, and storage systems are increasingly used in various fields. In the design and implementation of storage systems, networking solutions are a key consideration, which directly affects the performance, reliability and cost of the system.
[0075] At present, there are many storage system networking solutions on the market, including two controllers, four controllers and other different configuration forms. These networking solutions can realize system networking topology through one frame two controllers, one frame four controllers, two frames four controllers and other methods. For example, the one frame two controllers networking method is to insert two controllers into a chassis / frame, and each controller is equipped with an independent network card. As a network interface device, the cost of the network card accounts for a certain proportion of the entire storage system. Equipping each controller with an independent network card will undoubtedly increase the hardware cost of the system. Especially in large-scale storage systems, the network card requirements of a large number of controllers will lead to a significant increase in costs, which is a disadvantage for storage systems that pursue cost-effectiveness.
[0076] Specifically, Figure 1 As shown in the figure, the traditional one-frame four-controller networking solution inserts four controllers into a chassis, and each of these four controllers has its own plug-in card. The following uses a 100G network card to connect the EBOF expansion cabinet through a 100G optical fiber cable as an example. This example uses a 2-port 100G Ethernet, which supports the following protocols ROCE, iWARP / TCP and other technical protocols. The network card converts the bandwidth of PCIe5.0 X8 into a 2-port 100G network card. EBOF converts PCIe technology into Ethernet. The functions implemented by Ethernet are the same as described above. It supports the ROCE protocol, the port rate can reach 100G, and the PCIe interface supports Root complex. NVMe SSD disks can be connected and expanded through this PCIe, and the maximum number of disks supported is 25.
[0077] According to the above networking scheme, one frame and four controllers correspond to four controllers (hereinafter referred to as the head end), namely controller A, controller B, controller C, and controller D. These four controllers need to be plugged with corresponding Ethernet cards to connect to the EBOF expansion cabinet below. The network card and the corresponding controller are connected via PCIe, and the PCIe signal is converted into a network signal through the network card.
[0078] EBOF uses a 100G network to realize PCIE to network connection. The 100G network is connected to the corresponding 100G network card on the controller of one frame and four controllers through optical fiber. The PCIe end can expand multiple NVMe SSD disks.
[0079] In a four-controller-in-one-frame system, one controller is usually the master node. All customer services are on the master node, and the other nodes are standby nodes. When the network card of the master node fails, the service needs to be switched to other nodes, that is, both the network card and the controller need to be switched. For example, if the service is on controller A, when the network card on controller A fails, the service needs to be switched to controller B and the network card corresponding to controller B. For the head end, the network cards to which they belong must be managed, and there should be a corresponding controller to manage them. If a network card fails, such as the network card of controller A, controller A needs to synchronize the fault information to other controllers and switch the service on controller A to other controllers, such as controller B.
[0080] In summary, the existing storage system networking solutions have certain problems in cost control, and a new solution is needed to reduce system costs to meet the growing storage needs and market competition requirements.
[0081] The embodiment of the present application improves a communication board, such as Figure 2 As shown, the communication board is applied to the storage system, and the communication board includes at least one controller connection slot and at least one communication network port; wherein:
[0082] Each controller in the storage head of the storage system is installed on the communication board through each controller connection slot, and is communicated with the storage cabinet of the storage system based on each communication network port in the communication board.
[0083] Specifically, at least one controller connection slot is provided on the communication board. Each controller connection slot generally adopts a standardized interface form to adapt to each controller in the storage head. This standardized design greatly facilitates the connection between the controller and the communication board, so that the controller can be easily inserted or removed from the communication board during the assembly and maintenance of the storage system.
[0084] At least one communication network port on the communication board. These communication network ports can use high-speed Ethernet interface standards, such as the common Gigabit Ethernet (GbE) or 10 Gigabit Ethernet (10GbE) interfaces, to meet the needs of fast transmission of large amounts of data in the storage system. The controller in the storage head establishes a high-speed data transmission channel with the storage enclosure through these communication network ports. During the data storage process, whether the storage enclosure transmits storage data to the controller for processing, or the controller issues data write or read instructions to the storage enclosure, it all depends on the efficient operation of these communication network ports.
[0085] For example, in the storage system of a large data center, massive amounts of user data need to be frequently transmitted between storage cabinets and controllers. High-speed communication network ports can ensure that data is transmitted with extremely low latency and extremely high bandwidth, greatly improving the efficiency of data storage and access, and meeting the performance requirements of multiple users concurrently accessing the storage system.
[0086] The communication board provided in the embodiment of the present application is applied to a storage system, and the communication board includes at least one controller connection slot and at least one communication network port; wherein: each controller in the storage head of the storage system is installed to the communication board through each controller connection slot, and is connected to the storage cabinet of the storage system through each communication network port in the communication board. Therefore, it is not necessary to configure an independent network card for each controller, thereby reducing costs, and does not affect the communication between each controller and the storage cabinet of the storage system.
[0087] In an optional embodiment of the present application, as Figure 3 As shown, the communication board also includes a target integrated circuit; wherein:
[0088] Each controller is communicatively connected to the target integrated circuit via each controller connection slot;
[0089] The target integrated circuit is connected to each communication network port, and a communication connection is performed based on the communication network port storage cabinet.
[0090] Specifically, each controller establishes a communication connection with the target integrated circuit by means of the controller connection slot. The controller connection slot not only provides a physical connection interface, but also ensures the stability and reliability of signal transmission. In this connection process, the transmission of various signal types may be involved, such as control signals, data signals, and status signals. For example, the controller will transmit its own working status information, operation instructions for the storage cabinet, etc. to the target integrated circuit in the form of specific signals. The target integrated circuit is responsible for receiving and parsing these signals for subsequent processing and forwarding.
[0091] The target integrated circuit is connected to each communication network port, which makes it a key node in the communication link between the storage head and the storage cabinet. The data transmitted from the storage head controller will undergo a series of conversions and processing at the target integrated circuit to adapt to the transmission requirements of the communication network port, and then be accurately transmitted to the storage cabinet. For example, the data sent by the controller may be in a format based on an internal specific bus protocol, and the target integrated circuit will convert it into a data packet format suitable for transmission on the Ethernet communication network port, including adding appropriate frame headers, frame tails, and data encoding. Similarly, the data returned from the storage cabinet will first reach the target integrated circuit, which will perform reverse operations such as unpacking and format conversion before transmitting it to the corresponding controller.
[0092] During the data transmission process, the target integrated circuit also assumes the functions of flow control and data caching. Since there may be differences in the read and write speeds of data in the storage system, as well as the transmission rate matching problem between different devices, the target integrated circuit can reasonably adjust the data transmission rhythm according to the actual working conditions of the communication network port and the controller. For example, when the data transmission rate of the storage cabinet is high, and the controller processes data relatively slowly, the target integrated circuit will temporarily cache part of the data to avoid data loss or congestion during transmission. At the same time, it will also send flow control signals to the storage cabinet according to the usage of the cache, and adjust its data transmission rate to maintain the stable operation of the entire communication link.
[0093] The communication board provided in the embodiment of the present application also includes a target integrated circuit; wherein: each controller is connected to the target integrated circuit through each controller connection slot; thus, the communication connection between each controller can be realized based on the target integrated circuit. The problem in the prior art that each controller has its own plug-in card and the plug-in cards cannot communicate with each other, which results in that when information synchronization between different controllers is required, it can only be completed by the two controllers themselves, resulting in low efficiency of the information synchronization method, which may affect the overall performance of the storage system. For example, during data storage, if it is necessary to synchronize the relevant information of the storage task between the two controllers, such as the storage location, the size of the data block, etc., since communication can only be performed through the controllers themselves, it may cause information transmission delay, thereby affecting the storage efficiency. Therefore, the above-mentioned communication board improves the efficiency of communication between controllers. The target integrated circuit is connected to each communication network port, and the communication connection is performed based on the communication network port storage cabinet, so that communication between each controller and the storage cabinet can be realized.
[0094] In an optional implementation of the present application, the target integrated circuit is used to detect whether the communication connection between the target integrated circuit and each communication network port is normal; if there is a failure in the communication connection between the target integrated circuit and the communication network port, the target integrated circuit communicates with the storage cabinet based on other normal communication network ports.
[0095] Specifically, during the operation of the storage system, the detection of the communication connection between the target integrated circuit and the communication network port and the fault response mechanism are key links to ensure the continuous and stable operation of the system.
[0096] The target integrated circuit has a special detection module or functional unit that can monitor the connection status with each communication network port in real time. It will periodically send detection signals or monitor multiple indicators such as data flow and signal strength to determine whether the communication connection is normal. For example, by sending a specific heartbeat packet signal to each communication network port and waiting for its response, if no response is received from a certain communication network port within a predetermined time, or if the error rate during data transmission is detected to be beyond the normal range, such as a large number of bit errors, packet loss, etc., it can be determined that the communication connection between the target integrated circuit and the corresponding communication network port is faulty.
[0097] When a communication connection failure with a communication network port is detected, the intelligent switching function of the target integrated circuit will be activated. It will quickly switch the data transmission path to other normal communication network ports to ensure that the communication with the storage cabinet is not affected. This process involves complex routing and data redirection operations. The target integrated circuit maintains a status information table of the communication network port, which clearly records the working status of each communication network port, including normal, faulty, busy, etc. When switching, it will select the most appropriate normal communication network port based on this table and the current data transmission requirements. For example, if a large amount of data is currently being transmitted, the target integrated circuit will give priority to a normal communication network port with low bandwidth utilization and stable performance to avoid data transmission congestion or performance degradation caused by switching.
[0098] After switching the communication network port, the target integrated circuit will also perform a series of adjustments and optimizations. It will reconfigure the parameters of data transmission to adapt to the characteristics of the new communication network port. Different communication network ports may have different transmission rates, delay characteristics, etc. The target integrated circuit will adjust the data transmission frequency, cache strategy, etc. according to these differences. For example, if the transmission rate of the newly switched communication network port is low, the target integrated circuit may appropriately increase the data cache area to reduce the risk of data loss caused by transmission rate mismatch, while reducing the data transmission frequency to ensure that the data can be stably transmitted to the storage cabinet.
[0099] In addition, the target integrated circuit will also generate detailed log information during the entire fault handling process. These log records include the time when the fault occurred, the communication network port involved, the type of fault (such as connection interruption, signal abnormality, etc.), and the basis for selecting the communication network port after switching. These log information is extremely valuable to the operation and maintenance personnel of the storage system. They can quickly locate the cause of the fault by analyzing the log, understand the operating status of the system under fault conditions, and take targeted measures to repair or optimize it. For example, if a communication network port is found to frequently fail, the operation and maintenance personnel can further check the hardware connection, network equipment, etc. corresponding to the communication network port to eliminate potential hidden dangers. At the same time, they can also evaluate whether the network architecture of the storage system needs to be adjusted or upgraded based on the performance data of the communication network port after switching in the log.
[0100] For example, Figure 4 As shown, it is a communication board provided by an embodiment of the present application. The communication board mainly realizes PCIe to 100G Ethernet. The PCIe port is different from the network card mentioned above, which can only be connected to a single controller. The communication board of the present application occupies four controller connection slots, and four controllers can be connected through PCIe, and the bandwidth corresponding to each controller is PCIe4.0 X4. The communication board of the present application provides 4 100G network ports to the outside, and the network port can be connected to the EBoF expansion cabinet through optical fiber to expand the NVMe SSD disk externally. The communication board uses an ASIC chip, i.e., a target integrated circuit, to realize the PCIe to 100G network port. The 4 100G network ports can be used to realize independent networking and EBoF, or they can be used as backups between every two network ports, such as P1 and P3, P2 and P4 are each used as backups for each other, that is, if the P1 port fails, the service can be switched to the P3 port, and if the P2 port service fails, the service can be switched to the P4 port.
[0101] The communication board and target integrated circuit provided in the embodiment of the present application are used to detect whether the communication connection between the target integrated circuit and each communication network port is normal; if there is a failure in the communication connection between the target integrated circuit and the communication network port, the target integrated circuit is connected to the storage cabinet based on other normal communication network ports. Unlike the prior art, when the network card of the main node fails, the service needs to be switched to other nodes, that is, both the network card and the controller need to be switched. For example, if the service is on controller A, when the network card on controller A fails, the service needs to be switched to controller B and the network card corresponding to controller B. For the head end, the network cards to which they belong should be managed by corresponding controllers. If a network card fails, such as the network card of controller A, controller A needs to synchronize the fault information to other controllers and switch the service on controller A to other controllers, such as controller B. Therefore, based on the above-mentioned communication network card, there is no need to switch the network card and the controller. Thereby avoiding problems such as data loss and long service interruption time during the switching process, which will have a serious impact on the user experience and business operations. In addition, the above communication board increases the redundancy of the storage system, reduces the risk of business interruption due to controller failure, and thus improves the reliability of the entire storage system.
[0102] In an optional embodiment of the present application, each communication network port is an Ethernet port, and each controller is communicatively connected to the target integrated circuit via a high-speed serial computer expansion bus in each controller connection slot;
[0103] The target integrated circuit converts the high-speed serial computer expansion bus to Ethernet to achieve communication connection between each controller and storage enclosure.
[0104] Specifically, in the architecture of the storage system, there are many advantages to using Ethernet ports for each communication network port. Ethernet is a widely used and mature network technology with the characteristics of high speed, stability, and strong compatibility. Ethernet ports can support transmission at different rates, the most common of which are Gigabit Ethernet (GbE) and 10 Gigabit Ethernet (10GbE), which can meet the needs of fast transmission of large amounts of data in the storage system. Whether it is reading massive data from the storage cabinet to the controller for processing, or the controller writing the processed data back to the storage cabinet, the high-speed Ethernet port can efficiently complete the data transmission task, reduce the delay of data transmission, and improve the overall performance of the storage system.
[0105] Each controller is communicatively connected to the target integrated circuit through the High-Speed Serial Computer Extension Bus (I2C) in the controller connection slot. The I2C bus is a simple and effective serial communication bus that can achieve communication between multiple devices using only two wires (data line SDA and clock line SCL). In this architecture, the controller can transmit various control signals, status information, and a small amount of data to the target integrated circuit through the I2C bus. For example, the controller can inform the target integrated circuit of its own working status (such as whether it is busy, whether a fault has occurred, etc.) through the I2C bus, and at the same time, it can also receive configuration information or instructions from the target integrated circuit, such as parameter settings for certain functional modules. The advantage of this connection method lies in its simplicity and reliability, reducing the complexity of wiring, and the I2C bus has a certain anti-interference ability and can work stably in a relatively complex electromagnetic environment.
[0106] The target integrated circuit undertakes the key task of converting the High-Speed Serial Computer Extension Bus into Ethernet. The target integrated circuit parses and converts the information transmitted by the controller through the I2C bus, and repackages this information into an Ethernet packet format suitable for transmission on the Ethernet port. In this process, the target integrated circuit needs to perform a series of complex operations such as protocol conversion and data format adjustment. For example, the data transmitted on the I2C bus may be organized according to a specific byte order and format. The target integrated circuit needs to convert it into the payload part in the Ethernet packet and add appropriate Ethernet frame headers (including source MAC address, destination MAC address, frame type, etc.) and frame tails (such as CRC check code, etc.) to ensure that the data can be correctly transmitted, routed, and recognized by the receiving party in the Ethernet network. Similarly, when receiving data from the storage cabinet through the Ethernet port, the target integrated circuit will perform the opposite operation, unpack the Ethernet packet, extract the valid information, and convert it into a format suitable for transmission to the controller on the I2C bus.
[0107] This architectural design enables efficient and reliable communication connections between the various components of the storage system. It not only fully utilizes the advantages of Ethernet in large data volume transmission, but also takes advantage of the simple and stable communication characteristics of the I2C bus between the controller and the target integrated circuit. Through the conversion function of the target integrated circuit, seamless docking between different communication protocols and buses is achieved, allowing the storage system to flexibly adapt to different application scenarios and data processing requirements. For example, in an enterprise-level storage system, it can not only meet the high-speed data transmission requirements when a large number of users access stored data at the same time, but also ensure stable control and status information interaction between the controller and other components, improving the overall reliability, scalability and compatibility of the storage system, and providing a solid foundation for the long-term stable operation and functional upgrade of the storage system.
[0108] The communication board provided in the embodiment of the present application has each communication network port as an Ethernet port, and each controller is connected to the target integrated circuit through a high-speed serial computer expansion bus in each controller connection slot. Compared with some traditional communication methods, fast and stable information transmission between controllers is achieved. This helps the controller to synchronize fault information more quickly when the communication network port fails, so as to switch services in time, reduce service interruption time, and improve the communication efficiency of the system. The target integrated circuit converts the high-speed serial computer expansion bus into Ethernet to achieve communication connection between each controller and the storage cabinet. The complexity of the direct connection between the controller and the storage cabinet is reduced. At the same time, it is also convenient for the maintenance and management of the storage system, because only the target integrated circuit and the related connection bus need to be maintained and managed, rather than multiple independent network cards and complex connection lines, thereby simplifying the architecture of the storage system.
[0109] In an optional embodiment of the present application, as Figure 5 As shown, the communication board also includes at least one communication network card, the target integrated circuit is connected to each communication network card, and each communication network card is respectively connected to a corresponding communication network port.
[0110] Specifically, the communication network card will perform the final processing and packaging of the data transmitted from the target integrated circuit to make it conform to the standard format of the Ethernet communication protocol, and then send the data out through the corresponding communication network port. For example, when the storage system performs a data write operation, the controller transmits the data to the target integrated circuit, and the target integrated circuit integrates and converts the data and then passes it to the communication network card. The communication network card adds the necessary Ethernet frame header and frame tail information and performs error checking and coding operations to ensure that the data can be accurately transmitted on the Ethernet and finally reach the storage cabinet.
[0111] Among them, different communication network cards may have different performance indicators, such as transmission rate, supported network protocol versions, etc. Some optional storage systems may be equipped with communication network cards that support 10 Gigabit Ethernet or even higher transmission rates to meet the needs of large-scale data rapid transmission. At the same time, the communication network card may also support multiple network protocols. For example, in addition to the basic Ethernet protocol, it also supports storage network protocols such as iSCSI (Internet Small Computer System Interface), which enables the storage system to operate flexibly in different network environments and application scenarios. For example, when the storage system needs to interact with a server based on the iSCSI protocol for data, the communication network card can correctly process and transmit the iSCSI protocol data packets to achieve efficient storage data sharing and access.
[0112] The target integrated circuit can also dynamically adjust the working parameters of the communication network card, such as transmission power, data cache size, etc., according to the current workload and network status of the storage system. When network congestion is detected, the target integrated circuit can instruct the communication network card to reduce the transmission rate and increase the data cache to avoid data loss and relieve network pressure; when the network is idle, the transmission rate can be increased to make full use of the network bandwidth and improve data transmission efficiency.
[0113] Each communication network card is connected to the corresponding communication network port, ensuring that data can be accurately transmitted on a specific network channel. As an interface for connecting to an external network or storage device, the type and number of communication network ports will vary depending on the needs of the storage system. For example, in a storage system that requires high availability, multiple communication network ports may be configured and equipped with corresponding redundant communication network cards. When one of the communication network ports or communication network cards fails, other redundant components can quickly take over the work, ensuring uninterrupted data transmission and improving the reliability and stability of the storage system.
[0114] The communication board provided in the embodiment of the present application further includes at least one communication network card, the target integrated circuit is connected to each communication network card, and each communication network card is respectively connected to the corresponding communication network port. Thus, the network connection can be flexibly configured according to the actual needs of the storage system. For example, in different application scenarios, different communication network cards can be selected to connect different network devices or network segments to meet diverse network access needs. In addition, the above-mentioned communication board increases the redundancy of network communication. When one of the communication network cards fails, the target integrated circuit can continue to communicate with the storage cabinet through other normal communication network cards to ensure the continuity of data transmission. For example, in a storage system that has been running for a long time, if a communication network card cannot work normally due to hardware aging or accidental damage, the system can still maintain network communication through other communication network cards, reducing the risk of network interruption due to network card failure and enhancing the reliability of network communication.
[0115] In an optional embodiment, if Figure 5 As shown, the target integrated circuit communicates with each communication network card based on the I2C channel, and manages the optical modules in each communication network card based on the I2C channel.
[0116] Specifically, in the communication architecture of the storage system, the target integrated circuit can send various control instructions to the communication network card through the I2C channel, such as configuring the working mode of the network card (full-duplex or half-duplex), setting the transmission rate (adaptive or manually specified), enabling or disabling certain specific functions (such as flow control, VLAN division, etc.). At the same time, the communication network card can also feedback its own working status information to the target integrated circuit through the I2C channel, including the current connection status (whether it is connected to the network, the stability of the connection), data transmission statistics (the amount of data sent and received, the number of error packets, etc.) and hardware health status (such as whether the temperature and voltage are normal).
[0117] The target integrated circuit can monitor the working parameters of the optical module through the I2C channel, and the target integrated circuit can obtain the optical power output value of the optical module in real time. If the optical power is too low, it may cause bit errors or even data loss during long-distance transmission; if the optical power is too high, it may cause damage to the optical module itself and the optical fiber and other equipment connected to it. The target integrated circuit continuously monitors the optical power through the I2C channel. Once an abnormality is found, timely measures can be taken, such as adjusting the drive current of the optical module to stabilize the optical power.
[0118] In addition, the target integrated circuit can also monitor the optical wavelength information of the optical module. Different optical wavelengths have different transmission characteristics in optical fibers. In some complex storage network environments, it may be necessary to select the appropriate optical wavelength according to actual conditions to optimize the transmission effect. For example, in long-distance transmission, certain specific optical wavelengths can reduce signal attenuation; and in the scenario of multi-fiber parallel transmission, different optical wavelengths can avoid interference between signals. The target integrated circuit can configure and adjust the optical wavelength of the optical module through the I2C channel according to the overall architecture and network layout of the storage system.
[0119] The target integrated circuit can also monitor the operating temperature of the optical module. Excessive temperature will affect the performance and life of the optical module, and may even cause the optical module to fail. The target integrated circuit is connected to the temperature sensor inside the optical module through the I2C channel to obtain temperature data in real time. When the temperature exceeds the normal range, it can start the heat dissipation mechanism, such as controlling the speed of the cooling fan of the optical module to increase, or reducing the workload of the optical module to reduce the temperature and ensure that the optical module works stably in a suitable temperature environment.
[0120] In addition, the target integrated circuit can also perform firmware upgrade operations on the optical module. With the continuous development of storage technology and the upgrading of network standards, the firmware of the optical module also needs to be upgraded accordingly to support new functions and performance optimization. Through the I2C channel, the target integrated circuit can safely transmit the new firmware data to the optical module and complete the firmware update and upgrade process without affecting the normal operation of the storage system, which improves the adaptability and maintainability of the optical module and extends the service life of the optical module, thereby ensuring the long-term stability and efficient operation of the communication link of the entire storage system.
[0121] In an optional embodiment of the present application, as Figure 6 As shown, the ASIC implements some functions of PCIe to network (NIC) internally. For specific functions, see Figure 6 ASIC provides 4 network interfaces P1, P2, P3, and P4, and the maximum network rate supports 200G. The PCIe function mainly realizes the ability to split multiple PCIe bandwidths according to needs, such as splitting into x2 and x4 bandwidths, and realizing the multi-Host function, that is, to split the PCIe of a chip into multiple PCIe ports and connect multiple RC ends (controllers), which can realize the connection of multiple controllers. At the same time, the ASIC chip supports multiple 100M clocks, such as Figure 6 , supports up to 8 channels of 100M clocks.
[0122] In order to realize the management of NIC network by ASIC, the hardware design supports ASIC to manage the optical module of NIC through I2C channel, which can realize the online management function of the optical module's receiving and transmitting optical power, temperature, power supply voltage and current and other information. The hardware topology is shown above. Figure 6 shown.
[0123] The communication board provided in the embodiment of the present application, the target integrated circuit communicates with each communication network card based on the I2C channel, and manages the optical modules in each communication network card based on the I2C channel. Thereby reducing the complexity and cost of wiring. By managing the optical module in the communication network card through the I2C channel, precise control of the optical module can be achieved. In addition, the target integrated circuit can obtain detailed operating parameters of the optical module, such as optical power, optical wavelength, operating temperature, etc., and monitor the changes of these parameters in real time. According to the changes in these parameters, the working state of the optical module can be adjusted in time. For example, when the optical power is lower than a certain threshold, the driving current of the optical module can be increased to increase the optical power; when the operating temperature is too high, heat dissipation measures can be taken or the operating frequency of the optical module can be reduced to ensure that the optical module is always in the best working state and improve the quality and stability of optical transmission.
[0124] In an optional embodiment, if Figure 5As shown, the communication board also includes a clock buffer, one end of which is connected to the target integrated circuit; the other end is connected to each controller connection slot; wherein:
[0125] The clock buffer is used to process the clock signals of each controller to ensure that the clock signals corresponding to each controller have the same source.
[0126] In an optional implementation manner of the present application, the clock buffer is further used to divide the clock signal corresponding to each controller into at least two sub-clock signals; and transmit each sub-clock signal to the target integrated circuit;
[0127] The target integrated circuit is further used to determine a target clock signal from the sub-clock signal and transmit the target clock signal to each controller;
[0128] Each controller is used to communicate with a target integrated circuit based on a target clock signal.
[0129] Exemplarily, each controller has its own clock signal, which is first transmitted to the communication board. On the communication board, the clock signal of each controller is processed by a dedicated clock buffer. The function of the clock buffer is to enhance the driving ability of the clock signal while maintaining the integrity and stability of the clock signal and reducing the distortion and interference of the signal during transmission. For example, the clock buffer can shape the clock signal to make its rising and falling edges steeper to meet the requirements of the ASIC chip for the clock signal quality.
[0130] After passing through the clock buffer, the clock signal of each controller is fanned out (divided into multiple identical signal outputs) into two sub-clock signals. In this way, each controller has two selectable clock signal branches, which will be connected to the target integrated circuit of the communication board. The target integrated circuit is provided with the clock signal source selection from multiple controllers to ensure that the basic conditions for achieving clock homology are met in hardware connection.
[0131] The target integrated circuit can implement the selection of the clock source through the internal configuration register or control logic. For example, there may be a clock selection control bit. When the bit is set to different values, the target integrated circuit will select the clock signal of different controllers. This selection can be dynamically determined based on the actual needs of the system, the status of the controller, or other factors. For example, at the beginning of system startup, the clock of a controller can be selected as the initial clock source according to the pre-set default order; during the operation of the system, if a controller fails or the quality of its clock signal decreases, the target integrated circuit can switch to the clock source of other normal controllers to ensure the continuous and stable operation of the system.
[0132] In an optional embodiment of the present application, as Figure 6 As shown, the ASIC implements some functions of PCIe to network (NIC) internally. For specific functions, see Figure 6 ASIC provides 4 network interfaces P1, P2, P3, and P4, and the maximum network rate supports 200G. The PCIe function mainly realizes the ability to split multiple PCIe bandwidths according to needs, such as splitting into x2 and x4 bandwidths, and realizing the multi-Host function, that is, to split the PCIe of a chip into multiple PCIe ports and connect multiple RC ends (controllers), which can realize the connection of multiple controllers. At the same time, the ASIC chip supports multiple 100M clocks, such as Figure 6 , supporting up to 8 channels of 100M clock. In order to realize the management of NIC network by ASIC, the hardware design supports ASIC to manage the optical module of NIC through I2C channel, which can realize the online management function of the optical module's receiving and transmitting optical power, temperature, power supply voltage and current, etc. The hardware topology is shown in Figure 6 shown.
[0133] Because multiple controllers are connected to ASIC chips, in order to achieve the same clock source design, the 100M clock of ASIC needs to come from 4 controllers respectively to achieve the same source design of 100M clock, otherwise there will be problems such as bit errors in the PCIe link. Figure 7 It is a homologous design of 100M clock. That is, after the 100M clock of each controller reaches the big card, it passes through the 100M clock buffer on the big card, fans out 2 100M clocks, and connects to ASIC. Figure 7 See below. Inside the ASIC, you can choose which controller the 100M clock comes from, ensuring the same source design of the 100M clock.
[0134] The communication board provided by the embodiment of the present application further includes a clock buffer in the communication board, one end of the clock buffer is connected to the target integrated circuit; the other end is connected to each controller connection slot; wherein the clock buffer is also used to divide the clock signal corresponding to each controller into at least two sub-clock signals; and transmit each sub-clock signal to the target integrated circuit. Thus, the communication board can flexibly select a suitable clock signal as the target clock signal according to actual needs. For example, in different working modes or application scenarios, the communication board can select different sub-clock signals to optimize the communication between the controller and the target integrated circuit, thereby improving the flexibility and adaptability of the communication board. The target integrated circuit is also used to determine the target clock signal from the sub-clock signal, and transmit the target clock signal to each controller, thereby ensuring the accuracy of the determined target clock signal and being able to better adapt to the communication requirements between the controller and the target integrated circuit. For example, in the process of data transmission, a suitable target clock signal can ensure that the data is transmitted and received at the correct time point, reducing the data transmission delay or error caused by the inappropriate clock signal, and improving the overall performance and communication efficiency of the system. Each controller is used to communicate with the target integrated circuit based on the target clock signal, thereby optimizing the communication between each controller and the target integrated circuit. This can improve the stability of the storage system corresponding to the communication board. In addition, it helps to ensure the accuracy of data transmission between controllers and between controllers and storage enclosures, and makes the collaboration between controllers more efficient.
[0135] In an optional implementation, each controller connection slot is pluggably connected to each controller, wherein the controller connection slot includes a first hot-swap circuit; and each controller is connected to each first hot-swap circuit.
[0136] In an optional embodiment of the present application, the ground line in the first hot-swap circuit is connected to the controller based on the first pin. The clock line and the power line in the first hot-swap circuit are connected to the controller based on the second pin. The in-position signal detection line in the first hot-swap circuit is connected to the controller based on the third pin; wherein the length of the first pin is greater than the length of the second pin, and the length of the second pin is greater than the length of the third pin.
[0137] Specifically, in the architecture of the storage system, a pluggable connection design is provided between the controller connection slot and the controller.
[0138] Specifically, different lines in the first hot-swap circuit are connected to the controller through pins of different lengths, among which the first pin (connecting the ground wire) is the longest, in order to ensure that the ground wire is first contacted and connected when the controller is inserted into the connection slot. Its importance lies in that at the moment when the controller is electrically connected to the system, establishing a stable grounding path first can effectively prevent the controller from being damaged by electrical shocks such as static electricity accumulation or power supply fluctuations. In the actual operating environment, static electricity is everywhere, especially in the process of frequently plugging and unplugging the controller, static electricity can easily accumulate on the circuit components of the controller. If the ground wire is not connected first, these electrostatic charges may be released instantly when the power line is connected later, causing irreversible damage to the precision chips, capacitors and other components inside the controller. And by preferentially connecting the ground wire through the longest first pin, it is like building a safe electrostatic protection barrier for the controller, so that static electricity can be safely introduced into the earth, ensuring the safety and stability of the controller, and laying the foundation for subsequent normal electrical connections.
[0139] The second pin (connecting the clock line and the power line) has a moderate length. After the ground line is successfully connected through the first pin, the clock line and the power line are connected through the second pin. This order ensures that the controller has established a good grounding foundation before obtaining a stable power supply, and the clock signal is connected at the right time. The clock signal plays a key synchronization role in the normal operation of the internal circuit of the controller. It determines when each circuit module performs data sampling, transmission and processing operations. If the clock signal is connected to the unprepared controller circuit too early, it may cause data transmission errors or internal logic confusion. For example, in some digital circuit modules of the controller, data transmission is based on clock edge triggering. If the clock signal is connected before the power supply is stable or other circuit modules have not been initialized, it may cause data to be sampled or transmitted at the wrong time, causing the entire controller to work abnormally. According to this order of grounding first and then connecting the clock and power, such problems can be effectively avoided, so that the controller can be started and initialized in a stable and orderly electrical environment, and the reliability of system startup is improved.
[0140] The third pin (connecting the in-place signal detection line) is the shortest and is connected last. The in-place signal detection line is used to detect whether the controller is correctly inserted into the slot and working properly. It is designed to be the shortest pin so that the controller can detect the in-place signal only after completing key connections such as power and clock and working stably. This ensures the accuracy of the in-place signal detection and avoids misjudgment caused by in-place detection before the controller is fully ready. For example, if the in-place signal detection line is connected too early, the controller may be judged to be in place before it is fully working properly, and the system may misjudge it later due to reasons such as controller initialization failure. This misjudgment may cause a series of problems, such as the system mistakenly believing that the controller is working properly and assigning tasks to it, while in fact the controller may not perform the tasks correctly, resulting in data loss or system failure. By connecting the in-place signal detection line last, the in-place status is confirmed only when the controller is truly running stably, which improves the accuracy of the system's detection of the controller status and ensures the stable operation of the system.
[0141] The communication board provided in the embodiment of the present application has each controller connection slot connected to each controller in a pluggable manner, wherein the controller connection slot includes a first hot-swappable circuit; each controller is connected to each first hot-swappable circuit. This makes the replacement and maintenance of the controller extremely convenient. In addition, the first hot-swappable circuit can effectively manage the switching of power and signals to prevent damage to the system due to sudden power failure or signal interruption. The ground wire in the first hot-swappable circuit is connected to the controller based on the first pin. In this way, a stable grounding path can be established before other lines such as the clock line and the power line are connected, effectively preventing damage to the controller caused by electrical shocks that may be caused by static electricity accumulation or power supply fluctuations, ensuring that the controller is powered on safely, and ensuring the safety and stability of the controller. The clock line and power line in the first hot-swappable circuit are connected to the controller based on the second pin. It is ensured that the controller has established a good grounding foundation before obtaining a stable power supply, and the clock signal is connected at the appropriate time point. The appropriate clock signal access timing helps the internal circuit of the controller to start and initialize at the correct timing, avoids abnormal circuit operation caused by premature or late access to the clock signal, and improves the reliability of system startup. The in-place signal detection line in the first hot-swap circuit is connected to the controller based on the third pin. This allows the controller to detect the in-place signal only after completing key connections such as power and clock and working stably. This ensures the accuracy of the in-place signal detection and avoids misjudgment caused by in-place detection before the controller is fully ready. The above design improves the reliability of the storage system as a whole. By ensuring a safe and orderly power-on process, optimized signal timing, and accurate in-place status detection, the risk of system failure caused by electrical connection problems during the controller plugging and unplugging process is reduced. Whether in daily system maintenance and controller replacement operations, or in application scenarios where the system is frequently started and stopped, it can effectively ensure the stable operation of the system, reduce the probability of system shutdown due to hardware connection problems, and improve the availability and reliability of the system.
[0142] The present application embodiment provides a storage system, such as Figure 8 As shown, the storage system includes a storage head and a storage disk cabinet and a communication board card of any one of the above embodiments, wherein:
[0143] Each controller in the storage head is installed to the communication board through each controller connection slot in the communication board; and is communicated and connected with the storage cabinet of the storage system based on each communication network port in the communication board.
[0144] For a detailed introduction to this embodiment, please refer to the above introduction to the communication board, which will not be repeated here.
[0145] In the storage system provided by the embodiment of the present application, each controller in the storage head is installed to the communication board through each controller connection slot in the communication board; and is connected to the storage cabinet of the storage system based on each communication network port in the communication board. Therefore, it is not necessary to configure an independent network card for each controller, thereby reducing costs and not affecting the communication between each controller and the storage cabinet of the storage system.
[0146] In an optional embodiment, if Figure 8 The communication board shown also includes a target integrated circuit; wherein:
[0147] Each controller is connected to the target integrated circuit through each controller connection slot; the target integrated circuit is connected to each communication network port, and is connected to the storage cabinet based on the communication network port.
[0148] For a detailed introduction to this embodiment, please refer to the above introduction to the communication board, which will not be repeated here.
[0149] In the storage system provided by the embodiment of the present application, each controller is connected to the target integrated circuit through each controller connection slot; the target integrated circuit is connected to each communication network port, and is connected to the storage cabinet based on the communication network port. The problem in the prior art that each controller has its own plug-in card and the external cards cannot communicate with each other, which results in that when information synchronization between different controllers is required, it can only be completed by the two controllers themselves, resulting in low efficiency of the information synchronization method, which may affect the overall performance of the storage system. For example, during data storage, if it is necessary to synchronize the relevant information of the storage task between the two controllers, such as storage location, data block size, etc., since communication can only be performed through the controllers themselves, information transmission delay may occur, thereby affecting storage efficiency. Therefore, the above-mentioned communication board improves the efficiency of communication between controllers. The target integrated circuit is connected to each communication network port, and is connected to the storage cabinet based on the communication network port, so that communication between each controller and the storage cabinet can be achieved.
[0150] In an optional implementation, if there are multiple controllers, there are a main controller and a backup controller between the controllers, wherein:
[0151] The main controller is used to detect whether the communication connection between the main controller and each standby controller and the target integrated circuit is normal; if there is an abnormality in the communication connection between the standby controller and the target integrated circuit, the main controller or other normal standby controllers are controlled to complete the task of the abnormal standby controller.
[0152] Specifically, the main controller can periodically send specific detection data packets to the target integrated circuit and each standby controller. These data packets contain the source controller's identification, timestamp, and some data content for verification. After receiving the data packet, the target integrated circuit and the standby controller will process it according to predetermined rules and return a response data packet. The main controller determines whether the communication connection is normal based on factors such as the reception of the response data packet, content integrity, and time delay. If a response data packet from a standby controller is not received within the specified time, or the received data packet is erroneous or incomplete, the main controller will determine that the communication connection between the standby controller and the target integrated circuit is abnormal.
[0153] Once the main controller detects that the communication connection between a standby controller and the target integrated circuit is abnormal, it will quickly start the task allocation mechanism. At this time, the main controller will flexibly decide whether to complete the task of the abnormal standby controller itself or assign the task to other normal standby controllers based on multiple factors such as the current workload of the system, the performance status of each controller, and the priority of the task. For example, if the workload of the main controller itself is close to saturation, and the resource utilization rate of another standby controller is low and it has the ability to complete the task, the main controller will assign the task of the abnormal standby controller to the normal standby controller. During the task allocation process, the main controller will send detailed task instructions to the selected controller, including the type of task (such as data read and write operations, data cache management, etc.), the scope of the task (such as which storage areas are involved in the data), the goal of the task (such as writing data to a specific storage cabinet location or reading data from a specific location) and the time requirement of the task. After receiving the task instruction, the selected controller will immediately adjust its own working status, start executing the task, and continuously feedback the progress of the task to the main controller during the task execution.
[0154] In the storage system provided by the embodiment of the present application, if the number of controllers is multiple, there are main controllers and backup controllers between the controllers. The main controller is used to detect whether the communication connection between the main controller and each backup controller and the target integrated circuit is normal; if there is an abnormality in the communication connection between the backup controller and the target integrated circuit, the main controller or other normal backup controllers are controlled to complete the tasks of the abnormal backup controller. Thereby, the continuity of the business is greatly guaranteed. In addition, the risk of system paralysis caused by local failures is reduced, and the tolerance and resistance of the system to failures are enhanced. In addition, the reasonable allocation of abnormal backup controller tasks by the main controller helps to optimize system resource utilization. When a backup controller has abnormal communication, its tasks are transferred to other normal controllers, avoiding the waste of resources caused by the idleness of the faulty controller.
[0155] In an optional implementation, the main controller is further configured to determine the main controller from among the standby controllers if an abnormality occurs in the communication connection between the main controller and the target integrated circuit.
[0156] Specifically, in a multi-controller architecture of a storage system, when an abnormality occurs in the communication connection between the main controller and the target integrated circuit, each standby controller will continue to send its own status information to the main controller (during normal communication) or other normal standby controllers (when the main controller communication is abnormal), including but not limited to the hardware health status of the controller (such as CPU usage, memory utilization, temperature, etc.), the current task load, the communication connection status with other components (such as communication network cards, storage cabinets, etc.) and the performance data of historical task execution (such as data read and write speed, error rate, etc.). When the main controller detects that the communication connection between itself and the target integrated circuit is abnormal, it will collect and analyze the status information from the standby controllers to evaluate the suitability of each standby controller to serve as the new main controller, and then determine the main controller from the standby controllers according to the corresponding suitability of each standby controller. The standby controller selected as the new main controller will perform a series of initialization and role conversion operations. It will take over some key functions of the original main controller, such as global status monitoring of the entire storage system, task allocation and coordination of other controllers (including the original main controller, which is now transformed into the role of standby controller), and communication management with the target integrated circuit and storage cabinet. At the same time, the new main controller will also send identity change notifications to other components (such as storage enclosures, communication network cards, etc.) so that these components can promptly adjust the communication and interaction methods with the new main controller.
[0157] The storage system provided by the embodiment of the present application, the main controller, is also used to determine the main controller from the backup controllers if the communication connection between the main controller and the target integrated circuit is abnormal. This ensures that the entire storage system is always under effective control, enhances the robustness of the storage system, avoids the tedious process of manual intervention for switching, and effectively reduces the cost of manual intervention.
[0158] In an optional implementation, the main controller is further used to send a communication board power-off notification to each standby controller when it is necessary to control the communication board to power off;
[0159] Each standby controller is used to process the target task related to the communication board after receiving the power-off notification of the communication board; and send a target task processing completion notification to the main controller after the target task is processed;
[0160] The main controller is used to control the communication board to power off and generate log information after receiving the notification that the target task has been processed.
[0161] Specifically, when the storage system needs to control the communication board to power off due to some reasons (such as system maintenance, hardware upgrade or emergency fault handling), the main controller starts the corresponding power-off process. The main controller first sends a communication board power-off notification to each standby controller. This notification is sent through a reliable communication mechanism within the system, such as a high-speed internal bus or a dedicated control network protocol. The notification contains rich information, such as the reason for the communication board to power off, the expected time range for the power-off, and the priority of the power-off operation, so that the standby controller can fully understand the situation and make an appropriate response.
[0162] After receiving the power-off notification of the communication board, each standby controller will immediately start processing the target tasks related to the communication board. These target tasks cover multiple aspects, such as ongoing data transmission tasks. If data is being transmitted from the storage head to the storage cabinet or vice versa through the communication board, the standby controller will ensure that these data transmissions are completed completely and accurately to avoid data loss or damage. This may involve the management of the data cache, ensuring that all data in the cache is written to the target storage location, or properly saving the data that has not been processed for subsequent recovery. At the same time, the standby controller will also process some configuration information or status information update tasks related to the communication board, such as recording the working status information of the current communication board in the local storage, so that it can be referenced during subsequent system recovery or troubleshooting.
[0163] After completing the target task related to the communication board, the standby controller will send a target task processing completion notification to the main controller. This notification also follows a specific communication protocol and format, and contains information such as the standby controller's identification, the details of the completed task, and whether any abnormal conditions were encountered during the processing. After receiving the target task processing completion notification, the main controller will summarize and analyze the notification information of all standby controllers to confirm that all key tasks related to the communication board have been properly processed. Only in this case will the main controller control the communication board to power off. This control operation is implemented through the hardware control interface between the communication board. For example, a power-off command may be sent through a specific control signal line to gradually stop the communication board from working and cut off the electrical and data connections with the storage head and storage enclosure.
[0164] While controlling the power-off of the communication board, the main controller will also generate detailed log information. The log information records the key information of the entire communication board power-off process, including the start time of the power-off, the triggering reason, the time when each standby controller receives the notification, the time and details of the standby controller completing the target task, the final power-off time of the communication board, and whether any abnormal situation occurs during the power-off process (such as a standby controller failing to complete the task on time, communication failure causing the failure of notification sending, etc.). These log information is extremely valuable for the subsequent maintenance and troubleshooting of the storage system. For example, after the system is restarted, if some data is found to be lost or the storage system is in an abnormal working state, the operation and maintenance personnel can quickly locate the problem by checking the log information generated by the main controller, determine which link in the power-off process of the communication board has failed, and take targeted measures to repair it.
[0165] For multi-control scenarios, if you want to control the communication card, other controllers need to know or synchronize information among multiple controllers. A main controller informs the CPLD of the communication card through the BMC to perform power-on and power-off operations, such as Fig. 9 .
[0166] The storage system provided by the embodiment of the present application, the main controller, is also used to send a communication board power-off notification to each standby controller when it is necessary to control the communication board to power off; so that each standby controller can know and process the target tasks related to the communication board in advance. This process can effectively avoid the forced interruption of ongoing tasks such as data transmission and configuration changes related to the communication board when the communication board is suddenly powered off, thereby ensuring the integrity of the data. Each standby controller is used to process the target task related to the communication board after receiving the communication board power-off notification; and after the target task is processed, it sends a target task processing completion notification to the main controller; the main controller is used to control the communication board to power off after receiving the target task processing completion notification, ensuring that the communication board can be safely powered off when all related tasks are properly processed. At the same time, it helps to reasonably release system resources associated with the communication board to avoid problems such as resource occupation or resource conflicts. Then, the main controller generates log information, which provides detailed record basis for subsequent fault tracing and system maintenance.
[0167] In an optional implementation, each controller is used to generate a competition value when the storage system is started, and broadcast the competition value to other controllers through a shared communication channel;
[0168] Each controller is used to compare the competition value received from other controllers with the competition value generated by itself, and determine the controller with the largest competition value as the main controller.
[0169] In an optional implementation, each controller is used to detect its own CPU usage, memory utilization, I / O throughput, and network delay data; and generate a competition value based on the CPU usage, memory utilization, I / O throughput, and network delay data.
[0170] Specifically, each controller can detect its own CPU usage, memory utilization, I / O throughput, and network latency data. Then, each controller assigns different weights to each performance indicator, and then calculates a comprehensive value based on these weights as the competitive value. For example, for a storage system that requires extremely high data read and write speeds, a higher weight may be given to I / O throughput, while for a storage system that requires frequent data processing and calculations, the weight of CPU usage may be appropriately increased. Taking a simple weighted calculation example, if the CPU utilization weight is 0.3, the memory utilization weight is 0.2, the I / O throughput weight is 0.4, and the network delay weight is 0.1, a controller detects that its own CPU utilization is 60%, the memory utilization is 40%, the I / O throughput is 80MB / s, and the network delay is 5ms. Assuming that the CPU utilization and memory utilization are converted to values between 0 and 1 (such as 60% CPU utilization is converted to 0.6), and the network delay is normalized (such as 5ms is converted to 0.05, assuming that the network delay baseline value is 100ms), then the contention value of the controller = 0.6 * 0.3 + 0.4 * 0.2 + 0.8 * 0.4 + 0.05 * 0.1 = 0.585.
[0171] After generating a competition value, each controller broadcasts it to other controllers through a shared communication channel. Each controller compares the competition value received from other controllers with the competition value generated by itself, and determines the controller with the largest competition value as the main controller.
[0172] In the storage system provided by the embodiment of the present application, each controller is used to detect its own CPU usage, memory utilization, I / O throughput, and network delay data; based on the CPU usage, memory utilization, I / O throughput, and network delay data, a competition value is generated, which ensures the accuracy of the generated competition value and ensures that the generated competition value can characterize the corresponding performance of each controller. Then, each controller broadcasts the competition value to other controllers through a shared communication channel; each controller is used to compare the competition value received from other controllers with the competition value generated by itself; the controller with the largest competition value is determined as the main controller, which ensures the accuracy of the determined main controller and enhances the stability and fault tolerance of the storage system.
[0173] In an optional embodiment, the storage system includes at least one communication board, each controller in the storage head is respectively connected to each controller connection slot in each communication board, and each communication board is connected to the storage cabinet in the storage system based on each communication network port; wherein, if there is a failure in the communication connection between the target controller in each controller and the target communication board, the target controller completes the task with the storage cabinet based on the communication connection with other communication boards except the target communication board.
[0174] Specifically, the storage system includes at least one communication board. Each controller in the storage head is connected to each controller connection slot in each communication board respectively, and each communication board is connected to a storage cabinet in the storage system based on each communication network port; wherein, if a communication connection between a target controller in each controller and a target communication board fails, the target controller completes the task with the storage cabinet based on the communication connection with other communication boards except the target communication board.
[0175] For example, in most scenarios of storage systems, faulty services need to be switched across boards to achieve active / standby switching. In this case, two communication boards need to be inserted into a four-controller chassis. Fig.10 Rear view). Insert a communication card into each of the two slots of the one-frame four-controller system. The two communication cards are connected to the four controllers through PCIe and to the EBoF expansion cabinet through optical fiber. For example, the network ports P1, P2, P3, and P4 of communication card A are connected to the P1, P2, P3, and P4 of the first EBoF; the network ports P1, P2, P3, and P4 of communication card B are connected to the P1, P2, P3, and P4 of the second EBoF. The two communication boards are controlled by the MCS system to serve as the primary and backup of each other. For example, if the service of the P1 port of communication board A fails, it can automatically switch to the P1 port of communication board B. The P2 port of communication board A and the P2 port of communication board B serve as the primary and backup of each other. The P3 and P4 ports of communication board A and the P3 and P4 ports of communication board B serve as the primary and backup of each other. When any port or board fails, it can automatically switch to the backup board without switching the empty controller. For example, if the service is originally on controller A, after switching the communication board, the service is still on controller A, but the service of the communication board is switched to the backup communication board.
[0176] The above detailed networking description of the communication board in the one-frame four-controller storage system solves the design of the system networking solution and the realization of active-standby switching when the board fails. However, during the operation of the system, the BMC and MCS systems are required to manage the communication board. For example, the temperature information, power status, and optical module information of the communication board need to be monitored during operation. The management of the large card is described below.
[0177] In an optional implementation, each controller includes a second hot-swap circuit; the second hot-swap circuit is hot-swap connected to the first hot-swap circuit in the controller connection slot.
[0178] Specifically, each controller includes a second hot-swap circuit; the second hot-swap circuit is hot-swap connected to the first hot-swap circuit in the controller connection slot.
[0179] In an optional embodiment, the second hot-swap circuit communicates with the first hot-swap circuit based on a high-speed serial computer expansion bus. Fig.11 . It mainly includes three main parts: controller, backplane and communication board. The controller part includes BMC (Baseboard Management Controller). BMC is connected to the hot-swap circuit through I2C (Inter-Integrated Circuit). The backplane part has a hot-swap circuit. This hot-swap circuit is connected to the hot-swap circuit in the controller and the hot-swap circuit in the communication board through I2C. The communication board part includes CPLD (Complex Programmable Logic Device). CPLD is connected to the hot-swap circuit through I2C.
[0180] In the storage system provided by the embodiment of the present application, each controller includes a second hot-swap circuit; the second hot-swap circuit is hot-swapped to the first hot-swap circuit in the controller connection slot. This greatly improves the convenience of controller hardware replacement. When a controller fails or needs to be upgraded, there is no need to shut down the entire storage system. The operation and maintenance personnel can directly perform hot-swap operations on the controller, easily unplug the failed or old controller, and then insert a new or repaired controller. For example, in an enterprise-level storage system, if a controller has performance degradation due to hardware aging, it can be quickly replaced without shutting down the system, which minimizes system downtime, reduces the impact on business operations, and significantly improves the maintainability of the system. In addition, the hot-swap connection method ensures that the system can smoothly transition during the process of plugging and unplugging the controller, and will not affect the normal operation of the system due to problems such as sudden power outages and signal interruptions. The first hot-swap circuit and the second hot-swap circuit work together to properly handle the switching of power, signals, etc. at the moment of plugging and unplugging, avoiding electrical shocks, data loss, and system crashes. For example, when the controller needs to be replaced during data transmission, the hot-swap connection can ensure the continuity of data transmission, so that ongoing read and write operations in the storage system are not disturbed and continue to run stably, ensuring the stability of system operation and business continuity.
[0181] In addition, the second hot-swap circuit communicates with the first hot-swap circuit based on a high-speed serial computer expansion bus, thereby ensuring accurate, coordinated, efficient and stable communication between the second hot-swap circuit and the first hot-swap circuit.
[0182] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.
Claims
1. A communication board, characterized in that: The communication board is applied to a storage system, and the communication board includes at least one controller connection slot and at least two communication network ports; wherein: Each controller in the storage head of the storage system is installed on the communication board through each controller connection slot, and is communicatively connected with the storage cabinet of the storage system based on each communication network port in the communication board; Wherein, the communication board also includes a target integrated circuit; wherein: Each of the controllers is communicatively connected to the target integrated circuit via each of the controller connection slots; The target integrated circuit is connected to each of the communication network ports, and is communicatively connected to the storage cabinet based on the communication network ports; Among them, the target integrated circuit is used to detect whether the communication connection between the target integrated circuit and each of the communication network ports is normal; if there is a failure in the communication connection between the target integrated circuit and the communication network port, the target integrated circuit communicates with the storage disk cabinet based on other normal communication network ports.
2. The communication board according to claim 1, characterized in that: Each of the communication network ports is an Ethernet port, and each of the controllers is communicatively connected to the target integrated circuit via a high-speed serial computer expansion bus in each of the controller connection slots; The target integrated circuit converts the high-speed serial computer expansion bus into Ethernet to achieve communication connection between each of the controllers and the storage cabinet.
3. The communication board according to claim 2, characterized in that: The communication board also includes at least one communication network card, the target integrated circuit is connected to each of the communication network cards, and each of the communication network cards is connected to the corresponding communication network port.
4. The communication board according to claim 3, characterized in that: The target integrated circuit is connected to each of the communication network cards for communication based on an I2C channel, and manages the optical modules in each of the communication network cards based on the I2C channel.
5. The communication board according to claim 1, characterized in that: The communication board also includes a clock buffer, one end of which is connected to the target integrated circuit; the other end is connected to each of the controller connection slots; wherein, The clock buffer is used to process the clock signals of the controllers to ensure that the clock signals corresponding to the controllers are of the same source.
6. The communication board according to claim 5, characterized in that: The clock buffer is further used to divide the clock signal corresponding to each of the controllers into at least two sub-clock signals; and transmit each of the sub-clock signals to the target integrated circuit; The target integrated circuit is further used to determine a target clock signal from the sub-clock signal, and transmit the target clock signal to each of the controllers; Each of the controllers is used to communicate with the target integrated circuit based on the target clock signal.
7. The communication board according to claim 1, characterized in that: Each of the controller connection slots is pluggably connected to each of the controllers, wherein the controller connection slot includes a first hot-swap circuit; and each of the controllers is connected to each of the first hot-swap circuits.
8. The communication board according to claim 7, characterized in that: The ground wire in the first hot-swap circuit is connected to the controller based on a first pin; The clock line and the power line in the first hot-swap circuit are connected to the controller based on a second pin; The presence signal detection line in the first hot plug circuit is connected to the controller based on a third pin; wherein the length of the first pin is greater than the length of the second pin, and the length of the second pin is greater than the length of the third pin.
9. A storage system, characterized in that: The storage system comprises a storage head and a storage cabinet and a communication card according to any one of claims 1 to 8, wherein: Each controller in the storage head is installed to the communication board through each controller connection slot in the communication board; and is communicatively connected with the storage cabinet of the storage system based on each communication network port in the communication board.
10. The storage system according to claim 9, characterized in that: If there are multiple controllers, there are main controllers and backup controllers between the controllers, wherein: The main controller is used to detect whether the communication connection between the main controller and each of the standby controllers and the target integrated circuit is normal; if there is an abnormality in the communication connection between the standby controller and the target integrated circuit, the main controller or other normal standby controllers are controlled to complete the task of the abnormal standby controller.
11. The storage system according to claim 10, characterized in that: The main controller is further configured to determine the main controller from among the standby controllers if an abnormality occurs in the communication connection between the main controller and the target integrated circuit.
12. The storage system according to claim 10, characterized in that: The main controller is further configured to send a communication board power-off notification to each of the standby controllers when it is necessary to control the communication board to power off; Each of the standby controllers is used to process a target task related to the communication board after receiving a power-off notification of the communication board; After the target task is processed, a target task processing completion notification is sent to the main controller; The main controller is used to control the communication board to power off and generate log information after receiving the notification that the target task is processed.
13. The storage system according to claim 12, characterized in that: Each of the controllers is used to generate a competition value when the storage system is started, and broadcast the competition value to other controllers through a shared communication channel; Each of the controllers is used to compare the competition value received and sent by the other controllers with the competition value generated by itself; The controller with the largest competition value is determined as the main controller.
14. The storage system according to claim 13, characterized in that: Each of the controllers is used to detect its own CPU usage, memory utilization, I / O throughput, and network delay data; and generate the competition value according to the CPU usage, memory utilization, I / O throughput, and network delay data.
15. The storage system according to claim 9, characterized in that: The storage system includes at least two communication boards, and each of the controllers in the storage head is respectively connected to the controller connection slots in the communication boards, and each of the communication boards is connected to the storage cabinet in the storage system based on the communication network port; wherein, if a communication connection between a target controller in each of the controllers and a target communication board fails, the target controller completes the task with the storage cabinet based on the communication connection with other communication boards except the target communication board.
16. The storage system according to claim 14, characterized in that: Each of the controllers includes a second hot-swap circuit; the second hot-swap circuit is hot-swap connected to the first hot-swap circuit in the connection slot of the controller.
17. The storage system according to claim 16, characterized in that: The second hot swap circuit communicates with the first hot swap circuit based on a high-speed serial computer expansion bus.
Citation Information
Patent Citations
Control method and device based on PCIe chip in storage device
CN115543907A