Master-slave arbitration method and system for dual-controller storage system
By dividing the controller metadata area in the dual-controller storage system and cross-validating the status flags, the problem of misjudgment caused by communication link anomalies is solved, and more accurate master-slave arbitration and system stability are achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-18
- Publication Date
- 2026-03-13
AI Technical Summary
In existing technologies, dual-controller storage systems are prone to misjudging controller failures in the event of communication link anomalies or network latency, leading to unnecessary master-slave switching and increasing implementation costs and complexity.
By dividing the controller metadata area in the shared backend storage, generating and exchanging status flags periodically, and performing cross-validation to confirm the consistency of the other controller's status, misjudgment can be avoided.
It improves the accuracy of master-slave arbitration, reduces frequent switching caused by momentary interruptions in communication links, and ensures system stability and business continuity.
Smart Images
Figure CN121658402A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of storage control technology, and in particular to a master-slave arbitration method and system for a dual-controller storage system. Background Technology
[0002] Dual-controller storage systems offer a highly reliable and high-performance data storage solution. By connecting two controllers to shared backend storage resources, they ensure that if one controller fails, the other can take over the service, guaranteeing business continuity. Both controllers can process requests simultaneously, thereby increasing the system's throughput. In a dual-controller storage system, the node responsible for receiving data and processing related management commands is the master node, and the node coordinating its work is the slave node. When both nodes are functioning normally, the master and slave nodes are determined by a preset node priority. The master-slave arbitration mechanism is crucial for ensuring the stable operation of the dual-controller storage system. When one controller fails or goes offline, the other controller needs to promptly take over its work. The process of identifying which controller has failed involves master-slave arbitration.
[0003] In existing technologies, heartbeat detection is typically used by master and slave controllers to determine if the other controller is faulty or offline. However, heartbeat detection mechanisms can lead to misjudgments due to abnormal heartbeat communication links or network latency, thus triggering unnecessary master-slave switching. Chinese patent CN201153943Y, "A Multi-Heartbeat Signal Dual-Controlled Storage Controller," uses multiple heartbeat channels to independently detect whether the peer controller is responding. If multiple channels remain unresponsive, a fault is identified, and a switchover is triggered. However, this method still essentially infers the peer controller's status by obtaining signals from the peer through the communication link. In scenarios of systemic communication failures or common link failures, misjudgments may still occur, and hardware redundancy and software complexity also lead to higher implementation costs. Summary of the Invention
[0004] Based on this, and in response to the aforementioned technical problems, a master-slave arbitration method and system for a dual-controller storage system are provided to solve the problem that existing technologies are prone to misjudgment, leading to unnecessary master-slave controller switching.
[0005] Firstly, a master-slave arbitration method for a dual-controller storage system, executed by a local controller, wherein the local controller and the peer controller communicate via a communication link, the method comprising:
[0006] The local controller generates a local status flag at a set time interval and writes the local flag into the local controller's local status area in the controller metadata area. The controller metadata area is a region in the shared backend storage shared by the local controller and the peer controller for storing controller metadata. The controller metadata area is visible to both the local controller and the peer controller. The local controller's local status area is used by the local controller to store the local status flag. The local controller is either the primary controller or the backup controller.
[0007] The local status flag is sent to the peer controller via a communication link, so that the peer controller stores the local status flag in the peer controller local status area in the controller metadata area, or enables the peer controller to perform fault diagnosis on the peer controller; the peer controller local status area is used by the peer controller to store the local status flag.
[0008] Within a first preset time period, it is determined whether a peer status flag issued by the peer controller can be received via the communication link. If so, the peer status flag is stored in the local controller peer status area within the controller metadata area; the local controller peer status area is used by the local controller to store peer status flags. If not, within a second preset time period, the latest peer status flag stored in the peer controller peer status area within the controller metadata area is queried multiple times to see if it is consistent with the latest peer status flag stored in the local controller peer status area. If they are always consistent, the peer controller is considered to be faulty; the peer controller peer status area is used by the peer controller to store peer status flags.
[0009] Determine if it is the master controller; if not, promote itself to master controller.
[0010] Optionally, in the above scheme, the local status flag is an automatically generated random number.
[0011] Optionally, in the above scheme, the set time interval is 1 second:
[0012] Optionally, in the above scheme, the preset time is 5 seconds.
[0013] Optionally, in the above scheme, after considering the peer controller to be faulty, the method further includes: writing a suicide flag to identify the peer controller failure in the peer status area of the local controller in the controller metadata area, and sending the peer controller failure information to the management terminal.
[0014] Optionally, in the above scheme, after upgrading itself to a master controller, the process further includes: re-initializing the connection with the storage device and synchronizing the data of the peer controller;
[0015] Optionally, in the above scheme, the latest peer status flag stored in the local area of the peer controller in the peer controller's metadata area is queried multiple times within a second preset time period to see if it is consistent with the latest peer status flag stored in the peer area of the local controller. If they are inconsistent, the communication link is considered to be faulty, and the communication link fault information is sent to the management terminal.
[0016] Secondly, a master-slave arbitration system for a dual-controller storage system, the system comprising: a local controller and a peer controller; the local controller and the peer controller are interconnected via a communication link and simultaneously have read and write access rights to the shared backend storage;
[0017] The shared backend storage is divided into a controller metadata area; the controller metadata area is used to store controller metadata; the controller metadata area is visible to both the local controller and the peer controller.
[0018] The controller metadata area is divided into the local controller's local status area, the local controller's peer status area, the peer controller's local status area, and the peer controller's peer status area. The local controller's local status area is used to store the local controller's local status flag. The local controller's peer status area is used to store the peer controller's peer status flag. The peer controller's local status area is used to store the peer controller's local status flag. The peer controller's peer status area is used to store the peer controller's peer status flag.
[0019] The local controller is configured to: generate a local status flag at set time intervals and write the local flag into the local controller's local status area in the controller's metadata area; send the local status flag to the peer controller via a communication link, so that the peer controller stores the local status flag in the peer controller's local status area in the controller's metadata area; within a first preset time period, determine whether the peer status flag sent by the peer controller can be received via the communication link; if so, store the peer status flag in the peer controller's peer status area in the controller's metadata area; if not, within a second preset time period, repeatedly query whether the latest peer status flag stored in the peer controller's peer status area in the controller's metadata area is consistent with the latest peer status flag stored in the peer controller's peer status area in the local controller; if inconsistent, consider the peer controller to be faulty; determine whether it is the master controller; if not, upgrade itself to the master controller.
[0020] Thirdly, a computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of the master-slave arbitration method of the dual-controller storage system described in the first aspect.
[0021] Fourthly, a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the master-slave arbitration method of the dual-controller storage system described in the first aspect.
[0022] This application has at least the following beneficial effects:
[0023] In this application, both the master and slave controllers periodically update their states and exchange their states via a communication link, simultaneously writing the state flags of both controllers into a shared storage area visible to the other end. The system periodically retrieves the other party's state via the communication link. If the state cannot be retrieved, the system searches for the latest peer state flag stored in the peer's state area of the local controller within the shared storage area, and also searches for the latest peer state flag stored in the peer's state area of the local controller to see if they match. If they do not match, it indicates a fault in the peer controller, indicating that it has not been updated. If the local controller is not the master controller, a master-slave switch is performed. Therefore, this method allows for continuous comparison through backend storage even during a momentary interruption of the communication link, avoiding frequent switching caused by misjudgments in traditional single-heartbeat systems. Simultaneously, cross-validation of the states of the two controllers during verification ensures the accuracy of arbitration. Attached Figure Description
[0024] Figure 1 A flowchart illustrating a master-slave arbitration method for a dual-controller storage system provided in one embodiment of this application;
[0025] Figure 2 A detailed flowchart illustrating a master-slave arbitration method for a dual-controller storage system provided in one embodiment of this application;
[0026] Figure 3 This is a diagram showing the partitioning structure of the controller cloud data area from the perspective of dual controllers in one embodiment of this application;
[0027] Figure 4 This application provides a structural diagram of the controller cloud data area from the perspective of the local controller in one embodiment. Detailed Implementation
[0028] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.
[0029] In one embodiment, such as Figure 1 and Figure 2 As shown, a master-slave arbitration method for a dual-controller storage system is provided, executed by the local controller, which communicates with the peer controller via a communication link. The method includes:
[0030] Step S1: Generate a local status flag according to a set time interval, and write the local flag into the local controller's local status area in the controller metadata area. The controller metadata area is a region in the shared backend storage shared by the local controller and the peer controller for storing controller metadata. The controller metadata area is visible to both the local controller and the peer controller. The local controller's local status area is used by the local controller to store the local status flag. The local controller is either the primary controller or the backup controller.
[0031] Step S2: Send the local status flag to the peer controller via the communication link, so that the peer controller stores the local status flag in the peer controller local status area in the controller metadata area, or so that the peer controller performs fault diagnosis on the peer controller; the peer controller local status area is used by the peer controller to store the local status flag.
[0032] Step S3: Within a first preset time period, determine whether the peer status flag sent by the peer controller can be received through the communication link. If so, store the peer status flag in the peer status area of the local controller in the controller metadata area. The peer status area of the local controller is used by the local controller to store the peer status flag. If not, within a second preset time period, query multiple times whether the latest peer status flag stored in the peer status area of the peer controller in the controller metadata area is consistent with the latest peer status flag stored in the peer status area of the local controller. If they are always consistent, the peer controller is considered to be faulty. The peer status area of the peer controller is used by the peer controller to store the peer status flag.
[0033] Step S4: Determine if you are the main controller. If not, upgrade yourself to the main controller.
[0034] In this embodiment, the specific components include:
[0035] (1) When the system starts, the two controllers (the first controller and the second controller) are initialized and set to the initial master-slave state. Usually, the initial master controller can be determined by pre-configuring the priority. During the initialization process, the system will assign a unique identifier to each controller and establish a communication link between the controllers to ensure that the status information of both parties can be exchanged normally. At the same time, a shared area is allocated in the shared backend storage resources to store the controller status information, which is called the controller metadata area.
[0036] (2) Each controller maintains controller state metadata to record the status flags of its own controller and the status flags of the peer controller. After the initialization phase, the controller updates its controller state metadata according to the initial state settings. Subsequently, the first and second controllers periodically exchange and update the controller state metadata information. Specifically, each controller updates its own controller status flag and reads the status flag of the peer controller, writing it into its own controller state metadata. This process ensures that both controllers can perceive each other's state changes in real time. The specific scheme is as follows:
[0037] 1) The first and second controllers each maintain their own controller status metadata. Each controller status metadata includes the following two fields: local status flag slot0_flag and remote status flag slot1_flag; for example... Figure 3 As shown.
[0038] 2) The first controller and the second controller generate a random number at regular intervals, update the local status flag slot0_flag in the local controller status metadata, and exchange the local status flag slot0_flag in the controller status metadata information through the communication link.
[0039] 3) After receiving the status information from the peer controller, any controller updates its status flag to the peer status flag slot1_flag in the status metadata of its own controller.
[0040] 4) If any controller detects that the status flag of the peer controller has not been updated in time, the anomaly detection process is triggered.
[0041] (3) Cross-validation: The master controller and slave controllers cross-validate each other's controller status flags to detect whether there is a controller failure or offline status. If a controller detects an abnormal status of the peer controller, such as an uninitialized or unupdated controller status flag (the status received from the peer controller last time through the communication link is compared with the latest status stored directly by the peer controller itself; if they match, it is considered that the status flag has not been updated on time), then the controller will trigger a further fault detection process. The cross-validation mechanism can effectively avoid misjudgments caused by network latency or instantaneous jitter, thereby improving the reliability of the system.
[0042] (4) Fault Detection and Arbitration: When an abnormal controller status flag is detected, the controller will initiate a fault detection mechanism to determine whether a fault has actually occurred through multiple verifications and confirmations. If a fault is confirmed in the peer controller, the system will conduct master-slave arbitration based on the current status of the first and second controllers. Specifically, the system will mark the offline controller as "suicide" and trigger the master-slave switchover process. This mechanism ensures that the system can respond quickly and accurately when a fault occurs, avoiding business interruption or data loss due to the impact of a faulty controller.
[0043] (5) During the master-slave switchover, the controller in normal condition will be updated to the master controller and take full control of the storage device's services and data. During the switchover, the system will ensure service continuity and data consistency to avoid service interruption or data loss. The new master controller will reinitialize the connection with the storage device and synchronize the data status of the peer controller to ensure that the system can quickly resume normal operation after the switchover.
[0044] like Figure 3 The diagram shows the division of the controller metadata area in the shared storage region from the perspective of one controller as the local controller and the other as the remote controller.
[0045] In the aforementioned master-slave arbitration method for a dual-controller storage system, both the master and slave controllers periodically update their states and exchange their states via a communication link, simultaneously writing the state flags of both controllers into a shared storage area visible to the other end. The system periodically retrieves the other party's state via the communication link. If the state cannot be retrieved, the system searches for the latest peer state flag previously stored in the peer state area of the local controller within the shared storage area, and also searches for the latest peer state flag stored in the peer state area of the local controller to see if they match. If they do not match, it indicates a fault in the peer controller, indicating that it has not been updated. If the local controller is not the master controller, a master-slave switch is performed. Therefore, this method allows for continuous comparison through backend storage even during momentary communication link interruptions, avoiding frequent switching caused by misjudgments in traditional single-heartbeat systems. Simultaneously, cross-validation of the states of the two controllers during verification ensures the accuracy of the arbitration.
[0046] This application combines the characteristics of the dual-controller storage system itself. This solution saves the status information to the shared backend resources, periodically reads the content of the peer controller and verifies the data update status, avoids the defects of the traditional heartbeat detection mechanism, can accurately determine whether the controller is faulty, and improves the accuracy of master-slave arbitration.
[0047] In one embodiment, the local status flag is an automatically generated random number.
[0048] In one embodiment, the set time interval is 1 second.
[0049] In one embodiment, the first preset time is 5 seconds.
[0050] In one embodiment, the step of assuming a fault in the peer controller further includes: writing a suicide flag to identify the fault in the peer controller's status area in the local controller's metadata area, and sending the peer controller fault information to the management terminal.
[0051] In one embodiment, upgrading itself to a master controller further includes: re-initializing the connection with the storage device and synchronizing the data of the peer controller;
[0052] In one embodiment, the latest peer status flag stored in the local area of the peer controller in the controller metadata area is queried to see if it is consistent with the latest peer status flag stored in the peer area of the local controller. If they are inconsistent, the communication link is considered to be faulty, and the communication link fault information is sent to the management terminal.
[0053] In one embodiment, a master-slave arbitration system for a dual-controller storage system is provided. The system includes: a local controller and a peer controller; the local controller and the peer controller are interconnected via a communication link and simultaneously have read and write access rights to the shared backend storage.
[0054] The shared backend storage is divided into a controller metadata area; the controller metadata area is used to store controller metadata; the controller metadata area is visible to both the local controller and the peer controller.
[0055] The controller metadata area is divided into the local controller's local status area, the local controller's peer status area, the peer controller's local status area, and the peer controller's peer status area. The local controller's local status area is used to store the local controller's local status flag. The local controller's peer status area is used to store the peer controller's peer status flag. The peer controller's local status area is used to store the peer controller's local status flag. The peer controller's peer status area is used to store the peer controller's peer status flag.
[0056] The local controller is configured to: generate a local status flag at set time intervals and write the local flag into the local controller's local status area in the controller's metadata area; send the local status flag to the peer controller via a communication link, so that the peer controller stores the local status flag in the peer controller's local status area in the controller's metadata area; within a first preset time period, determine whether the peer status flag sent by the peer controller can be received via the communication link; if so, store the peer status flag in the peer controller's peer status area in the controller's metadata area; if not, within a second preset time period, repeatedly query whether the latest peer status flag stored in the peer controller's peer status area in the controller's metadata area is consistent with the latest peer status flag stored in the peer controller's peer status area in the local controller; if inconsistent, consider the peer controller to be faulty; determine whether it is the master controller; if not, upgrade itself to the master controller.
[0057] The advantages of this application are:
[0058] 1. Controller metadata is stored in a shared backend storage to ensure state consistency.
[0059] 2. Regularly exchange and update the metadata information of each controller, and maintain each controller independently to ensure that the status information is accurate.
[0060] 3. The controller status metadata includes the status flags of the local controller and the status flags of the remote controller. During verification, the status of the two controllers is cross-verified to ensure the accuracy of arbitration.
[0061] 4. By analyzing the metadata status of the two controllers, it can be determined whether there is a problem with the communication link between the two controllers.
[0062] For specific limitations regarding the master-slave arbitration system of a dual-controller storage system, please refer to the limitations on the master-slave arbitration method of a dual-controller storage system mentioned above, which will not be repeated here. Each module in the aforementioned master-slave arbitration system of a dual-controller storage system can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in the computer device in hardware form, or stored in the memory of the computer device in software form, so that the processor can call and execute the corresponding operations of each module.
[0063] In one embodiment, a computer device is provided, which may be a server. The computer device includes a processor, memory, and a network interface connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The network interface is used to communicate with external terminals via a network connection. When the computer program is executed by the processor, it implements the master-slave arbitration method of a dual-controller storage system described above.
[0064] In one embodiment, a computer-readable storage medium is also provided, on which a computer program is stored relating to all or part of the processes in the methods of the above embodiments.
[0065] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the methods described above. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, or optical storage, etc. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.
[0066] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0067] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.
Claims
1. A master-slave arbitration method for a dual-controller storage system, characterized in that, The method, executed by the local controller, which communicates with the remote controller via a communication link, includes: The local controller generates a local status flag at a set time interval and writes the local flag into the local controller's local status area in the controller metadata area. The controller metadata area is a region in the shared backend storage shared by the local controller and the peer controller for storing controller metadata. The controller metadata area is visible to both the local controller and the peer controller. The local controller's local status area is used by the local controller to store the local status flag. The local controller is either the primary controller or the backup controller. The local status flag is sent to the peer controller via a communication link, so that the peer controller stores the local status flag in the peer controller local status area in the controller metadata area, or enables the peer controller to perform fault diagnosis on the peer controller; the peer controller local status area is used by the peer controller to store the local status flag. Within a first preset time period, it is determined whether a peer status flag issued by the peer controller can be received via the communication link. If so, the peer status flag is stored in the local controller peer status area within the controller metadata area; the local controller peer status area is used by the local controller to store peer status flags. If not, within a second preset time period, the latest peer status flag stored in the peer controller peer status area within the controller metadata area is queried multiple times to see if it is consistent with the latest peer status flag stored in the local controller peer status area. If they are always consistent, the peer controller is considered to be faulty; the peer controller peer status area is used by the peer controller to store peer status flags. Determine if it is the master controller; if not, promote itself to master controller.
2. The master-slave arbitration method for a dual-controller storage system according to claim 1, characterized in that, The local status flag is an automatically generated random number.
3. The master-slave arbitration method for a dual-controller storage system according to claim 1, characterized in that, The set time interval is 1 second.
4. The master-slave arbitration method for a dual-controller storage system according to claim 1, characterized in that, The first preset time is 5 seconds.
5. The master-slave arbitration method for a dual-controller storage system according to claim 1, characterized in that, The process of determining that the peer controller has failed also includes: writing a suicide flag to identify the peer controller failure in the peer status area of the local controller in the controller metadata area, and sending the peer controller failure information to the management terminal.
6. The master-slave arbitration method for a dual-controller storage system according to claim 1, characterized in that, The process of upgrading itself to a master controller also includes: re-initializing the connection with the storage device and synchronizing the data of the peer controller.
7. The master-slave arbitration method for a dual-controller storage system according to claim 1, characterized in that, Within a second preset time period, the system repeatedly queries whether the latest peer status flag stored in the local area of the peer controller in the controller metadata area is consistent with the latest peer status flag stored in the peer area of the local controller. If they are inconsistent, the communication link is considered to be faulty, and the communication link fault information is sent to the management terminal.
8. A master-slave arbitration system for a dual-controller storage system, characterized in that, The system includes: a local controller and a remote controller; the local controller and the remote controller are interconnected through a communication link and simultaneously have read and write access rights to the shared backend storage; The shared backend storage is divided into a controller metadata area; the controller metadata area is used to store controller metadata; the controller metadata area is visible to both the local controller and the peer controller. The controller metadata area is divided into the local controller's local status area, the local controller's peer status area, the peer controller's local status area, and the peer controller's peer status area. The local controller's local status area is used to store the local controller's local status flag. The local controller's peer status area is used to store the peer controller's peer status flag. The peer controller's local status area is used to store the peer controller's local status flag. The peer controller's peer status area is used to store the peer controller's peer status flag. The local controller is configured to: generate a local status flag at set time intervals and write the local flag into the local controller's local status area in the controller's metadata area; send the local status flag to the peer controller via a communication link, so that the peer controller stores the local status flag in the peer controller's local status area in the controller's metadata area; within a first preset time period, determine whether the peer status flag sent by the peer controller can be received via the communication link; if so, store the peer status flag in the peer controller's peer status area in the controller's metadata area; if not, within a second preset time period, repeatedly query whether the latest peer status flag stored in the peer controller's peer status area in the controller's metadata area is consistent with the latest peer status flag stored in the peer controller's peer status area in the local controller; if inconsistent, consider the peer controller to be faulty; determine whether it is the master controller; if not, upgrade itself to the master controller.
9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.
Citation Information
Patent Citations
Multi-heartbeat-signal double-control storage control
CN201153943Y