Method, device and electronic equipment for monitoring controller status
The state of the storage system is obtained and judged in real time through software monitoring methods, which solves the shortcomings of relying on hardware status monitoring in the prior art, realizes efficient monitoring of the controller status, and improves the stability and reliability of the storage system.
Patent Information
- Application Number
- CN202410820887.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-06-24
- Publication Date
- 2025-03-04
- Estimated Expiration
- 2044-06-24
AI Technical Summary
The existing methods of monitoring the controller's status mainly rely on the hardware status, and cannot effectively monitor the software status of the controller in real time, resulting in untimely task transfer when the controller fails, affecting the reliability and performance of the storage system.
By using software monitoring, by obtaining the status of the storage system, we determine whether the peer controller is online, and obtain the disk information of the local and peer controllers respectively to determine whether the controller status of the controller group is normal. If the peer controller is not online, determine whether a split brain occurs and arbitrate is carried out through the arbitration system to determine the status of the controller group.
It realizes efficient real-time monitoring of the controller status, improves the stability and reliability of the storage system, and ensures that tasks can be taken over in time when the controller fails and returns in time when recovery.
Smart Images

Figure CN118820008B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of computer storage technology, and in particular to a method, device and electronic equipment for monitoring controller status. Background Art
[0002] A storage system, such as a disk array, usually includes multiple hard disks, and then one or more controllers are used to collaboratively manage each hard disk. When one of the controllers fails, the management tasks it is responsible for can be transferred to other controllers for processing, avoiding the problem of being unable to perform read / write operations on the storage system due to the failure of the controller, thereby improving the reliability of the storage system; and when the controller recovers to normal, the previously transferred management tasks are returned to the controller, so that the controllers of the entire storage system work together, avoiding the problem that one controller bears the tasks of multiple controllers for a long time, which affects the performance of the storage system and the service life of the controller. In order to achieve that when a controller fails, the management tasks responsible for the controller can be taken over in time, and when the failed controller recovers, the management tasks responsible for the controller can be returned in time; it is necessary to monitor the status of each controller in the controller group in real time. However, the inventor found that the existing method of monitoring the status of the controller is usually implemented by monitoring the hardware status of the controller, for example, monitoring the status of the central processing unit of the controller. If the status of the central processing unit of the controller is abnormal, the controller is considered to be failed. Summary of the invention
[0003] The embodiments of the present invention provide a method, device and electronic device for monitoring the state of a controller, which are used to implement efficient monitoring of the state of the controller in a software monitoring manner, thereby improving the stability and reliability of a storage system.
[0004] The specific technical solutions are as follows:
[0005] In a first aspect, an embodiment of the present invention provides a method for monitoring controller status, which is applied to a storage system, wherein the storage system includes: a controller group and a shared disk frame; the controller group includes: a first controller and a second controller; the shared disk frame includes at least one disk; the method includes:
[0006] Enable an online controller in the controller group to obtain the state of the storage system; wherein the state of the storage system includes: the number and identification of online controllers; the online controller is a controller in a working state;
[0007] Determining whether the opposite controller is online according to the state of the storage system;
[0008] If the opposite controller is online, respectively obtain disk information in the local controller and disk information in the opposite controller; determine whether the controller status in the controller group is normal according to the disk information in the local controller and the disk information in the opposite controller;
[0009] If the controller at the other end is not online, determine whether a brain split occurs in the controller at the other end; if a brain split occurs at the other end, use the arbitration system to arbitrate the controller at the other end; based on the arbitration result, determine whether the states of the controllers in the controller group are normal; if a brain split does not occur at the other end, determine that the states of the controllers in the controller group are all normal.
[0010] Optionally, the step of determining whether the controller status in the controller group is normal according to the disk information in the local controller and the disk information in the opposite controller includes:
[0011] Determine whether the disk information in the local controller is consistent with the disk information in the peer controller;
[0012] If yes, determining that the controller states in the controller group are all normal;
[0013] If not, determine the inconsistent disk information in the local controller and the opposite controller, and issue a disk asynchronous alarm message; obtain the disk information of the used disks in the local controller and the opposite controller respectively; determine whether the disk information of the used disks in the local controller is consistent with the disk information of the used disks in the opposite controller; if yes, determine that the controller states in the controller group are all normal; if not, determine whether the number of valid disks in the used disks in the local controller and the opposite controller is equal according to the purpose of the disk; if they are equal and the local controller is the first controller, determine that the controller states in the controller group are all normal; if they are equal and the local controller is the second controller, determine that the controller states in the controller group are abnormal; if the number of valid disks in the used disks in the local controller is more than that in the opposite controller, determine that the controller states in the controller group are all normal; if the number of valid disks in the used disks in the local controller is less than that in the opposite controller, determine that the controller states in the controller group are abnormal.
[0014] Optionally, the step of determining whether the controller status in the controller group is normal according to the arbitration result includes:
[0015] If the arbitration is successful, determining that the states of the controllers in the controller group are all normal;
[0016] If arbitration fails, it is determined that the states of controllers in the controller group are abnormal.
[0017] Optionally, the first controller and the second controller both include the following states:
[0018] starting state; indicating that the resource agent on the controller is starting; wherein the starting state is a temporary state. If the controller is in this state for a period longer than a preset period, it indicates that the controller state is abnormal and an abnormal state alarm needs to be issued;
[0019] The started state indicates that the resource agent on the controller has been started successfully.
[0020] The stopping state indicates that the resource agent on the controller is stopping. The stopping state is a temporary state. If the controller is in this state for a period longer than a preset period, it indicates that the controller is abnormal and an abnormal state alarm needs to be issued.
[0021] The stopped state indicates that the resource agent on the controller has been successfully stopped, or the resource agent on the controller has failed to start on both the first controller and the second controller;
[0022] Unmanaged state: indicates that the resource agent on the controller has entered an abnormal state and the monitoring system needs to issue an abnormal alarm.
[0023] Optionally, the state of the storage system is determined by the states of the first controller and the second controller; wherein the resource agent corresponding to the first controller is a first resource agent, and the resource agent corresponding to the second controller is a second resource agent;
[0024] When the first resource agent is in the started state on the first controller, and the second resource agent is in the started state on the second controller, it is confirmed that the state of the storage system is normal;
[0025] When both the first resource agent and the second resource agent are in the started state on the first controller, it is confirmed that the state of the storage system is normal; wherein the second controller is abnormal, and the first controller has successfully taken over the second resource agent;
[0026] When both the first resource agent and the second resource agent are in the started state on the second controller, it is confirmed that the state of the storage system is normal; wherein the first controller is abnormal and the second controller has successfully taken over the first resource agent;
[0027] When the first resource agent is in the started state on the first controller and the second resource agent is in the stopped state, it is confirmed that the state of the storage system is abnormal; wherein the second controller is abnormal and the first controller fails to take over the second resource agent;
[0028] When the second resource agent is in the started state on the second controller and the first resource agent is in the stopped state, it is confirmed that the state of the storage system is abnormal; wherein the first controller is abnormal and the second controller fails to take over the first resource agent;
[0029] When the first resource agent is in the started state on the second controller and the second resource agent is in the stopped state, it is confirmed that the state of the storage system is an abnormal state; wherein both the first controller and the second controller are in the abnormal state, and the second controller has taken over the first resource agent;
[0030] When the second resource agent is in the started state on the first controller and the first resource agent is in the stopped state, it is confirmed that the state of the storage system is an abnormal state; wherein both the first controller and the second controller are in the abnormal state, and the first controller has taken over the second resource agent;
[0031] When both the first resource agent and the second resource agent are in the stopped state, it is confirmed that the state of the storage system is an abnormal state;
[0032] When the first resource agent and the second resource agent are both in the started state on the first controller and the second controller, it is confirmed that the state of the storage system is abnormal; wherein the arbitration system is abnormal and a brain split occurs in the storage system.
[0033] Optionally, the process of starting the resource agent on the controller is:
[0034] Enable the controller in the controller group to obtain disk information in the local controller and disk information of used disks in the local controller;
[0035] If the acquisition is successful, the identification information of the local controller is obtained; whether the identification letter of the controller corresponding to the resource agent program to be started is the same as the identification information of the local controller; if the identification letter of the controller corresponding to the resource agent program to be started is the same as the identification information of the local controller, the disk information of the used disks in the local controller is obtained; whether the disk information in the local controller completely contains the disk information of the used disks of the local controller; if it is completely contained, it is confirmed that the resource agent program to be started is started successfully; if it is not completely contained, it is confirmed that the resource agent program to be started fails to start; if the identification letter of the controller corresponding to the resource agent program to be started is not the same as the identification information of the local controller, the disk information of the used disks in the opposite controller is obtained; whether the disk information in the local controller completely contains the disk information of the used disks of the opposite controller; if it is completely contained, it is confirmed that the resource agent program to be started is started successfully and a takeover notification is issued; if it is not completely contained, it is confirmed that the resource agent program to be started fails to start;
[0036] If the acquisition fails, it is confirmed that the resource agent program to be started fails to start.
[0037] Optionally, the resource agent on the controller is stopped in the following manner:
[0038] The resource agent on the controller is stopped by using the stop method; wherein the stop method is a no-operation method.
[0039] In a second aspect, an embodiment of the present invention provides a device for monitoring a state of a controller, the device comprising:
[0040] An acquisition module, used to enable an online controller in the controller group to acquire the state of the storage system; wherein the state of the storage system includes: the number and identification of online controllers; the online controller is a controller in a working state;
[0041] A judgment module, used to judge whether the opposite end controller is online according to the state of the storage system;
[0042] The first determination module is used to obtain the disk information in the local controller and the disk information in the opposite controller respectively when the opposite controller is online; and determine whether the controller status in the controller group is normal according to the disk information in the local controller and the disk information in the opposite controller;
[0043] The second determination module is used to determine whether a brain split occurs in the controller at the other end when the controller at the other end is not online; if a brain split occurs at the other end, arbitrate the controller at the other end using the arbitration system; determine whether the status of the controllers in the controller group is normal based on the arbitration result; if a brain split does not occur at the other end, determine that the status of the controllers in the controller group are all normal.
[0044] Optionally, the first determining module is specifically configured to:
[0045] Determine whether the disk information in the local controller is consistent with the disk information in the peer controller;
[0046] If yes, determining that the controller states in the controller group are all normal;
[0047] If not, determine the inconsistent disk information in the local controller and the opposite controller, and issue a disk asynchronous alarm message; obtain the disk information of the used disks in the local controller and the opposite controller respectively; determine whether the disk information of the used disks in the local controller is consistent with the disk information of the used disks in the opposite controller; if yes, determine that the controller states in the controller group are all normal; if not, determine whether the number of valid disks in the used disks in the local controller and the opposite controller is equal according to the purpose of the disk; if they are equal and the local controller is the first controller, determine that the controller states in the controller group are all normal; if they are equal and the local controller is the second controller, determine that the controller states in the controller group are abnormal; if the number of valid disks in the used disks in the local controller is more than that in the opposite controller, determine that the controller states in the controller group are all normal; if the number of valid disks in the used disks in the local controller is less than that in the opposite controller, determine that the controller states in the controller group are abnormal.
[0048] Optionally, the second determining module is specifically configured to:
[0049] When the arbitration succeeds, determining that the states of the controllers in the controller group are all normal;
[0050] When arbitration fails, it is determined that the states of controllers in the controller group are abnormal.
[0051] Optionally, the first controller and the second controller both include the following states:
[0052] starting state; indicating that the resource agent on the controller is starting; wherein the starting state is a temporary state. If the controller is in this state for a period longer than a preset period, it indicates that the controller state is abnormal and an abnormal state alarm needs to be issued;
[0053] The started state indicates that the resource agent on the controller has been started successfully.
[0054] The stopping state indicates that the resource agent on the controller is stopping. The stopping state is a temporary state. If the controller is in this state for a period longer than a preset period, it indicates that the controller is abnormal and an abnormal state alarm needs to be issued.
[0055] The stopped state indicates that the resource agent on the controller has been successfully stopped, or the resource agent on the controller has failed to start on both the first controller and the second controller;
[0056] Unmanaged state: indicates that the resource agent on the controller has entered an abnormal state and the monitoring system needs to issue an abnormal alarm.
[0057] Optionally, the state of the storage system is determined by the states of the first controller and the second controller; wherein the resource agent corresponding to the first controller is a first resource agent, and the resource agent corresponding to the second controller is a second resource agent;
[0058] When the first resource agent is in the started state on the first controller, and the second resource agent is in the started state on the second controller, it is confirmed that the state of the storage system is normal;
[0059] When both the first resource agent and the second resource agent are in the started state on the first controller, it is confirmed that the state of the storage system is normal; wherein the second controller is abnormal, and the first controller has successfully taken over the second resource agent;
[0060] When both the first resource agent and the second resource agent are in the started state on the second controller, it is confirmed that the state of the storage system is normal; wherein the first controller is abnormal and the second controller has successfully taken over the first resource agent;
[0061] When the first resource agent is in the started state on the first controller and the second resource agent is in the stopped state, it is confirmed that the state of the storage system is abnormal; wherein the second controller is abnormal and the first controller fails to take over the second resource agent;
[0062] When the second resource agent is in the started state on the second controller and the first resource agent is in the stopped state, it is confirmed that the state of the storage system is abnormal; wherein the first controller is abnormal and the second controller fails to take over the first resource agent;
[0063] When the first resource agent is in the started state on the second controller and the second resource agent is in the stopped state, it is confirmed that the state of the storage system is an abnormal state; wherein both the first controller and the second controller are in the abnormal state, and the second controller has taken over the first resource agent;
[0064] When the second resource agent is in the started state on the first controller and the first resource agent is in the stopped state, it is confirmed that the state of the storage system is an abnormal state; wherein both the first controller and the second controller are in the abnormal state, and the first controller has taken over the second resource agent;
[0065] When both the first resource agent and the second resource agent are in the stopped state, it is confirmed that the state of the storage system is an abnormal state;
[0066] When the first resource agent and the second resource agent are both in the started state on the first controller and the second controller, it is confirmed that the state of the storage system is abnormal; wherein the arbitration system is abnormal and a brain split occurs in the storage system.
[0067] Optionally, the device further includes: a program startup module, configured to:
[0068] Enable the controller in the controller group to obtain disk information in the local controller and disk information of used disks in the local controller;
[0069] If the acquisition is successful, the identification information of the local controller is obtained; whether the identification letter of the controller corresponding to the resource agent program to be started is the same as the identification information of the local controller; if the identification letter of the controller corresponding to the resource agent program to be started is the same as the identification information of the local controller, the disk information of the used disks in the local controller is obtained; whether the disk information in the local controller completely contains the disk information of the used disks of the local controller; if it is completely contained, it is confirmed that the resource agent program to be started is started successfully; if it is not completely contained, it is confirmed that the resource agent program to be started fails to start; if the identification letter of the controller corresponding to the resource agent program to be started is not the same as the identification information of the local controller, the disk information of the used disks in the opposite controller is obtained; whether the disk information in the local controller completely contains the disk information of the used disks of the opposite controller; if it is completely contained, it is confirmed that the resource agent program to be started is started successfully and a takeover notification is issued; if it is not completely contained, it is confirmed that the resource agent program to be started fails to start;
[0070] If the acquisition fails, it is confirmed that the resource agent program to be started fails to start.
[0071] Optionally, the device further includes: a program stopping module, configured to:
[0072] The resource agent on the controller is stopped by using the stop method; wherein the stop method is a no-operation method.
[0073] In a third aspect, an embodiment of the present invention further provides an electronic device, comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus;
[0074] The memory is used to store computer programs;
[0075] The processor is used to implement the method steps of monitoring the controller status described in the first aspect when executing the program stored in the memory.
[0076] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method steps for monitoring the controller status described in the first aspect are implemented.
[0077] As can be seen from the above, the method provided by the embodiment of the present invention can enable the online controller in the controller group to obtain the state of the storage system; determine whether the opposite controller is online according to the state of the storage system; if the opposite controller is online, obtain the disk information in the local controller and the disk information in the opposite controller respectively; determine whether the controller state in the controller group is normal according to the disk information in the local controller and the disk information in the opposite controller; if the opposite controller is not online, determine whether the opposite controller has a brain split; if the opposite end has a brain split, use the arbitration system to arbitrate the opposite controller; determine whether the controller state in the controller group is normal according to the arbitration result; if the opposite end does not have a brain split, determine that the controller state in the controller group is normal. Therefore, the method provided by the embodiment of the present invention can be implemented to efficiently monitor the state of the controller in a software monitoring manner, thereby improving the stability and reliability of the storage system.
[0078] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description, claims, and drawings.
[0079] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS
[0080] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:
[0081] Figure 1 A schematic diagram of a flow chart of a method for monitoring controller status provided by an embodiment of the present invention;
[0082] Figure 2 A schematic diagram of a process flow of starting a resource agent program on a controller provided by an embodiment of the present invention;
[0083] Figure 3 Another flowchart of a method for monitoring controller status provided by an embodiment of the present invention;
[0084] Figure 4 A schematic diagram of a structure of a device for monitoring controller status provided by an embodiment of the present invention;
[0085] Figure 5 A schematic diagram of the internal structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0086] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.
[0087] Before the present invention realizes monitoring the status of the controller in the dual-control storage system, an HA resource agent is first written for the dual-control storage system based on pacemaker and Corosync to realize the start, stop, and monitor methods of the resource agent program. Among them, HA is the abbreviation of High Available. In the present invention, the dual-control storage system refers to a storage system including two controllers; HA refers to the Red Hat high availability open source component; the resource agent is essentially a program designed according to the interface standard given by HA, and the program can be used as a resource of HA for high availability management, for example, to realize resource start, stop, monitoring, etc.; resources refer to the running resource agent program.
[0088] Pacemaker: A resource manager split from the original Heartbeat, used to manage the entire HA control center. The client uses pacemaker to configure, manage, and monitor the entire storage system. Pacemaker cannot provide the underlying heartbeat information transmission function. If it wants to communicate with the other node, it needs to use the underlying (newly split heartbeat or corosync) heartbeat transmission service to notify the other party of the information.
[0089] Corosync is an intermediate layer that integrates the API provided by the underlying nodes and pacemaker. When transmitting information, Corosync can define the information transmission method and protocol through a simple configuration file.
[0090] In a specific embodiment, the resource agent name may be RedStor2, and the method of use is: name the file name of the resource agent program RedStor2, put it into the / usr / lib / ocf / resource.d / heartbeat / directory, and specify the resource agent through ocf:heartbeat:RedStor2 when creating resources.
[0091] The resource agent in the present invention is implemented according to the requirements of the HA resource agent. The present invention monitors the status of the controller in the dual-control storage system based on the following ideas:
[0092] If the resource agent program can run normally, it means that the controller has the conditions to run the program, that is, the controller is in a normal state.
[0093] If the disks identified by the controller completely contain the disks used by the controller, it can be regarded as the successful startup of the resource agent. In the present invention, the used disks are used as the condition for judging whether the resource agent is successfully started, so that the method of controlling the startup of the resource agent associated with the storage business is more flexible.
[0094] Use the consistency of disk information as a condition for monitoring the controller status. In a dual-controller storage system, the disks on both controllers can be identified. Therefore, the consistency of disk information can be used as a condition for determining the controller status. Since the WWN (world wide number, the hardware's globally unique identifier) of the same disk is fixed, the disk link file / dev / disk / by-id / wwn-x...xx can be used as the basis for consistency judgment.
[0095] In the present invention, the core of the start method is to ensure that the disks identified by the local controller completely include the used disks, so that the dual-control storage system can provide storage services. To achieve this goal, the dual-control storage system needs to accurately record the disk usage information on both controllers when using the disks.
[0096] In the present invention, the stop method is a no-operation method, that is, it returns a successful operation without performing any operation. This design is due to the following reasons: 1. The start method does not involve the operation of starting the resource agent, so there is no reverse operation. 2. Directly returning a successful operation will make the resource agent more robust, and will not cause the resource to enter an abnormal state due to the failure of the stop method. For example, if the resource fails to stop, the HA will set the resource to the unmanaged state and wait for the administrator to intervene, and will no longer manage the resource. 3. It meets the ultimate goal of using the running status of the resource agent to represent the controller status. As long as the controller has the status of running the program, the resource agent must be able to run all the time.
[0097] There are two main types of Pacemaker resources, namely LSB and OCF. Among them, LSB is the Linux standard service, usually some scripts in the / etc / init.d directory. Pacemaker can use these scripts to start and stop the resource agent service; the currently supported resources can be seen through the command crm ra list lsb. OCF is actually an extension of the LSB service. Compared with LSB, it adds some high-availability cluster management functions (such as fault monitoring, etc.) and more meta-information; the currently supported resources can be seen through the command crm ralist ocf. In order for Pacemaker to be able to provide high-availability guarantees for services, an OCF resource must be implemented. Therefore, in the present invention, the resource agent under the OCF standard is used.
[0098] Figure 1 A flow chart of a method for monitoring controller status provided by an embodiment of the present invention is provided. The method is applied to a storage system. The storage system is a dual-controller storage system. The storage system includes: a controller group and a shared disk frame; the controller group includes: a first controller and a second controller; the shared disk frame includes at least one disk; the method includes:
[0099] S101: enabling an online controller in a controller group to obtain a state of a storage system; wherein the state of the storage system includes: the number and identification of online controllers; an online controller is a controller in a working state;
[0100] Get the status of the storage system, that is, get the status of HA.
[0101] Specifically, the monitoring method provided by the embodiment of the present invention can be implemented through a script program, namely, the resource agent RedStor2.
[0102] S102: judging whether the peer controller is online according to the state of the storage system; if the peer controller is online, executing step S103; if the peer controller is not online, executing step S104;
[0103] The peer controller refers to another controller relative to the local controller.
[0104] It can be judged by the number of nodes and node names in the HA status. For example, if the number of nodes is 1, it means that only the local controller is online, and if it is 2, it means that both controllers are online. The node refers to the controller.
[0105] In a specific embodiment, if the peer controller is online, the step of the split-brain check is skipped directly, and the steps of obtaining the disk information in the local controller and the disk information in the peer controller are directly executed; if the peer controller is offline, that is, the peer controller is not online, it is necessary to use the split-brain system to determine whether the peer controller is in a split-brain state. If there is no split-brain, it means that the peer controller is offline and the returned resources are normal. If a split-brain occurs, it is necessary to use the arbitration result of the arbitration system to make the node that succeeds in arbitration return normal resources and the node that fails in arbitration return abnormal resources. If the HA itself has configured the arbitration system (fence device), this step is ignored, because when the split-brain occurs, the HA has already stonithed the node that failed in arbitration. Among them, split-brain refers to a problem in the path between the controllers, that is, the controller is actually normal, but cannot detect the existence of the other party, so the two controllers will try to take over the storage resources of the other party, which is similar to schizophrenia.
[0106] S103: respectively obtaining disk information in the local controller and disk information in the opposite controller; and determining whether the controller status in the controller group is normal according to the disk information in the local controller and the disk information in the opposite controller;
[0107] In a specific embodiment, the step of determining whether the controller status in the controller group is normal according to the disk information in the local controller and the disk information in the opposite controller includes:
[0108] Determine whether the disk information in the local controller is consistent with the disk information in the peer controller;
[0109] If yes, then determine that the controller statuses in the controller group are all normal;
[0110] If not, determine the inconsistent disk information in the local controller and the opposite controller, and issue a disk asynchronous alarm message; obtain the disk information of the used disks in the local controller and the opposite controller respectively; determine whether the disk information of the used disks in the local controller and the opposite controller are consistent; if yes, determine that the controller states in the controller group are all normal; if not, determine whether the number of valid disks in the used disks in the local controller and the opposite controller are equal according to the purpose of the disk; if they are equal and the local controller is the first controller, determine that the controller states in the controller group are all normal; if they are equal and the local controller is the second controller, determine that the controller states in the controller group are abnormal; if the number of valid disks in the used disks in the local controller is more than that in the opposite controller, determine that the controller states in the controller group are all normal; if the number of valid disks in the used disks in the local controller is less than that in the opposite controller, determine that the controller states in the controller group are abnormal.
[0111] In practical applications, the validity of a disk needs to be judged based on the specific purpose of the disk. For example, in a RAID scenario, the number of RAIDs that can be run on the disk, the service priority of the RAID, etc. can be used as the criteria for judging the validity of the disk to judge the validity of the disk. In an embodiment of the present invention, if the number of valid disks in the local controller is large, the resource agent status is returned as normal; if the number of valid disks in the opposite controller is large, the resource agent status is returned as abnormal; if the number of valid disks in the local controller and the opposite controller is the same, the resource agent is only allowed to run on the first controller. This is because although the number of valid disks in the local controller and the opposite controller is the same, the disk information in the local controller and the opposite controller is inconsistent, so only the executable resource agent can be run as much as possible.
[0112] S104: Determine whether a split-brain occurs in the controller at the other end; if a split-brain occurs in the other end, use the arbitration system to arbitrate the controller at the other end; based on the arbitration result, determine whether the status of the controllers in the controller group is normal; if a split-brain does not occur in the other end, determine that the status of the controllers in the controller group are all normal.
[0113] In a specific embodiment, the step of determining whether the controller status in the controller group is normal according to the arbitration result may include:
[0114] If arbitration is successful, it is determined that the controllers in the controller group are all in normal status;
[0115] If arbitration fails, it is determined that the controller status in the controller group is abnormal.
[0116] In a specific embodiment, the controllers in the controller group of the dual-controller storage system of the present invention, namely the first controller and the second controller, may both have the following states:
[0117] Starting state: indicates that the resource agent on the controller is starting. The starting state is a temporary state. If the controller is in this state for longer than the preset time, it indicates that the controller is abnormal and an abnormal state alarm needs to be issued.
[0118] The started state indicates that the resource agent on the controller has been started successfully.
[0119] The stopping state indicates that the resource agent on the controller is stopping. The stopping state is a temporary state. If the controller is in this state for a period longer than the preset time, it indicates that the controller is abnormal and an abnormal state alarm needs to be issued.
[0120] Stopped state; indicates that the resource agent on the controller is stopped successfully, or the resource agent on the controller fails to start on both the first controller and the second controller;
[0121] Unmanaged state: indicates that the resource agent on the controller has entered an abnormal state, and the monitoring system needs to issue an abnormal alarm. The abnormal alarm is used to issue an asynchronous alarm when the disk information in the two controllers is inconsistent. The asynchronous alarm does not take up too much monitoring time.
[0122] In the present invention, the state of the storage system is determined by the states of the first controller and the second controller; wherein the resource agent corresponding to the first controller is the first resource agent, and the resource agent corresponding to the second controller is the second resource agent;
[0123] When the first resource agent is in the started state on the first controller, and the second resource agent is in the started state on the second controller, it is confirmed that the state of the storage system is normal;
[0124] When both the first resource agent and the second resource agent are in the started state on the first controller, it is confirmed that the storage system is in a normal state; wherein the second controller is abnormal, and the first controller has successfully taken over the second resource agent;
[0125] When both the first resource agent and the second resource agent are in the started state on the second controller, it is confirmed that the state of the storage system is normal; wherein the first controller is abnormal and the second controller has successfully taken over the first resource agent;
[0126] When the first resource agent is in the started state on the first controller and the second resource agent is in the stopped state, it is confirmed that the state of the storage system is abnormal; wherein the second controller is abnormal and the first controller fails to take over the second resource agent;
[0127] When the second resource agent is in the started state on the second controller and the first resource agent is in the stopped state, it is confirmed that the state of the storage system is abnormal; wherein the first controller is abnormal and the second controller fails to take over the first resource agent;
[0128] When the first resource agent is in the started state on the second controller and the second resource agent is in the stopped state, it is confirmed that the state of the storage system is an abnormal state; wherein both the first controller and the second controller are in the abnormal state, and the second controller has taken over the first resource agent;
[0129] When the second resource agent is in the started state on the first controller and the first resource agent is in the stopped state, it is confirmed that the state of the storage system is an abnormal state; wherein both the first controller and the second controller are in the abnormal state, and the first controller has taken over the second resource agent;
[0130] When the first resource agent and the second resource agent are both in the stopped state, it is confirmed that the state of the storage system is an abnormal state;
[0131] When the first resource agent and the second resource agent are both in the started state on the first controller and the second controller, it is confirmed that the state of the storage system is abnormal; wherein the arbitration system is abnormal and a brain split occurs in the storage system.
[0132] Depend on Figure 1 As can be seen from the illustrated embodiments, the method provided by the embodiments of the present invention can enable the online controller in the controller group to obtain the state of the storage system; determine whether the opposite controller is online according to the state of the storage system; if the opposite controller is online, obtain the disk information in the local controller and the disk information in the opposite controller respectively; determine whether the controller state in the controller group is normal according to the disk information in the local controller and the disk information in the opposite controller; if the opposite controller is not online, determine whether the opposite controller has a brain split; if the opposite end has a brain split, use the arbitration system to arbitrate the opposite controller; determine whether the controller state in the controller group is normal according to the arbitration result; if the opposite end has no brain split, determine that the controller state in the controller group is normal. Therefore, the method provided by the embodiments of the present invention can be implemented to efficiently monitor the state of the controller in a software monitoring manner, thereby improving the stability and reliability of the storage system.
[0133] An example is given below to further illustrate the method provided in the embodiment of the present invention.
[0134] In this embodiment, the name of the HA resource agent is RedStor2, and RedStor2 is used to create resource agent programs for the first controller and the second controller, respectively, wherein the resource agent program corresponding to the first controller is named c1, and the resource agent program corresponding to the second controller is named c2. In this embodiment, in order to ensure that resources and the belonging controller are always bound first, resource location constraints (Location Constraints) are set for the resource agent program, wherein the location constraint score of the resource agent program on the belonging controller is INFINITY (infinity), and the location constraint score on the peer controller is 0. Once the location constraints are set, c1 will run on the first controller first, and c2 will run on the second controller first. The belonging controller refers to the controller bound to the resource agent program.
[0135] The running state of the resource agent is the controller state. In this embodiment, the running state of the resource agent includes starting, started, stopped, stopping and unmanaged states, where:
[0136] Starting state: indicates that the resource agent on the controller is starting. The starting state is a temporary state. If the controller is in this state for longer than the preset time, it indicates that the controller is abnormal and an abnormal state alarm needs to be issued.
[0137] The started state indicates that the resource agent on the controller has been started successfully.
[0138] The stopping state indicates that the resource agent on the controller is stopping. The stopping state is a temporary state. If the controller is in this state for a period longer than the preset time, it indicates that the controller is abnormal and an abnormal state alarm needs to be issued.
[0139] Stopped state; indicates that the resource agent on the controller is stopped successfully, or the resource agent on the controller fails to start on both the first controller and the second controller;
[0140] Unmanaged state: indicates that the resource agent on the controller has entered an abnormal state and the monitoring system needs to issue an abnormal alarm.
[0141] In this embodiment, the status of the storage system is as follows:
[0142] When c1 is in the started state on the first controller and c2 is in the started state on the second controller, the storage system is in a normal state;
[0143] When c1 and c2 are both in the started state on the first controller, the storage system is in a normal state; wherein the second controller is abnormal, and the first controller has successfully taken over c2;
[0144] When c1 and c2 are both in the started state on the second controller, the storage system is in a normal state; the first controller is abnormal, and the second controller has successfully taken over c1;
[0145] When c1 is in the started state on the first controller and c2 is in the stopped state, the storage system is in an abnormal state; wherein the second controller is abnormal and the first controller fails to take over c2;
[0146] When c2 is in the started state on the second controller and c1 is in the stopped state, the storage system is in an abnormal state; wherein the first controller is abnormal and the second controller fails to take over c1;
[0147] When c1 is in the started state on the second controller and c2 is in the stopped state, the storage system is in an abnormal state; wherein both the first controller and the second controller are in an abnormal state, and the second controller has taken over c1;
[0148] When c2 is in the started state on the first controller and c1 is in the stopped state, the storage system is in an abnormal state; wherein both the first controller and the second controller are in an abnormal state, and the first controller has taken over c2;
[0149] When c1 and c2 are both in the stopped state, the storage system is in an abnormal state. The possible reasons for this situation are: HA is exiting the service state, or the resource agent program startup timeout, or monitoring timeout, etc., resulting in continuous errors, and the resource enters an abnormal state. This abnormality requires the intervention of the management personnel to determine the specific cause of the program startup failure.
[0150] When c1 and c2 are both in the started state on the first controller and the second controller, the storage system is in an abnormal state; wherein the arbitration system is abnormal and a split-brain occurs in the storage system. That is, when c1 is in the started state on the first controller and the second controller, and c2 is also in the started state on the first controller and the second controller, it can be confirmed that the storage system is in an abnormal state; and this abnormal state is caused by a split-brain in the storage system and an abnormality in the arbitration system that makes it impossible to arbitrate for the split-brain.
[0151] In this embodiment, once other status scenarios other than the above-listed scenarios of the status of c1 and c2 on the first controller and the second controller are monitored, the method for monitoring the controller status provided in this embodiment will issue an alarm message to request management personnel to intervene and handle.
[0152] See also Figure 2 ,In this embodiment, the process of starting the resource agent program on the controller is as follows;
[0153] S201: enabling a controller in a controller group to obtain disk information in a local controller and disk information of a used disk in the local controller; if the acquisition is successful, executing step S202; if the acquisition fails, confirming that the resource agent to be started fails to start;
[0154] If the controller does not recognize the disk on the local side, or recognizes the disk but the disk is not in use, it indicates that the disk information acquisition fails.
[0155] S202: Obtain identification information of the local controller;
[0156] The purpose of obtaining the identification information of the local controller is to prepare for the next step of determining whether the resource agent belongs to the local controller.
[0157] S203: Determine whether the identification information of the controller corresponding to the resource agent to be started is the same as the identification information of the local controller; if they are the same, execute step S204; if they are different, execute step S206;
[0158] The purpose of this step is to determine whether the resource agent to be started belongs to the local controller. If it belongs to the local controller, further determine whether the disks identified by the local controller completely include the disks that have been used. If not, the program startup fails. If it is completely included, the program startup succeeds. If it belongs to the opposite controller, further obtain the disk information used by the opposite controller. If the acquisition fails, the program startup fails. If the acquisition succeeds, determine whether the disks identified by the local controller completely include the disk information used by the opposite controller. If it is completely included, the program startup succeeds and issues a notification to take over the opposite resource agent. If not, the program startup fails.
[0159] In a specific embodiment, it is possible to determine whether a resource agent belongs to a controller based on the resource agent's identifier, such as its name and the name of the controller. For example, the name of the resource agent of the first controller is xxx1, and the name of the resource agent of the second controller is xxx2. If the name of the resource agent to be started is xxx1, and the name of the controller is the first controller, then it can be determined that the resource agent to be started belongs to the controller.
[0160] S204: Obtain disk information of used disks in the local controller;
[0161] S205: determining whether the disk information in the local controller completely contains the disk information of the disks used by the local controller; if it completely contains, confirming that the resource agent program to be started is successfully started; if it does not completely contain, confirming that the resource agent program to be started fails to start;
[0162] S206: Obtain disk information of used disks in the peer controller;
[0163] S207: Determine whether the disk information in the local controller completely contains the disk information of the disk used by the other controller; if it completely contains, confirm that the resource agent program to be started is started successfully and issue a takeover notification; if it does not completely contain, confirm that the resource agent program to be started fails to start.
[0164] A takeover notification is issued to inform the system that an abnormality has occurred in the peer controller, and the local controller has successfully taken over the resource agent program of the peer controller.
[0165] In this embodiment, the storage system is a dual-controller storage system, which includes: a controller group and a shared disk frame; the controller group includes two controllers, namely: a first controller and a second controller; the shared disk frame includes at least one disk; Figure 3 As shown, the process of the method for monitoring the controller status in this embodiment is as follows:
[0166] S301: An online controller in a controller group obtains a storage system status;
[0167] The state of the storage system includes: the number and identification of online controllers; an online controller is a controller in a working state.
[0168] S302: judging whether the peer controller is online according to the state of the storage system; if the peer controller is online, executing step S303; if the peer controller is not online, executing step S309;
[0169] S303: Obtain disk information in the local controller and disk information in the peer controller respectively;
[0170] S304: Determine whether the disk information in the local controller is consistent with the disk information in the opposite controller; if yes, determine that the controller states in the controller group are all normal; if no, execute step S305;
[0171] S305: Determine the inconsistent disk information between the local controller and the peer controller, and issue a disk asynchronous alarm message;
[0172] S306: Obtain disk information of used disks in the local controller and the peer controller respectively;
[0173] S307: Determine whether the disk information of the used disk in the local controller is consistent with the disk information of the used disk in the opposite controller; if yes, determine that the controller states in the controller group are all normal; if no, execute step S308;
[0174] S308: According to the usage of the disks, determine whether the number of valid disks in the used disks in the local controller and the opposite controller is equal; if they are equal and the local controller is the first controller, determine that the controller states in the controller group are all normal; if they are equal and the local controller is the second controller, determine that the controller states in the controller group are abnormal; if the number of valid disks in the used disks in the local controller is more than that in the opposite controller, determine that the controller states in the controller group are all normal; if the number of valid disks in the used disks in the local controller is less than that in the opposite controller, determine that the controller states in the controller group are abnormal;
[0175] S309: Determine whether a brain split occurs in the peer controller; if yes, execute step S310; if no, determine that the controllers in the controller group are all in normal state;
[0176] S310: using the arbitration system to arbitrate the opposite controller; if the arbitration succeeds, it is determined that the states of the controllers in the controller group are normal; if the arbitration fails, it is determined that the states of the controllers in the controller group are abnormal.
[0177] In summary, by implementing the method provided in the embodiment of the present invention and using the state of the resource agent to represent the state of the controller, the state of the controller can be efficiently monitored in a software monitoring manner, thereby improving the stability and reliability of the storage system.
[0178] and Figure 1 Corresponding to the embodiment shown, the embodiment of the present invention further provides a device for monitoring the state of a controller, the device comprising: an acquisition module 401, a judgment module 402, a first determination module 403, and a second determination module 404, wherein:
[0179] The acquisition module 401 is used to enable the online controllers in the controller group to acquire the state of the storage system; wherein the state of the storage system includes: the number and identification of online controllers; the online controllers are controllers in working state;
[0180] The judgment module 402 is used to judge whether the opposite end controller is online according to the state of the storage system;
[0181] The first determination module 403 is used to obtain disk information in the local controller and disk information in the opposite controller respectively when the opposite controller is online; and determine whether the controller status in the controller group is normal according to the disk information in the local controller and the disk information in the opposite controller;
[0182] The second determination module 404 is used to determine whether a brain split occurs in the controller at the other end when the controller at the other end is not online; if a brain split occurs in the other end, arbitrate the controller at the other end using the arbitration system; determine whether the status of the controllers in the controller group is normal based on the arbitration result; if a brain split does not occur in the other end, determine that the status of the controllers in the controller group are all normal.
[0183] Optionally, the first determining module 403 is specifically configured to:
[0184] Determine whether the disk information in the local controller is consistent with the disk information in the peer controller;
[0185] If yes, then determine that the controller statuses in the controller group are all normal;
[0186] If not, determine the inconsistent disk information in the local controller and the opposite controller, and issue a disk asynchronous alarm message; obtain the disk information of the used disks in the local controller and the opposite controller respectively; determine whether the disk information of the used disks in the local controller and the opposite controller are consistent; if yes, determine that the controller states in the controller group are all normal; if not, determine whether the number of valid disks in the used disks in the local controller and the opposite controller are equal according to the purpose of the disk; if they are equal and the local controller is the first controller, determine that the controller states in the controller group are all normal; if they are equal and the local controller is the second controller, determine that the controller states in the controller group are abnormal; if the number of valid disks in the used disks in the local controller is more than that in the opposite controller, determine that the controller states in the controller group are all normal; if the number of valid disks in the used disks in the local controller is less than that in the opposite controller, determine that the controller states in the controller group are abnormal.
[0187] Optionally, the second determining module 404 is specifically configured to:
[0188] When arbitration succeeds, it is determined that the controllers in the controller group are all in normal state;
[0189] When arbitration fails, it is determined that the status of the controllers in the controller group is abnormal.
[0190] Optionally, the first controller and the second controller both include the following states:
[0191] Starting state: indicates that the resource agent on the controller is starting. The starting state is a temporary state. If the controller is in this state for longer than the preset time, it indicates that the controller is abnormal and an abnormal state alarm needs to be issued.
[0192] The started state indicates that the resource agent on the controller has been started successfully.
[0193] The stopping state indicates that the resource agent on the controller is stopping. The stopping state is a temporary state. If the controller is in this state for a period longer than the preset time, it indicates that the controller is abnormal and an abnormal state alarm needs to be issued.
[0194] Stopped state; indicates that the resource agent on the controller is stopped successfully, or the resource agent on the controller fails to start on both the first controller and the second controller;
[0195] Unmanaged state: indicates that the resource agent on the controller has entered an abnormal state and the monitoring system needs to issue an abnormal alarm.
[0196] Optionally, the state of the storage system is determined by the states of the first controller and the second controller; wherein the resource agent corresponding to the first controller is the first resource agent, and the resource agent corresponding to the second controller is the second resource agent;
[0197] When the first resource agent is in the started state on the first controller, and the second resource agent is in the started state on the second controller, it is confirmed that the state of the storage system is normal;
[0198] When both the first resource agent and the second resource agent are in the started state on the first controller, it is confirmed that the storage system is in a normal state; wherein the second controller is abnormal, and the first controller has successfully taken over the second resource agent;
[0199] When both the first resource agent and the second resource agent are in the started state on the second controller, it is confirmed that the state of the storage system is normal; wherein the first controller is abnormal and the second controller has successfully taken over the first resource agent;
[0200] When the first resource agent is in the started state on the first controller and the second resource agent is in the stopped state, it is confirmed that the state of the storage system is abnormal; wherein the second controller is abnormal and the first controller fails to take over the second resource agent;
[0201] When the second resource agent is in the started state on the second controller and the first resource agent is in the stopped state, it is confirmed that the state of the storage system is abnormal; wherein the first controller is abnormal and the second controller fails to take over the first resource agent;
[0202] When the first resource agent is in the started state on the second controller and the second resource agent is in the stopped state, it is confirmed that the state of the storage system is an abnormal state; wherein both the first controller and the second controller are in the abnormal state, and the second controller has taken over the first resource agent;
[0203] When the second resource agent is in the started state on the first controller and the first resource agent is in the stopped state, it is confirmed that the state of the storage system is an abnormal state; wherein both the first controller and the second controller are in the abnormal state, and the first controller has taken over the second resource agent;
[0204] When the first resource agent and the second resource agent are both in the stopped state, it is confirmed that the state of the storage system is an abnormal state;
[0205] When the first resource agent and the second resource agent are both in the started state on the first controller and the second controller, it is confirmed that the state of the storage system is abnormal; wherein the arbitration system is abnormal and a brain split occurs in the storage system.
[0206] Optionally, the device further includes: a program startup module, configured to:
[0207] Enable the controller in the controller group to obtain disk information in the local controller and disk information of the used disk in the local controller;
[0208] If the acquisition is successful, the identification information of the local controller is obtained; whether the identification letter of the controller corresponding to the resource agent program to be started is the same as the identification information of the local controller; if the identification letter of the controller corresponding to the resource agent program to be started is the same as the identification information of the local controller, the disk information of the used disks in the local controller is obtained; whether the disk information in the local controller completely contains the disk information of the used disks of the local controller; if it is completely contained, it is confirmed that the resource agent program to be started is started successfully; if it is not completely contained, it is confirmed that the resource agent program to be started fails to start; if the identification letter of the controller corresponding to the resource agent program to be started is not the same as the identification information of the local controller, the disk information of the used disks in the opposite controller is obtained; whether the disk information in the local controller completely contains the disk information of the used disks of the opposite controller; if it is completely contained, it is confirmed that the resource agent program to be started is started successfully and a takeover notification is issued; if it is not completely contained, it is confirmed that the resource agent program to be started fails to start;
[0209] If the acquisition fails, it is confirmed that the resource agent to be started fails to start.
[0210] Optionally, the device further includes: a program stopping module, configured to:
[0211] Use the stop method to stop the resource agent on the controller; the stop method is a no-operation method.
[0212] By implementing the device provided by the embodiment of the present invention, the state of the resource agent program is used to represent the state of the controller, so that the state of the controller can be efficiently monitored in a software monitoring manner, thereby improving the stability and reliability of the storage system.
[0213] and Figure 1 Corresponding to the embodiment shown, the embodiment of the present invention further provides an electronic device, see Figure 5 , including a processor 501, a communication interface 502, a memory 503 and a communication bus 504, wherein the processor 501, the communication interface 502, and the memory 503 communicate with each other through the communication bus 504;
[0214] Memory 503, used for storing computer programs;
[0215] The processor 501 is used to implement any method step of monitoring the controller status described in the above embodiments when executing the program stored in the memory.
[0216] By implementing the electronic device provided by the embodiment of the present invention, the state of the resource agent program is used to represent the state of the controller, so that the state of the controller can be efficiently monitored in a software monitoring manner, thereby improving the stability and reliability of the storage system.
[0217] and Figure 1 Corresponding to the embodiment shown, an embodiment of the present invention further provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the method steps for monitoring the controller status described in the above embodiment are implemented.
[0218] By implementing the storage medium provided in the embodiment of the present invention and using the state of the resource agent to represent the state of the controller, the state of the controller can be efficiently monitored in a software monitoring manner, thereby improving the stability and reliability of the storage system.
[0219] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) containing computer-usable program code.
[0220] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0221] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0222] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0223] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.
Claims
1. A method for monitoring the state of a controller, characterized in that: Applied to a storage system, the storage system includes: a controller group and a shared disk frame; the controller group includes: a first controller and a second controller; the shared disk frame includes at least one disk; the method includes: Enable an online controller in the controller group to obtain the state of the storage system; wherein the state of the storage system includes: the number and identification of online controllers; the online controller is a controller in a working state; Determining whether the opposite controller is online according to the state of the storage system; If the opposite controller is online, respectively obtain disk information in the local controller and disk information in the opposite controller; determine whether the controller status in the controller group is normal according to the disk information in the local controller and the disk information in the opposite controller; If the peer controller is not online, determine whether the peer controller has a brain split; if the peer controller has a brain split, use the arbitration system to arbitrate the peer controller; determine whether the controller status in the controller group is normal according to the arbitration result; if the peer controller has no brain split, determine that the controller status in the controller group is normal; The step of determining whether the controller status in the controller group is normal according to the disk information in the local controller and the disk information in the opposite controller includes: Determine whether the disk information in the local controller is consistent with the disk information in the peer controller; If yes, determining that the controller states in the controller group are all normal; If not, determine the inconsistent disk information in the local controller and the opposite controller, and issue a disk asynchronous alarm message; obtain the disk information of the used disks in the local controller and the opposite controller respectively; determine whether the disk information of the used disks in the local controller is consistent with the disk information of the used disks in the opposite controller; if yes, determine that the controller states in the controller group are all normal; if not, determine whether the number of valid disks in the used disks in the local controller and the opposite controller is equal according to the purpose of the disk; if they are equal and the local controller is the first controller, determine that the controller states in the controller group are all normal; if they are equal and the local controller is the second controller, determine that the controller states in the controller group are abnormal; if the number of valid disks in the used disks in the local controller is more than that in the opposite controller, determine that the controller states in the controller group are all normal; if the number of valid disks in the used disks in the local controller is less than that in the opposite controller, determine that the controller states in the controller group are abnormal.
2. The method according to claim 1, characterized in that The step of determining whether the controller status in the controller group is normal according to the arbitration result includes: If the arbitration is successful, determining that the states of the controllers in the controller group are all normal; If arbitration fails, it is determined that the states of controllers in the controller group are abnormal.
3. The method according to claim 1, characterized in that The first controller and the second controller both include the following states: starting state; indicating that the resource agent on the controller is starting; wherein the starting state is a temporary state. If the controller is in this state for a period longer than a preset period, it indicates that the controller state is abnormal and an abnormal state alarm needs to be issued; The started state indicates that the resource agent on the controller has been started successfully. The stopping state indicates that the resource agent on the controller is stopping. The stopping state is a temporary state. If the controller is in this state for a period longer than a preset period, it indicates that the controller is abnormal and an abnormal state alarm needs to be issued. The stopped state indicates that the resource agent on the controller has been successfully stopped, or the resource agent on the controller has failed to start on both the first controller and the second controller; Unmanaged state: indicates that the resource agent on the controller has entered an abnormal state and the monitoring system needs to issue an abnormal alarm.
4. The method according to claim 3, characterized in that The state of the storage system is determined by the states of the first controller and the second controller; wherein the resource agent corresponding to the first controller is the first resource agent, and the resource agent corresponding to the second controller is the second resource agent; When the first resource agent is in the started state on the first controller, and the second resource agent is in the started state on the second controller, it is confirmed that the state of the storage system is normal; When both the first resource agent and the second resource agent are in the started state on the first controller, it is confirmed that the state of the storage system is normal; wherein the second controller is abnormal, and the first controller has successfully taken over the second resource agent; When both the first resource agent and the second resource agent are in the started state on the second controller, it is confirmed that the state of the storage system is normal; wherein the first controller is abnormal and the second controller has successfully taken over the first resource agent; When the first resource agent is in the started state on the first controller and the second resource agent is in the stopped state, it is confirmed that the state of the storage system is abnormal; wherein the second controller is abnormal and the first controller fails to take over the second resource agent; When the second resource agent is in the started state on the second controller and the first resource agent is in the stopped state, it is confirmed that the state of the storage system is abnormal; wherein the first controller is abnormal and the second controller fails to take over the first resource agent; When the first resource agent is in the started state on the second controller and the second resource agent is in the stopped state, it is confirmed that the state of the storage system is an abnormal state; wherein both the first controller and the second controller are in the abnormal state, and the second controller has taken over the first resource agent; When the second resource agent is in the started state on the first controller and the first resource agent is in the stopped state, it is confirmed that the state of the storage system is an abnormal state; wherein both the first controller and the second controller are in the abnormal state, and the first controller has taken over the second resource agent; When both the first resource agent and the second resource agent are in the stopped state, it is confirmed that the state of the storage system is an abnormal state; When the first resource agent and the second resource agent are both in the started state on the first controller and the second controller, it is confirmed that the state of the storage system is abnormal; wherein the arbitration system is abnormal and a brain split occurs in the storage system.
5. The method according to claim 3, characterized in that: The process of starting the resource agent on the controller is as follows: Enable the controller in the controller group to obtain disk information in the local controller and disk information of used disks in the local controller; If the acquisition is successful, the identification information of the local controller is obtained; whether the identification letter of the controller corresponding to the resource agent program to be started is the same as the identification information of the local controller; if the identification letter of the controller corresponding to the resource agent program to be started is the same as the identification information of the local controller, the disk information of the used disks in the local controller is obtained; whether the disk information in the local controller completely contains the disk information of the used disks of the local controller; if it is completely contained, it is confirmed that the resource agent program to be started is started successfully; if it is not completely contained, it is confirmed that the resource agent program to be started fails to start; if the identification letter of the controller corresponding to the resource agent program to be started is not the same as the identification information of the local controller, the disk information of the used disks in the opposite controller is obtained; whether the disk information in the local controller completely contains the disk information of the used disks of the opposite controller; if it is completely contained, it is confirmed that the resource agent program to be started is started successfully and a takeover notification is issued; if it is not completely contained, it is confirmed that the resource agent program to be started fails to start; If the acquisition fails, it is confirmed that the resource agent program to be started fails to start.
6. The method according to claim 3, characterized in that The resource agent on the controller is stopped in the following ways: The resource agent on the controller is stopped by using the stop method; wherein the stop method is a no-operation method.
7. A device for monitoring the state of a controller, characterized in that: The device comprises: An acquisition module is used to enable an online controller in the controller group to acquire a state of the storage system; wherein the state of the storage system includes: the number and identification of online controllers; the online controller is a controller in a working state; A judgment module, used to judge whether the opposite end controller is online according to the state of the storage system; The first determination module is used to obtain the disk information in the local controller and the disk information in the opposite controller respectively when the opposite controller is online; and determine whether the controller status in the controller group is normal according to the disk information in the local controller and the disk information in the opposite controller; The second determination module is used to determine whether the peer controller has a brain split when the peer controller is not online; if the peer controller has a brain split, arbitrate the peer controller using the arbitration system; determine whether the controller status in the controller group is normal according to the arbitration result; if the peer controller has no brain split, determine that the controller status in the controller group is normal; The step of determining whether the controller status in the controller group is normal according to the disk information in the local controller and the disk information in the opposite controller includes: Determine whether the disk information in the local controller is consistent with the disk information in the peer controller; If yes, determining that the controller states in the controller group are all normal; If not, determine the inconsistent disk information in the local controller and the opposite controller, and issue a disk asynchronous alarm message; obtain the disk information of the used disks in the local controller and the opposite controller respectively; judge whether the disk information of the used disks in the local controller is consistent with the disk information of the used disks in the opposite controller; if yes, determine that the controller states in the controller group are all normal; if not, judge whether the number of valid disks in the used disks in the local controller and the opposite controller is equal according to the purpose of the disk; if they are equal and the local controller is the first controller, determine that the controller states in the controller group are all normal; if they are equal and the local controller is the second controller, determine that the controller states in the controller group are abnormal; if the number of valid disks in the used disks in the local controller is more than that in the opposite controller, determine that the controller states in the controller group are all normal; if the number of valid disks in the used disks in the local controller is less than that in the opposite controller, determine that the controller states in the controller group are abnormal.
8. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; The memory is used to store computer programs; The processor is used to implement the method steps described in any one of claims 1 to 6 when executing the program stored in the memory.
9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method steps described in any one of claims 1 to 6 are implemented.
Citation Information
Patent Citations
Highly available double-control storage system and operation method thereof based on quorum disc
CN105095125A