A service recovery method, device, arbitration server, and storage system

Abstract

A service recovery method, apparatus, arbitration server, and storage system. The method includes: after the storage system switches from a dual-slave state to a fault recovery state, a first storage array first sends a first arbitration request to the arbitration server. When the arbitration server receives the first arbitration request sent by the first storage array, it determines the first arbitration result of the first storage array and the second arbitration result of the second storage array. After determining the first arbitration result and the second arbitration result, it sends the first arbitration result to the first storage array and the second arbitration result to the second storage array. After receiving the first arbitration result, the first storage array initiates negotiation with the second storage array to determine the negotiation result. If it is determined that the negotiation result is that the first storage array is in the winning state and the second storage array is in the losing state, the first storage array provides service to the client.

Description

Technical Field

[0001] This application relates to the field of storage technology, and in particular, to a service recovery method, apparatus, arbitration server, and storage system. Background Art

[0002] With the advent of the Internet era, the business requirements of all walks of life are increasing. To ensure the reliability of services, a dual-active storage system is adopted to provide cloud computing services.

[0003] Please refer to Figure 1 , which is a schematic structural diagram of a dual-active storage system. As Figure 1 shown, the dual-active storage system includes an arbitration server and two mutually backup storage arrays (Storage Array A and Storage Array B respectively). When the dual-active storage system is running normally, both storage arrays are in the running state, and services are provided by the two storage arrays simultaneously. When Storage Array A loses power or encounters a device failure and cannot process services, the arbitration server arbitrates the states of Storage Array A and Storage Array B, so that the services originally distributed on Storage Array A are smoothly taken over by Storage Array B, and the working mode of the dual-active storage system changes from providing services by Storage Array A and Storage Array B simultaneously to providing services independently by Storage Array B. In this way, higher data reliability and service continuity can be provided to users.

[0004] However, due to the limitations of the networking environment, during the use of the dual-active storage system, a dual-slave state where both storage arrays in the dual-active storage system cannot process services often occurs due to reasons such as network failures and power supply failures. After the dual-active storage system is in the dual-slave state, even if the failure is recovered, it is necessary to manually analyze the logs of each storage array, and then select one of the storage arrays to resume services. The time required for manual log analysis is relatively long, which often affects the timely recovery of services. Summary of the Invention

[0005] Embodiments of this application provide a service recovery method, apparatus, arbitration server, and storage system, which are used to improve the service recovery efficiency of the dual-active storage system.

[0006] In a first aspect, an embodiment of the present application provides a service recovery method, which is applied to a storage system. The storage system includes a first storage array, a second storage array, and an arbitration server. After the storage system switches from a dual-slave state to a fault recovery state, the first storage array first sends a first arbitration request to the arbitration server. When the arbitration server receives the first arbitration request sent by the first storage array, it determines the first arbitration result of the first storage array and the second arbitration result of the second storage array. After determining the first arbitration result and the second arbitration result, it sends the first arbitration result to the first storage array and the second arbitration result to the second storage array. After receiving the first arbitration result, the first storage array initiates negotiation with the second storage array to determine the negotiation result. If it is determined that the negotiation result is that the first storage array is in a winning state and the second storage array is in a losing state, the first storage array provides service to the client.

[0007] In the above technical solution, after the storage system changes from a dual-slave state to a fault recovery state, the negotiation process can be actively sent by the first storage array to automatically determine the winning storage array, and the winning storage array actively pulls up the service, thereby reducing the waiting time for manually pulling up the service and improving the service recovery efficiency of the storage system.

[0008] In a possible design, after receiving the first arbitration result, the first storage array initiates negotiation with the second storage array. When the second storage array receives the negotiation request, it feeds back the arbitration result of the arbitration server for the second storage array, that is, the second arbitration result, to the first storage array. After the first storage array receives the second arbitration result from the second storage array, it determines the negotiation result according to the first arbitration result and the second arbitration result.

[0009] In the above technical solution, the first storage array can initiate negotiation according to the arbitration result sent by the arbitration server, which can improve the accuracy of the negotiation result.

[0010] In a possible design, the first storage array can determine the negotiation result according to the following negotiation principles, and the negotiation principles can include but are not limited to the following situations:

[0011] If the first arbitration result is in a winning state and the second arbitration result is in a checking state, the first storage array determines that the first storage array is in a winning state and the second storage array is in a losing state; or

[0012] If the first arbitration result is in the winning state and the second arbitration result is in the losing state, the first storage array determines that the first storage array is in the winning state and the second storage array is in the losing state; or

[0013] If the first arbitration result is in the losing state and the second arbitration result is in the winning state, the first storage array determines that the first storage array is in the losing state and the second storage array is in the winning state; or

[0014] If the first arbitration result is in the checking state and the second arbitration result is in the checking state, the first storage array determines that the first storage array is in the winning state and the second storage array is in the losing state.

[0015] In the above technical solution, by presetting the above-mentioned multiple negotiation principles, the flexibility of the storage system during the service recovery process can be increased.

[0016] In a possible design, after the first storage array provides service to the client, the first storage array sends synchronization data to the second storage array.

[0017] After the first storage array provides the service, the latest data of the service is sent to the second storage array, so that the service data in the second storage array is synchronized with the first storage array. After all the differential data of the first storage array is synchronized to the second storage array, the dual-active feature of the storage system is restored, and thus the first storage array and the second storage array can provide services simultaneously.

[0018] In a possible design, if the replication link between the first storage array and the second storage array is disconnected and before entering the dual-slave state, the arbitration server has determined and stored the arbitration result based on the third arbitration request sent by the first storage array and / or the fourth arbitration request sent by the second storage array, and has not fed back to the first storage array and the second storage array, then, after receiving the first arbitration request sent by the first storage array, the arbitration server directly determines the first arbitration result and the second arbitration result from the stored arbitration result.

[0019] In the above technical solution, if the arbitration server has stored the arbitration results of the first storage array and the second storage array before the storage system becomes dual-slave, the arbitration server can directly send the stored arbitration results to the first storage array and the second storage array, which can reduce the energy consumption of the arbitration server.

[0020] In a possible design, the arbitration results stored in the arbitration server include, but are not limited to, the following situations:

[0021] The first arbitration result is to check the winning state and the second arbitration result is the losing state; or

[0022] The first arbitration result is an unknown winning state and the second arbitration result is the checking state; or

[0023] The first arbitration result is to check the losing state and the second arbitration result is the winning state;

[0024] The first arbitration result is to check the checking state and the second arbitration result is the checking state.

[0025] In a possible design, if the arbitration server receives a second arbitration request from the second storage array before determining the first arbitration result of the first storage array and the second arbitration result of the second storage array, the arbitration server determines the first arbitration result and the second arbitration result according to the order of receiving the first arbitration request and the second arbitration request. For example, if the arbitration server receives the first arbitration request first and then receives the second arbitration request, the arbitration server determines that the first arbitration result is the winning state and the second arbitration result is the losing state.

[0026] In the above technical solution, the arbitration server can also determine the arbitration result according to the order of receiving the arbitration requests of the two storage arrays. For example, the storage array corresponding to the arbitration request received first is determined to be in the winning state, while the other storage array is determined to be in the losing state, which can improve the flexibility of the arbitration server.

[0027] In a possible design, if the arbitration server determines that it does not receive a second arbitration request from the second storage array within a preset duration starting from receiving the first arbitration request, the arbitration server determines that the first arbitration result is the winning state and the second arbitration result is the checking state.

[0028] In the above technical solution, if the arbitration server does not receive an arbitration request from a certain storage array, it can be considered that the storage array has failed. Thus, the storage array corresponding to the received arbitration request can be set to the winning state, and the storage array that does not receive the arbitration request can be set to the checking state, which can improve the flexibility of the arbitration server.

[0029] Second aspect, an embodiment of the present application provides a service recovery device, which is applied to a first storage array. The first storage array is located in a storage system, and the storage system further includes a second storage array and an arbitration server. The device includes a processor for implementing the method executed by the first storage array in the method described in the first aspect above. The device may further include a memory for storing program instructions and data. The memory is coupled to the processor, and the processor can call and execute the program instructions stored in the memory to implement the method executed by the first storage array in the method described in the first aspect above. The device may further include a communication interface for the device to communicate with other devices. Exemplarily, the other devices include the second storage array or the arbitration server mentioned in the first aspect above.

[0030] In a possible design, the device includes a communication interface and a processor. Specifically, after the storage system switches from a dual-slave state to a failure recovery state, the processor sends a first arbitration request to the arbitration server through the communication interface, receives a first arbitration result from the arbitration server through the communication interface, and according to the first arbitration result, initiates a negotiation with the second storage array through the communication interface to determine the negotiation result. If it is determined that the negotiation result is that the first storage array is in a winning state and the second storage array is in a losing state, the processor provides service to the client through the communication interface.

[0031] In a possible design, the processor initiates a negotiation with the second storage array through the communication interface according to the first arbitration result, and the communication interface receives a second arbitration result from the second storage array. The second arbitration result is the arbitration result of the arbitration server for the second storage array. The negotiation result is determined according to the first arbitration result and the second arbitration result.

[0032] In a possible design, if the first arbitration result is in the winning state, the second arbitration result is in the checking state, the processor determines that the storage array is in the winning state and the second storage array is in the losing state; or, if the first arbitration result is in the winning state and the second arbitration result is in the losing state, the processor determines that the storage array is in the winning state and the second storage array is in the losing state; or, if the first arbitration result is in the losing state and the second arbitration result is in the winning state, the processor determines that the storage array is in the losing state and the second storage array is in the winning state; or, if the first arbitration result is in the checking state and the second arbitration result is in the checking state, the processor determines that the storage array is in the winning state and the second storage array is in the losing state.

[0033] In a possible design, after the processor provides a service to a client through the communication interface, the processor is further configured to send synchronization data to the second storage array.

[0034] In a third aspect, an arbitration server provided by an embodiment of the present application is located in a storage system. The storage system further includes a first storage array and a second storage array. The arbitration server includes a processor configured to implement the method executed by the arbitration server in the method described in the first aspect above. The arbitration server may further include a memory configured to store program instructions and data. The memory is coupled to the processor, and the processor may call and execute the program instructions stored in the memory to implement the method executed by the arbitration server in the method described in the first aspect above. The arbitration server may further include a communication interface configured to communicate the arbitration server with other devices. Exemplarily, the other devices include the first storage array or the second storage array mentioned in the first aspect above.

[0035] In a possible design, the arbitration server includes a processor and a communication interface. Specifically, after the storage system switches from the dual-slave state to the fault recovery state, the processor receives a first arbitration request from the first storage array through the communication interface. The processor determines the first arbitration result of the first storage array and the second arbitration result of the second storage array. The processor sends the first arbitration result to the first storage array through the communication interface, and sends the second arbitration result to the second storage array through the communication interface.

[0036] In a possible design, the processor determines the first arbitration result and the second arbitration result from the stored arbitration results. The stored arbitration results are the arbitration results determined by the arbitration server before the replication link of the storage system is disconnected and the system is in the dual-slave state, and have not been fed back to the first storage array and the second storage array. The replication link is the link between the first storage array and the second storage array;

[0037] Before the replication link of the storage system is disconnected and the system is in the dual-slave state, the processor determines the stored arbitration results according to the third arbitration request sent by the first storage array received through the communication interface and / or the fourth arbitration request sent by the second storage array received through the communication interface.

[0038] In a possible design, the stored arbitration results include:

[0039] The first arbitration result is to check the winning state and the second arbitration result is the losing state; or

[0040] The first arbitration result is an unknown winning state and the second arbitration result is the checking state; or

[0041] The first arbitration result is to check the losing state and the second arbitration result is the winning state;

[0042] The first arbitration result is to check the checking state and the second arbitration result is the checking state.

[0043] In a possible design, before the processor determines the first arbitration result of the first storage array and the second arbitration result of the second storage array, a second arbitration request is received from the second storage array through the communication interface. The processor determines the first arbitration result and the second arbitration result according to the order of receiving the first arbitration request and the second arbitration request. If the processor receives the second arbitration request after receiving the first arbitration request, it determines that the first arbitration result is the winning state and the second arbitration result is the losing state.

[0044] In a possible design, if the processor determines that no second arbitration request is received from the second storage array within a preset duration starting from the receipt of the first arbitration request through the communication interface, it determines that the first arbitration result is the winning state and the second arbitration result is the checking state.

[0045] Fourthly, an embodiment of the present application provides a service recovery device, which is applied to a first storage array. The first storage array is located in a storage system, and the storage system further includes a second storage array and an arbitration server. The device may include a transceiver unit and a processing unit. These modules may perform the corresponding functions of the first storage array in any design example of the first aspect above, and these modules may be implemented by software modules or by corresponding hardware entities. For example, when implemented by corresponding hardware entities, the function of the transceiver unit is similar to the function of the communication interface in the second aspect above, and the function of the processing unit is similar to the function of the processor in the second aspect above.

[0046] Fifthly, an embodiment of the present application provides an arbitration server, which may include a transceiver unit and a processing unit. These modules may perform the corresponding functions of the arbitration server in any design example of the first aspect above, and these modules may be implemented by software modules or by corresponding hardware entities. For example, when implemented by corresponding hardware entities, the function of the transceiver unit is similar to the function of the communication interface in the third aspect above, and the function of the processing unit is similar to the function of the processor in the third aspect above.

[0047] Sixthly, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. The computer program includes program instructions. When the program instructions are executed by a computer, the computer is caused to execute any one of the methods performed by the first storage array in the first aspect.

[0048] Seventhly, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. The computer program includes program instructions. When the program instructions are executed by a computer, the computer is caused to execute any one of the methods performed by the arbitration server in the first aspect.

[0049] Eighthly, an embodiment of the present application provides a computer program product, which stores a computer program. The computer program includes program instructions. When the program instructions are executed by a computer, the computer is caused to execute any one of the methods performed by the first storage array in the first aspect.

[0050] Ninthly, an embodiment of the present application provides a computer program product, which stores a computer program. The computer program includes program instructions. When the program instructions are executed by a computer, the computer is caused to execute any one of the methods performed by the arbitration server in the first aspect.

[0051] Tenth aspect, the present application provides a chip system, which includes a processor and may further include a memory for implementing the method executed by the first storage array or the method executed by the arbitration server in the first aspect. The chip system may be composed of chips or may include chips and other discrete devices.

[0052] Eleventh aspect, the present application provides a storage system, which includes the service recovery device in the second aspect or the fourth aspect and the arbitration server in the third aspect or the fifth aspect.

[0053] Twelfth aspect, the present application provides a storage system, which includes the service recovery device in the second aspect or the fourth aspect, the arbitration server in the third aspect or the fifth aspect, and a second storage array.

[0054] For the beneficial effects of the second aspect to the twelfth aspect and their implementation manners, reference may be made to the description of the beneficial effects of the method and its implementation manner in the first aspect. Description of the Drawings

[0055] Figure 1 It is a structural diagram of a dual-active storage system provided by an embodiment of the present application;

[0056] Figure 2 It is a structural diagram of another dual-active storage system provided by an embodiment of the present application;

[0057] Figure 3A It is a structural diagram of another dual-active storage system provided by an embodiment of the present application;

[0058] Figure 3B It is a structural diagram of another dual-active storage system provided by an embodiment of the present application;

[0059] Figure 4 It is a flowchart of an example of the service recovery method provided by an embodiment of the present application;

[0060] Figure 5 It is a schematic diagram of the state change process of the storage array in an embodiment of the present application;

[0061] Figure 6 It is a flowchart of another example of the service recovery method provided by an embodiment of the present application;

[0062] Figure 7 It is a structural schematic diagram of a service recovery device provided by an embodiment of the present application;

[0063] Figure 8 It is a structural schematic diagram of another service recovery device provided by an embodiment of the present application;

[0064] Figure 9Schematic diagram of an arbitration server provided by an embodiment of the present application;

[0065] Figure 10 Another schematic diagram of an arbitration server provided by an embodiment of the present application. Detailed implementation manners

[0066] In order to make the objectives, technical solutions, and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described in detail below with reference to the accompanying drawings of the specification and specific implementation manners.

[0067] The term "and / or" in this document is merely a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone. In addition, the character " / " in this document generally represents an "or" relationship between the associated objects before and after, unless otherwise specified.

[0068] In addition, it should be understood that the "multiple" involved in the embodiments of the present application refers to two or more than two. Terms such as "first" and "second" are only used for the purpose of distinguishing descriptions, and cannot be understood as indicating or implying relative importance, nor can they be understood as indicating or implying an order.

[0069] The embodiments of the present application provide a service recovery method, which is applied to a dual-active storage system. The dual-active storage system can be a file storage system, a block storage system, or an object storage system, or a combination of the above storage systems, which is not limited in the embodiments of the present application.

[0070] Please refer to Figures 1 to 3B , which are four possible architecture diagrams of the dual-active storage system in the embodiments of the present application. As Figure 1 shown, the dual-active storage system has been described above and will not be elaborated here. Different from the dual-active storage system shown in Figure 1 , in the dual-active storage system shown in Figure 2 , each storage array is a coupled node set composed of multiple storage nodes, which cooperate to provide services externally. As Figure 2 shown, the storage array A of the dual-active storage system includes storage nodes 0 to 2, and the storage array B includes storage nodes 3 to 5.

[0071] It should be noted that in the dual-active storage system shown in Figure 1 or Figure 2 , the storage array A and the storage array B can be located in the same region, as Figure 3AAs shown, storage array A and storage array B are located in the same region. Storage array A and storage array B can be connected through a high-speed network to ensure low latency. Storage array A and storage array B can also be located in different regions. For example, Figure 3B as shown, storage array A is located in region 1 and storage array B is located in region 2. The networks between regions support wired connections such as fiber channel (FC) and internet small computer system interface (iSCSI).

[0072] It should be noted that the active-active storage system is not limited to the architecture as Figures 1 to 3B shown. The active-active storage system described in the embodiments of this application is to more clearly illustrate the technical solutions of the embodiments of this application, and does not constitute a limitation on the technical solutions provided by the embodiments of this application. Those of ordinary skill in the art know that with the evolution of storage technology and storage system architecture, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0073] Hereinafter, terms related to the storage system will be described to facilitate understanding by those skilled in the art.

[0074] (1) Replication link, which represents the link between two storage arrays in the active-active storage system.

[0075] (2) Arbitration link, which represents the links between two storage arrays in the active-active storage system and the arbitration server respectively. For example, the link between storage array A and the arbitration server and the link between storage array B and the arbitration server.

[0076] (3) Active-active means that two storage arrays are backup to each other and both are in the running state, and can process the same business simultaneously. When a storage array in the active-active storage system fails, the business quickly switches to the other storage array to ensure business continuity.

[0077] (4) Passive-passive means that neither of the two storage arrays can process the business. When the business in the active-active storage system cannot be switched to any storage array, the business is interrupted. The reasons for the active-active storage system to change to the passive-passive state include but are not limited to the following three: a) Both arrays of the active-active storage system lose power at the same time; b) Both arrays of the active-active storage system and the arbitration server lose power at the same time; c) The replication link and the arbitration link of the active-active storage system fail at the same time.

[0078] (5) Consistency group, which is composed of logical storage units (LUNs) or files on two storage arrays that are backups of each other in the active-active storage system. In the two storage arrays that make up the consistency group, a preferred array and a non-preferred array are set. For example, storage array A is the preferred array and storage array B is the non-preferred array.

[0079] (6) Arbitration server. When the replication link is disconnected, the arbitration server arbitrates in units of the active-active consistency group. The arbitration server can be an independent device, which can be a computer or a mobile terminal, etc., or it can be a logical concept, such as a software module, or a virtual machine under virtualization implementation, which is not limited here.

[0080] Next, taking Figure 1 the above-mentioned active-active storage system, and taking the case where the reason for the active-active storage system to become active-standby is that both the replication link and the arbitration link fail at the same time, the business recovery process in the prior art will be introduced.

[0081] Step 1. When the active-active storage system is in normal use, the two storage arrays provide services at the same time. For example, it is set that storage array A processes a part of tasks and storage array B processes another part of services. When storage array A or storage array B processes services, it will send synchronization data to the other party in real time to synchronize the data in the two storage arrays. For example, when storage array A processes service 1, storage array A will send the data of service 1 to storage array B, or when storage array B processes service 2, storage array B will also send the data of service 2 to storage array A.

[0082] In this case, since there is no need for the arbitration server to arbitrate, the status of storage array A and storage array B in the arbitration server is the unknown state.

[0083] Step 2. When the computer room of the active-active storage system loses power, or storage array A, storage array B, and the arbitration server lose power at the same time, or after the replication link fails, the arbitration link between storage array A and the arbitration server and the arbitration link between storage array B and the arbitration server also fail, etc. In this case, the arbitration server will be unable to arbitrate the storage arrays. Storage array A and storage array B determine that the arbitration times out, so both storage array A and storage array B become the losing state, and thus the active-active storage system becomes the active-standby state.

[0084] Step 3. After the failures of the replication link and the arbitration link are recovered, to ensure the continuity of services, it is necessary to select the storage array storing the latest data corresponding to the service to continue processing the service. Therefore, technicians need to analyze the logs of the two storage arrays to determine the time points when each storage array becomes the losing state. Since the time when storage array A becomes the losing state is later than that of storage array B, the technicians select storage array A to resume the service. After receiving the instruction to resume the service, storage array A continues to process the service.

[0085] Since it takes a long time for technicians to analyze the logs of the storage arrays manually, the service recovery method in the prior art has the problem of affecting the timely recovery of services.

[0086] In view of this, the embodiments of the present application provide a service recovery method to improve the service recovery efficiency of a dual-active storage system. The technical solutions provided by the embodiments of the present application are introduced below with reference to the accompanying drawings.

[0087] Please refer to Figure 4 , which is a flowchart of the service recovery method provided by the embodiments of the present application. The description of this process is as follows:

[0088] Step 401. The storage system runs normally, and the storage arrays process services.

[0089] In the embodiments of the present application, the storage system includes a first storage array, a second storage array, and an arbitration server. It can be understood that the storage system is a dual-active storage system, specifically one of the storage systems as shown in Figures 1 to 3B . For ease of explanation, in the following introduction, the storage system shown in Figure 1 will be used as an example for illustration. That is to say, the first storage array is the storage array A in Figure 1 , the second storage array is the storage array B in Figure 1 , and the arbitration server is the arbitration server in Figure 1 .

[0090] When the storage system runs normally, the storage array A and the storage array B process services together. The specific process is the same as step 1 in the prior art and will not be elaborated here.

[0091] It should be noted that since the arbitration server is not required for arbitration when the storage system runs normally, the states of the storage array A and the storage array B in the arbitration server are both in the unknown state. Please refer to Figure 5 , which is the state change process of the storage array in the embodiments of the present application. In this case, the state of the storage array is state 1.

[0092] Step 402: A replication link of the storage system fails. Storage array A sends a third arbitration request to the arbitration server and / or storage array B sends a fourth arbitration request to the arbitration server. The arbitration server receives the arbitration request.

[0093] It should be noted that the third arbitration request and the fourth arbitration request are respectively arbitration requests sent by the storage system before it becomes a dual-slave state.

[0094] Specifically, storage array A sending a third arbitration request to the arbitration server and / or storage array B sending a fourth arbitration request to the arbitration server includes the following three cases:

[0095] The first sending case: When the replication link between storage array A and storage array B fails. For example, the replication link is disconnected, etc. If both storage array A and storage array B are in normal states, that is, both storage array A and storage array B are powered on and no device failures have occurred, then both storage array A and storage array B will send arbitration requests to the arbitration server. If the arbitration links between storage array A and storage array B and the arbitration server have not failed, then the arbitration server receives two arbitration requests sent by the two storage arrays respectively, that is, the third arbitration request and the fourth arbitration request. If the arbitration link between one of the storage arrays and the arbitration server fails, then although the storage array sends an arbitration request to the arbitration server, the arbitration server cannot receive the arbitration request. For example, if the arbitration link between storage array A and the arbitration server is disconnected, then the arbitration server cannot receive the third arbitration request sent by storage array A.

[0096] The second sending case: If storage array A is in a normal state while storage array B is powered off or has a device failure, then only storage array A sends a third arbitration request to the arbitration server. If the arbitration link between storage array A and the arbitration server has not failed, the arbitration server only receives the third arbitration request sent by storage array A.

[0097] The third sending case: If storage array B is in a normal state while storage array A is powered off or has a device failure, then only storage array B sends a fourth arbitration request to the arbitration server. If the arbitration link between storage array B and the arbitration server has not failed, the arbitration server only receives the fourth arbitration request sent by storage array B.

[0098] It should be noted that in step 402, the storage system is not yet in a dual-slave state, and the arbitration process is triggered only due to a failure in the replication link. Additionally, in the embodiments of the present application, the arbitration request can be a data packet carrying specific content or an empty packet carrying a specific packet header, where the specific content and the specific packet header are pre-agreed upon by the arbitration server and the storage array. Of course, the arbitration request can also be in other forms, which are not limited herein. In Figure 4 it, taking the storage array A sending a third arbitration request to the arbitration server and the storage array B sending a fourth arbitration request to the arbitration server as an example.

[0099] Step 403, the arbitration server determines and stores the arbitration result.

[0100] In the embodiments of the present application, the arbitration server determining the arbitration result includes but is not limited to the following two methods. These two methods will be described separately below.

[0101] The first determination method, the arbitration server determines the arbitration result based on the received third arbitration request and / or fourth arbitration request. The following will describe different arbitration processes.

[0102] The first arbitration process, if the arbitration server receives the third arbitration request and the fourth arbitration request sent by two storage arrays respectively, then the arbitration server determines the arbitration result according to the order of receiving the third arbitration request and the fourth arbitration request, that is, if it first receives the arbitration request sent by a certain storage array, then it determines the arbitration result of that storage array as the winning state, and the arbitration result of the other storage array is the losing state. For example, the arbitration request carries the identification information of the storage array, and this identification information can be the number or index number of the storage array, etc. After receiving the first arbitration request, the arbitration server obtains the identification information of the storage array from this arbitration request. For example, if the arbitration server determines that the first arbitration request received carries the number of storage array A, then it determines the arbitration result of storage array A as the winning state and the arbitration result of storage array B as the losing state.

[0103] The second arbitration process, if the arbitration server only receives the arbitration request sent by one storage array, then the arbitration server can determine that the sender of this arbitration request is storage array A or storage array B according to the identification information of the storage array carried in this arbitration request, and determine that the storage array sending this arbitration request is in the winning state, while the other storage array is in the losing state or the unknown state or the checking state. For example, if the arbitration server determines that the received arbitration request carries the number of storage array A, then the arbitration server determines the arbitration result of storage array A as the winning state and the arbitration result of storage array B as the checking state.

[0104] It should be noted that when the status of a certain storage array is in the checking state, it means that the storage array is in a state where it cannot be read or written.

[0105] In the third arbitration process, the arbitration server can also determine the arbitration result of the storage array together with the received arbitration request and other information. This other information can be the heartbeat of the storage array. For example, storage array A and storage array B respectively send the third arbitration request and the fourth arbitration request to the arbitration server. However, due to a failure in the arbitration link between the arbitration server and storage array B, the arbitration server only receives the arbitration request sent by one storage array. The arbitration server determines the sender of the arbitration request as storage array A based on the identification information of the storage array carried in the arbitration request. Then, the arbitration server determines the arbitration result of storage array B, which did not receive the arbitration request, as the checking state according to the received arbitration request. If storage array A loses power for other reasons at the moment of sending the third arbitration request, the arbitration server determines that it has received the third arbitration request sent by this storage array A but cannot detect the heartbeat of this storage array, and thus determines the arbitration result of storage array A as the checking state. It should be noted that the arbitration server can determine whether each storage array is in a normal state through the heartbeat of each storage array. For example, each storage array can periodically send a heartbeat to the arbitration server. If the heartbeat of a certain storage array is detected, it is determined that the storage array is in a normal state. If the heartbeat of a certain storage array cannot be detected, it is determined that the storage array is in a faulty state. The communication link for carrying the heartbeat of each storage array is the same as the arbitration link, such as the iSCSI link, etc.

[0106] Of course, the arbitration server can also use other methods to determine the arbitration result, which will not be listed one by one here. Please continue to refer to Figure 5 , in this case, the status of the storage array changes to: if the arbitration result is successful, it changes to the winning state, that is, state 2; if the arbitration result is a failure, it changes to the losing state, that is, state 3.

[0107] After the arbitration server determines the arbitration result, it stores the arbitration result.

[0108] Step 404: The statuses of both storage array A and storage array B are in the checking state, and the storage system becomes a dual-slave state.

[0109] If the computer room of the storage array loses power, or the storage array A, the storage array B, and the arbitration server lose power simultaneously, or after a failure occurs in the replication link, the arbitration links between the storage array A and the arbitration server and between the storage array B and the arbitration server also fail, etc., resulting in the arbitration server being unable to send the arbitration result to the storage array after determining the arbitration result of the storage array A and the storage array B. Therefore, since the storage array A and the storage array B do not receive the arbitration result, the states of both the storage array A and the storage array B are in the checking state, and the storage system will become a dual-slave state. In this case, both the storage array A and the storage array B are in a state where they cannot be read or written and cannot provide services.

[0110] Please continue to refer to Figure 5 , in this case, since the storage system becomes a dual-slave state, the state of the storage array changes to the checking state, that is, state 4.

[0111] It should be noted that the scenarios that cause the storage system to become a dual-slave state include but are not limited to the above situations, and are not restricted in the embodiments of this application.

[0112] Steps 401 to 404 are optional steps, that is, they are not necessarily to be executed, and are illustrated by a dotted line in Figure 4 as an example.

[0113] Step 405, the storage system switches from the dual-slave state to the fault recovery state, and the storage array A sends a first arbitration request to the arbitration server, and the arbitration server receives this first arbitration request.

[0114] If the storage system becomes a dual-slave state due to the power failure of the computer room of the storage system or the simultaneous power failure of the storage array A, the storage array B, and the arbitration server, then when the storage system is powered on, the storage array A will send a first arbitration request to the arbitration server. Or, if the storage system becomes a dual-slave state due to the failure of the replication link of the storage system, the arbitration link between the storage array A and the arbitration server, and the arbitration link between the storage array B and the arbitration server, then when the storage array A detects the recovery of the failures of the replication link and the two arbitration links, the storage array A will send a first arbitration request to the arbitration server.

[0115] It should be noted that the first arbitration request can be understood as the arbitration request sent by the storage array A to the arbitration server after the storage system changes from the dual-slave state to the fault recovery state. In Figure 1 the storage system shown, the storage array A is the main storage array of the storage system. Therefore, the storage array A sending a first arbitration request to the arbitration server can be understood as the main storage array of the storage system sending the first arbitration request when the storage system changes from the dual-slave state to the fault recovery state.

[0116] Step 406: Storage array B sends a second arbitration request to the arbitration server, and the arbitration server receives the second arbitration request.

[0117] After the storage system changes from the dual-slave state to the fault recovery state, storage array B can also send an arbitration request to the arbitration server, that is, the second arbitration request. That storage array B sends a second arbitration request to the arbitration server can be understood as other storage arrays in the storage system except the primary storage array send arbitration requests to the arbitration server. When there are 3 or more storage arrays in the storage system, it can be understood that each storage array among other storage arrays in the storage system except the primary storage array sends an arbitration request to the arbitration server.

[0118] It should be noted that step 406 is an optional step, that is, it is not necessary to execute. If step 406 is executed, step 406 can be executed simultaneously with step 405, or step 406 can be executed first and then step 405, or step 405 can be executed first and then step 406. In the embodiments of the present application, the execution order of step 405 and step 406 is not restricted. In Figure 4 this example, step 405 is executed first and then step 406 is taken as an example.

[0119] Step 407: The arbitration server determines the first arbitration result of storage array A and the second arbitration result of storage array B.

[0120] After the arbitration server receives the first arbitration request sent by storage array A and / or the second arbitration request sent by storage array B, the arbitration server will determine whether it has stored the arbitration results of storage array A and storage array B. Since the arbitration server saves the persistent arbitration results and the arbitration results determined in step 403 have not been fed back to storage array A and storage array B, the arbitration server directly determines the first arbitration result and the second arbitration result from the stored arbitration results. For example, when the stored arbitration result is that storage array A is in the winning state and storage array B is in the losing state, the arbitration server determines that the first arbitration result is the winning state and the second arbitration result is the losing state; when the stored arbitration result is that storage array A is in the unknown state and storage array B is in the checking state, the first arbitration result is the unknown state and the second arbitration result is the checking state. Of course, if the stored arbitration result is other content, the first arbitration result and the second arbitration result will also change accordingly, which will not be listed one by one here.

[0121] Step 408: The arbitration server sends the first arbitration result to storage array A and the second arbitration result to storage array B. Storage array A receives the first arbitration result, and storage array B receives the second arbitration result.

[0122] Step 409: Storage array A initiates a negotiation with storage array B based on the first arbitration result.

[0123] Specifically, after storage array A receives the first arbitration result, it sends a negotiation request to storage array B. The negotiation request can be used to inquire about the request information of the second arbitration result of storage array B. Of course, it can also be a data packet carrying specific content or an empty packet with a specific packet header. The specific content and specific packet header are pre-agreed by storage array A and storage array B. Of course, the negotiation request can also be in other forms, which are not limited here.

[0124] Step 410: Storage array B feeds back the second arbitration result to storage array A, and storage array A receives the second arbitration result.

[0125] After storage array B receives the negotiation request sent by storage array A, it feeds back the second arbitration result obtained from the arbitration server to storage array A, and storage array A receives the second arbitration result.

[0126] Step 411: Storage array A determines the negotiation result based on the first arbitration result and the second arbitration result.

[0127] After the storage array A receives the second arbitration result fed back by the storage array B, it determines the negotiation result with the storage array B according to the preset negotiation principle. As an example, the negotiation principle of the storage array A can refer to the content shown in Table 1. As shown in Table 1, the negotiation principle is that when the local array status is in the checking state and the remote array status is in the winning state, the negotiation result is that the local array status is in the losing state and the remote array status is in the winning state; when the local array status is in the checking state and the remote array status is in the losing state, the negotiation result is that the local array status is in the winning state and the remote array status is in the losing state; when the local array status is in the checking state and the remote array status is in the unknown state, the negotiation result is that the local array status is in the losing state and the remote array status is in the winning state; when the local array status is in the checking state and the remote array status is in the checking state, the negotiation result is that the local array status is in the winning state and the remote array status is in the losing state. In Table 1, the local array is the array that determines the negotiation result, for example, the storage array A, and the remote array is the array that negotiates with the array that determines the negotiation result, for example, the storage array B. Of course, if the array that determines the negotiation result is the storage array B, then the local array is the storage array B, and the remote array is the storage array A. Those skilled in the art need to flexibly interpret the meanings of the local array and the remote array, which will not be elaborated here one by one.

[0128] Table 1

[0129] Local array status Remote array status Negotiation result Checking status Winning status (losing, winnig) Checking status Losing status (winnig, losing) Checking status Unknown status (losing, winnig) Checking status Checking status (winnig, losing)

[0130] Specifically, if storage array A determines that the first arbitration result is in the winning state and the second arbitration result is in the checking state, then according to the negotiation principle shown in Table 1, it is determined that the negotiation result is that storage array A is in the winning state and storage array B is in the losing state; if storage array A determines that the first arbitration result is in the checking state and the second arbitration result is in the losing state, then according to the negotiation principle shown in Table 1, it is determined that the negotiation result is that storage array A is in the winning state and storage array B is in the losing state; if storage array A determines that the first arbitration result is in the unknown state and the second arbitration result is in the checking state, then according to the negotiation principle shown in Table 1, storage array A determines that the negotiation result is that storage array A is in the winning state and storage array B is in the losing state; if storage array A determines that the first arbitration result is in the checking state and the second arbitration result is in the checking state, then according to the negotiation principle shown in Table 1, storage array A determines that the negotiation result is that storage array A is in the winning state and storage array B is in the losing state.

[0131] It should be noted that the negotiation principle shown in Table 1 is only illustrated by taking the state of the local array as the checking state as an example, which is just an example and should not be understood as a limitation to the negotiation principle. When the arbitration result of the local array is in other states, the negotiation principle can be adjusted with reference to the principle shown in Table 1, which will not be listed one by one here.

[0132] In addition, it should be noted that the negotiation result only includes two states, namely the winning state and the losing state.

[0133] Please continue to refer to Figure 5 , in this case, since the storage system changes from the dual-slave state to the fault recovery state, when the storage array determines that the negotiation is successful through negotiation, the state of the storage array changes to: if the negotiation is successful, it changes to the winning state, that is, state 2, and if the negotiation fails, it changes to the losing state, that is, state 3.

[0134] Step 412: Storage array A determines that the negotiation result is that storage array A is in the winning state and storage array B is in the losing state, and storage array A provides business services to the client.

[0135] When storage array A determines that the negotiation result is that the local array wins, storage array A actively restores the state of the local array to the winning state, so as to provide business services. Of course, when storage array A determines that the negotiation result is that the peer array wins, storage array A can send the negotiation result to storage array B. In this way, when storage array B determines that the array wins, it actively starts the business and provides business services.

[0136] It should be noted that if the storage array A determines that both the first arbitration result and the second arbitration result are in the winning state, or if the storage array A determines that both the first arbitration result and the second arbitration result are in the losing state, in this case, the storage array A determines that the negotiation fails. If the storage array A determines that the negotiation fails, the storage system cannot automatically resume services.

[0137] Step 413: The storage array A sends synchronization data to the storage array B.

[0138] After the storage array A provides business services, it sends the latest data of the business to the storage array B, so that the business data in the storage array B is synchronized with the storage array A. After all the differential data of the array A is synchronized to the array B, the dual-active feature of the storage system is restored, and thus the business can be provided by both the storage array A and the storage array B simultaneously.

[0139] Step 413 is an optional step, that is, it is not necessary to execute.

[0140] In the above technical solution, when the storage system changes from the dual-slave state to the fault recovery state, the winning storage array can be automatically determined through the above negotiation principle, and the winning storage array actively starts the business, so that the waiting time for manually starting the business can be reduced, and the business recovery efficiency of the storage system can be improved.

[0141] In Figure 4 In the shown embodiment, when the storage system changes from the dual-slave state to the fault recovery state, the arbitration server determines the arbitration results of each storage array after the fault recovery state according to the pre-stored arbitration results. Next, an embodiment of the arbitration server re-determining the arbitration results when the storage system changes from the dual-slave state to the fault recovery state will be introduced.

[0142] Please refer to Figure 6 , which is a flowchart of another embodiment of the service recovery method provided by the embodiment of the present application. The description of this process is as follows:

[0143] Step 601: The storage system operates normally, and the storage array processes the business.

[0144] In the embodiment of the present application, the storage system includes a first storage array, a second storage array, and an arbitration server. It can be understood that the storage system is a dual-active storage system, and specifically, it can be one of the storage systems as shown in Figures 1 to 3B For convenience of description, in the following introduction, the storage system shown in Figure 1 will be used as an example for illustration. That is to say, the first storage array is the storage array A in Figure 1 , the second storage array is the storage array B in Figure 1 , and the arbitration server isFigure 1 The arbitration server in

[0145] Step 601 is the same as step 401 and will not be elaborated here.

[0146] Step 602: The storage system becomes a dual-slave state, and the states of both storage array A and storage array B are in the checking state.

[0147] In the embodiment of the present application, the storage system directly changes from normal operation to a dual-slave state. For example, the computer room of the storage array loses power, or storage array A, storage array B, and the arbitration server lose power simultaneously, or at the same moment when the replication link fails, the arbitration links between storage array A and the arbitration server and between storage array B and the arbitration server also fail, etc. In this case, storage array A and storage array B directly become a dual-slave state before they have time to request arbitration from the arbitration server.

[0148] It should be noted that steps 601 to 602 are optional steps, that is, they are not necessarily to be executed.

[0149] Step 603: The storage system switches from the dual-slave state to the fault recovery state. Storage array A sends a first arbitration request to the arbitration server, and the arbitration server receives this first arbitration request.

[0150] Step 603 is the same as step 405, and the first arbitration request is the same as the first arbitration request in step 405.

[0151] Step 604: Storage array B sends a second arbitration request to the arbitration server, and the arbitration server receives this second arbitration request.

[0152] Step 604 is the same as step 406, and the second arbitration request is the same as the second arbitration request in step 406.

[0153] It should be noted that step 604 is an optional step, that is, it is not necessarily to be executed. For example, when the storage system changes from the dual-slave state to the fault recovery state, it may be due to a fault in the arbitration link between storage array B and the arbitration server or a failure of the sending device of storage array B, etc., resulting in storage array B being unable to send a second arbitration request to the arbitration server.

[0154] Step 605: The arbitration server determines the first arbitration result of storage array A and the second arbitration result of storage array B.

[0155] After the arbitration server receives the first arbitration request sent by storage array A and / or the second arbitration request sent by storage array B, the arbitration server determines whether it has already stored the arbitration results of storage array A and storage array B. Since the storage system directly changes from normal operation to a dual-slave state, that is, before the storage system changes to the dual-slave state, neither storage array A nor storage array B has sent an arbitration request to the arbitration server. Therefore, there is no arbitration result in the arbitration server for storage array A and storage array B that has not been arbitrated yet.

[0156] If the method in the embodiment of the present application executes step 603 and step 604, that is, the arbitration server receives two arbitration requests from storage array A and storage array B, then the arbitration server needs to re-arbitrate the status of the storage array according to the first arbitration request received from storage array A and the second arbitration request received from storage array B. Specifically, the arbitration server determines the first arbitration result and the second arbitration result according to the order of receiving the first arbitration request and receiving the second arbitration request. If the arbitration server receives the second arbitration request after receiving the first arbitration request, then it is determined that the first arbitration result is in the winning state, and the second arbitration result is in the losing state. For example, after the arbitration server receives the first arbitration request, it determines that the arbitration request carries the identification information of storage array A, and then the arbitration server receives the second arbitration request, which carries the identification information of storage array B. Then it is determined that the first arbitration request is received first and the second arbitration request is received later, so as to determine that the arbitration result of storage array A is in the winning state and the arbitration result of storage array B is in the losing state.

[0157] If the method in the embodiment of the present application only executes step 603, that is, the arbitration server only receives the first arbitration request from storage array A and does not receive the second arbitration request from storage array B within a preset time period starting from the receipt of the first arbitration request, then the arbitration server determines that the first arbitration result is in the winning state and the second arbitration result is in the checking state.

[0158] It should be noted that the preset time period can be pre-configured by the arbitration server or pre-agreed by the arbitration server and the storage array in advance. Specifically, it can be 2ms or 5ms, etc., and there is no limitation here.

[0159] Step 606: The arbitration server sends the first arbitration result to storage array A and sends the second arbitration result to storage array B. Storage array A receives the first arbitration result, and storage array B receives the second arbitration result.

[0160] Step 607: Storage array A initiates negotiation with storage array B according to the first arbitration result.

[0161] Step 608: Storage array B feeds back the second arbitration result to storage array A, and storage array A receives the second arbitration result.

[0162] Step 609: Storage array A determines the negotiation result according to the first arbitration result and the second arbitration result.

[0163] Step 610: Storage array A determines that the negotiation result is that storage array A is in the winning state and storage array B is in the losing state, and storage array A provides business services to the client.

[0164] Step 611: Storage array A sends synchronization data to storage array B.

[0165] Steps 606 to 611 are the same as steps 408 to 413, and will not be elaborated here.

[0166] In the above technical solution, when the storage system changes from the dual-slave state to the fault recovery state, the winning storage array can be automatically determined through the above negotiation principle, and the winning storage array actively starts the service, thereby reducing the waiting time for manually starting the service and improving the service recovery efficiency of the storage system.

[0167] In Figure 4 and Figure 6 's implementation mode, the storage system needs to start the service according to the arbitration result of the arbitration server. In other scenarios, for example, in the multiple-fault scenario, the storage system can also initiate negotiation actively by the master array without relying on the arbitration server to recover the service. The service recovery process in the multiple-fault scenario is described below.

[0168] As an example, when the dual-active storage system shown in Figure 1 is in the normal working state, the arbitration link between the arbitration server and storage array A may be disconnected due to certain reasons. Since the disconnection of the arbitration link does not affect the operation of the storage system, in this case, storage array A and storage array B can still maintain the normal working state, that is, storage array A and storage array B process the service simultaneously. When storage array A and storage array B process the service simultaneously, the data of the service will be written into both storage array A and storage array B. If storage array B fails, the data of the service cannot be successfully written into storage array B. At this time, storage array B will set its own state to the losing state, thereby stopping processing the service, while storage array A determines that the data of the service cannot be successfully written into storage array B, then confirms that storage array B has failed, and thus determines that it needs to set its own state to the winning state, and storage array A processes the service.

[0169] If storage array A determines that it needs to set its own state to the winning state, and before storage array A becomes the winning state, the replication link between storage array A and storage array B is disconnected. At this time, the arbitration process of the arbitration server will be triggered. Since the arbitration link between the arbitration server and storage array A is disconnected, the arbitration server cannot receive the third arbitration request sent by storage array A. Therefore, storage array A will not receive the arbitration result from the arbitration service network. When storage array A does not receive the arbitration result within the preset duration, it is determined that the arbitration has timed out, and thus its own state is set to the checking state.

[0170] In this way, when the replication link between storage array A and storage array B is restored, storage array A can actively initiate a negotiation with storage array B. Then, storage array B sends its stored state, that is, the losing state, to storage array A. Storage array A can determine that the negotiation result of storage array A is the winning state and the negotiation result of storage array B is the losing state according to the negotiation principle shown in Table 1, so as to determine that storage array A processes the service.

[0171] As another example, when the Figure 1 dual-active storage system shown is in a normal working state, the arbitration link between the arbitration server and storage array A may be disconnected due to certain reasons. Since the disconnection of the arbitration link does not affect the operation of the storage system, in this case, storage array A and storage array B can still maintain a normal working state, that is, storage array A and storage array B process services simultaneously. Since storage array B is in a normal state, the state of storage array B is the unknown state. If storage array B powers off and then restarts, it is also in the unknown state after the restart is completed.

[0172] If the replication link between storage array A and storage array B is disconnected when storage array B powers off and restarts, at this time, the arbitration process of the arbitration server will be triggered. Since the arbitration link between the arbitration server and storage array A is disconnected, the arbitration server cannot receive the third arbitration request sent by storage array A. Therefore, the arbitration server will not perform arbitration. Thus, storage array A will not receive the arbitration result from the arbitration server either. When storage array A does not receive the arbitration result within the preset duration, it is determined that the arbitration has timed out, and thus its own state is set to the checking state. Since storage array B powers off and then restarts, the state of storage array B after restart is the unknown state.

[0173] In this way, when the replication link between storage array A and storage array B is restored, storage array A can actively initiate negotiation with storage array B. Then, storage array B sends its stored state, i.e., the unknown state, to storage array A. Storage array A can determine that the negotiation result of storage array A is the losing state and the negotiation result of storage array B is the winning state according to the negotiation principle shown in Table 1, so as to determine that storage array B processes the service.

[0174] Through the above technical solution, storage arrays can also actively negotiate with each other, so as to determine the storage array for processing services after a failure recovery, which can reduce the interaction between the storage array and the arbitration server and improve the processing efficiency.

[0175] In the above embodiments provided by the present application, the method provided by the embodiments of the present application is introduced from the perspective of the interaction among the first storage array, the second storage array, and the arbitration server. To implement each function in the method provided by the above embodiments of the present application, the first storage array may include a hardware structure and / or a software module, and implement the above functions in the form of a hardware structure, a software module, or a combination of a hardware structure and a software module. Whether a certain function among the above functions is executed in the form of a hardware structure, a software module, or a combination of a hardware structure and a software module depends on the specific application and design constraints of the technical solution.

[0176] Figure 7 The structural schematic diagram of a service recovery device 700 is shown. Among them, the service recovery device 700 can be applied to the first storage array or a device in the first storage array, and can implement the functions of the first storage array in the method provided by the embodiments of the present application; the service recovery device 700 can also be a device that supports the first storage array to implement the functions of the first storage array in the method provided by the embodiments of the present application. The service recovery device 700 can be a hardware structure, a software module, or a combination of a hardware structure and a software module. The service recovery device 700 can be implemented by a chip system. In the embodiments of the present application, the chip system can be composed of chips or can include chips and other discrete devices.

[0177] The service recovery device 700 may include a transceiver unit 701 and a processing unit 702.

[0178] The transceiver unit 701 can be used to execute Figure 4 steps 401, 402, 405, 408 - 410, and 413 in the embodiments shown, and / or used to execute Figure 6Steps 601, 603, 606 - 608, and 611 in the illustrated embodiment, and / or other processes for supporting the techniques described herein. The transceiver unit 701 is used to communicate with the storage array 700 and other modules, and it can be a circuit, device, interface, bus, software module, transceiver, or any other device capable of implementing communication.

[0179] The processing unit 702 can be used to execute Figure 4 Steps 401, 404, 411, and 412 in the illustrated embodiment, and / or for executing Figure 6 Steps 601, 602, 609, and 610 in the illustrated embodiment, and / or other processes for supporting the techniques described herein.

[0180] Among them, all relevant contents of each step involved in the above method embodiment can be cited in the function description of the corresponding functional module, and will not be elaborated here.

[0181] The division of modules in the embodiments of the present application is illustrative, merely a logical function division. In actual implementation, there may be other division methods. Additionally, in each embodiment of the present application, each functional module can be integrated in one processor, or exist separately physically, or two or more modules can be integrated in one module. The above integrated modules can be implemented in the form of hardware or in the form of software functional modules.

[0182] As Figure 8 Shown is the service recovery device 800 provided by the embodiment of the present application. Among them, the service recovery device 800 can be Figure 4 or Figure 6 The first storage array in the illustrated embodiment, or a device in the first storage array, capable of implementing the present application Figure 4 or Figure 6 The function of the first storage array in the illustrated embodiment; the service recovery device 800 can also be a device capable of supporting the first storage array to implement the function of the first storage array in the method provided by the embodiment of the present application Figure 4 or Figure 6 Shown in the illustrated embodiment. Among them, the service recovery device 800 can be a chip system. In the embodiments of the present application, the chip system can be composed of chips, or can include chips and other discrete devices.

[0183] The service recovery device 800 includes at least one processor 820, which is used to implement or support the service recovery device 800 to implement the present application Figure 4 or Figure 6The function of the first storage array in the illustrated embodiment. Exemplarily, the processor 820 may initiate a negotiation with the second storage array according to the first arbitration result, determine the negotiation result, and if the negotiation result is that the storage array is in the winning state and the second storage array is in the losing state, provide business services to the client. For specific details, refer to the detailed description in the method example, which will not be elaborated here.

[0184] The service recovery device 800 may further include at least one memory 830 for storing program instructions and / or data. The memory 830 is coupled to the processor 820. The coupling in the embodiments of the present application is an indirect coupling or communication connection between devices, units or modules, which may be electrical, mechanical or other forms for information interaction between devices, units or modules. The processor 820 may cooperate with the memory 830. The processor 820 may execute the program instructions stored in the memory 830. At least one of the at least one memory may be included in the processor. When the processor 820 executes the program instructions in the memory 830, it may implement Figure 4 or Figure 6 the method shown.

[0185] The service recovery device 800 may further include a communication interface 810 for communicating with other devices through a transmission medium, so that the devices in the storage array 800 can communicate with other devices. Exemplarily, the other device may be a second storage array or an arbitration server. The processor 820 may use the communication interface 810 to send and receive data.

[0186] In the embodiments of the present application, the specific connection medium between the communication interface 810, the processor 820 and the memory 830 is not limited. In the embodiments of the present application Figure 8 it is shown that the memory 830, the processor 820 and the communication interface 810 are connected through a bus 840. The bus is represented by a thick line in Figure 8 for illustrative purposes only, and the connection manners between other components are only for illustrative purposes and are not to be construed as limiting. The bus may be divided into an address bus, a data bus, a control bus, etc. For the sake of simplicity of representation, Figure 8 only one thick line is used in

[0187] In an embodiment of the present application, the processor 820 may be a general-purpose processor, a digital signal processor, an application-specific integrated circuit, a field-programmable gate array, or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, and can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the method disclosed in combination with the embodiments of the present application may be directly embodied as being executed by a hardware processor, or executed by a combination of hardware and software modules in the processor.

[0188] In an embodiment of the present application, the memory 830 may be a non-volatile memory, such as a hard disk drive (HDD) or a solid-state drive (SSD), etc., or may also be a volatile memory, such as a random-access memory (RAM). The memory is any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory in the embodiments of the present application may also be a circuit or any other device capable of implementing a storage function, for storing program instructions and / or data.

[0189] Figure 9 The structure diagram of an arbitration server 900 is shown. Among them, the arbitration server 900 can implement the functions of the arbitration server in the method provided in the embodiments of the present application; the arbitration server 900 may also be a device capable of supporting the arbitration server to implement the functions of the arbitration server in the method provided in the embodiments of the present application. The arbitration server 900 may be a hardware structure, a software module, or a combination of a hardware structure and a software module. The arbitration server 900 may be implemented by a chip system. In the embodiments of the present application, the chip system may be composed of chips, or may include chips and other discrete devices.

[0190] The arbitration server 900 may include a transceiver unit 901 and a processing unit 902.

[0191] The transceiver unit 901 may be used to execute Figure 4 Steps 402, 405, 406, and 408 in the embodiments shown, and / or used to execute Figure 6 Steps 603, 604, and 606 in the embodiments shown, and / or used to support other processes of the technologies described herein. The transceiver unit 901 is used for the arbitration server 900 to communicate with other modules, and it may be a circuit, a device, an interface, a bus, a software module, a transceiver, or any other device capable of implementing communication.

[0192] The processing unit 902 can be used to execute Figure 4 step 403 and step 407 in the embodiments shown, and / or to execute Figure 6 step 605 in the embodiments shown, and / or to support other processes of the technologies described herein.

[0193] All relevant content of each step involved in the above method embodiments can be cited in the function descriptions of the corresponding functional modules, which will not be elaborated here.

[0194] The division of modules in the embodiments of the present application is illustrative, merely a logical function division. In actual implementation, there may be other division methods. Additionally, in each embodiment of the present application, each functional module can be integrated in a processor, can exist separately physically, or two or more modules can be integrated in one module. The above integrated modules can be implemented in the form of hardware or in the form of software functional modules.

[0195] As Figure 10 shown, the arbitration server 1000 provided by the embodiments of the present application, where the arbitration server 1000 can be Figure 4 or Figure 6 the arbitration server in the embodiments shown, and can implement the present application Figure 4 or Figure 6 the functions of the arbitration server in the embodiments shown; the arbitration server 1000 can also be a device capable of supporting the arbitration server to implement the functions of the arbitration server in the methods provided by the embodiments of the present application Figure 4 or Figure 6 shown. Among them, the arbitration server 1000 can be a chip system. In the embodiments of the present application, the chip system can be composed of chips or can include chips and other discrete devices.

[0196] The arbitration server 1000 includes at least one processor 1020, which is used to implement or support the arbitration server 1000 to implement Figure 4 or Figure 6 the functions of the arbitration server in the embodiments shown. Exemplarily, the processor 1020 can determine the first arbitration result of the first storage array and the second arbitration result of the second storage array according to the first arbitration request. For specific details, refer to the detailed description in the method examples, which will not be elaborated here.

[0197] The arbitration server 1000 may further include at least one memory 1030 for storing program instructions and / or data. The memory 1030 is coupled to the processor 1020. The coupling in the embodiments of the present application is an indirect coupling or communication connection between devices, units or modules, which can be electrical, mechanical or other forms for information interaction between devices, units or modules. The processor 1020 may cooperate with the memory 1030. The processor 1020 may execute the program instructions stored in the memory 1030. At least one of the at least one memory may be included in the processor. When the processor 1020 executes the program instructions in the memory 1030, it can implement Figure 4 or Figure 6 the method shown.

[0198] The arbitration server 1000 may further include a communication interface 1010 for communicating with other devices through a transmission medium, so that the devices in the storage array 1000 can communicate with other devices. Exemplarily, the other device may be a second storage array or a first storage array. The processor 1020 may use the communication interface 1010 to send and receive data.

[0199] In the embodiments of the present application, the specific connection medium between the communication interface 1010, the processor 1020 and the memory 1030 is not limited. In the embodiments of the present application Figure 10 it is shown that the memory 1030, the processor 1020 and the communication interface 1010 are connected through a bus 1040. The bus is represented by a thick line in Figure 10 The connection manners between other components are only for illustrative purposes and are not to be construed as limiting. The bus may be divided into an address bus, a data bus, a control bus, etc. For the sake of simplicity of representation, Figure 10 only one thick line is used to represent it in

[0200] In the embodiments of the present application, the processor 1020 may be a general-purpose processor, a digital signal processor, an application-specific integrated circuit, a field-programmable gate array or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, and can implement or execute the various methods, steps and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the method disclosed in combination with the embodiments of the present application may be directly embodied as being executed by a hardware processor, or executed by a combination of hardware and software modules in the processor.

[0201] In the embodiments of the present application, the memory 1030 may be a non-volatile memory, such as a hard disk drive (HDD) or a solid-state drive (SSD), etc., or may also be a volatile memory, such as a random-access memory (RAM). A memory is any other medium that can be used to carry or store desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory in the embodiments of the present application may also be a circuit or any other device capable of implementing a storage function, for storing program instructions and / or data.

[0202] In the embodiments of the present application, there is also provided a computer-readable storage medium, including instructions, which when running on a computer, cause the computer to execute Figure 4 or Figure 6 the method executed by the first storage array in the embodiments shown.

[0203] In the embodiments of the present application, there is also provided a computer-readable storage medium, including instructions, which when running on a computer, cause the computer to execute Figure 4 or Figure 6 the method executed by the arbitration server in the embodiments shown.

[0204] In the embodiments of the present application, there is also provided a computer program product, including instructions, which when running on a computer, cause the computer to execute Figure 4 or Figure 6 the method executed by the first storage array in the embodiments shown.

[0205] In the embodiments of the present application, there is also provided a computer program product, including instructions, which when running on a computer, cause the computer to execute Figure 4 or Figure 6 the method executed by the arbitration server in the embodiments shown.

[0206] The embodiments of the present application provide a chip system, which includes a processor and may also include a memory for implementing the functions of the first storage array in the foregoing method. The chip system may be composed of chips or may include chips and other discrete devices.

[0207] The embodiments of the present application provide a chip system, which includes a processor and may also include a memory for implementing the functions of the arbitration server in the foregoing method. The chip system may be composed of chips or may include chips and other discrete devices.

[0208] An embodiment of the present application further provides a storage system, which includes the first storage array described in the foregoing method and the arbitration server in the foregoing method.

[0209] An embodiment of the present application further provides a storage system, which includes the first storage array, the second storage array described in the foregoing method, and the arbitration server in the foregoing method.

[0210] In the method provided by the embodiment of the present application, it can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in the embodiment of the present application are generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, a network device, a user device, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website, a computer, a server, or a data center to another website, a computer, a server, or a data center in a wired manner (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or a wireless manner (such as infrared, wireless, microwave, etc.). The computer-readable storage medium may be any available medium that can be accessed by a computer, or a data storage device such as a server or a data center that includes one or more integrated available media. The available medium may be a magnetic medium (such as a floppy disk, a hard disk, a magnetic tape), an optical medium (such as a digital video disc (DVD)), or a semiconductor medium (such as an SSD), etc.

[0211] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalent technologies, the present application is also intended to include these changes and modifications.

Claims

1. A service recovery method, characterized in that, Applied to a storage system, the storage system includes a first storage array, a second storage array, and an arbitration server, and the method includes: After the storage system switches from a dual-slave state to a fault recovery state, the first storage array sends a first arbitration request to the arbitration server, and the second storage array sends a second arbitration request to the arbitration server; when the storage system is in the dual-slave state, neither the first storage array nor the second storage array can process services. The first storage array receives a first arbitration result from the arbitration server. The second storage array receives a second arbitration result from the arbitration server. If the first arbitration result indicates that the first storage array is in a winning state and the second arbitration result indicates that the second storage array is in a losing state, the first storage array provides service to the client.

2. The method according to claim 1, wherein After the first storage array provides service to the client, the method further includes: The first storage array sends synchronized data to the second storage array.

3. A service recovery method, characterized in that, Applied to a storage system, the storage system includes a first storage array and a second storage array, and the method includes: After the storage system switches from a dual-slave state to a fault recovery state, the first storage array initiates a negotiation with the second storage array to determine a negotiation result; when the storage system is in the dual-slave state, neither the first storage array nor the second storage array can process services. If it is determined that the negotiation result is that the first storage array is in a winning state and the second storage array is in a losing state, the first storage array provides service to the client.

4. The method according to claim 3, wherein After the first storage array provides service to the client, the method further includes: The first storage array sends synchronized data to the second storage array.

5. A storage system, characterized in that, The storage system includes a first storage array, a second storage array, and an arbitration server. The first storage array is configured to send a first arbitration request to the arbitration server after the storage system switches from a dual-slave state to a fault recovery state. The second storage array is configured to send a second arbitration request to the arbitration server. When the storage system is in the dual-slave state, neither the first storage array nor the second storage array can process services. The first storage array is further configured to receive a first arbitration result from the arbitration server. The second storage array is further configured to receive a second arbitration result from the arbitration server. The first storage array is further configured to provide service to the client when the first arbitration result indicates that the first storage array is in a winning state and the second arbitration result indicates that the second storage array is in a losing state.

6. The storage system according to claim 5, wherein The first storage array is further configured to send synchronized data to the second storage array after the first storage array provides service to the client.

7. A storage system, characterized in that, The storage system includes a first storage array and a second storage array. The first storage array is used to initiate negotiation with the second storage array after the storage system switches from the dual-slave state to the fault recovery state, and determine the negotiation result; When the storage system is in the dual-slave state, neither the first storage array nor the second storage array can process services; And The first storage array provides service to the client when the negotiation result indicates that the first storage array is in the winning state and the second storage array is in the losing state.

8. The storage system according to claim 7, wherein The first storage array is further used to send synchronization data to the second storage array after providing service to the client.

9. A computer-readable storage medium, characterized in that, Instructions are stored on the medium, and when the instructions are run on a computer, the computer is caused to execute the method according to any one of claims 1-2 or 3-4.

Images