Method and apparatus for implementing distributed lock
By implementing a distributed lock on the storage-side management node, the problem of I/O conflicts in a shared storage environment is solved, enabling highly reliable data operations without hardware modifications and reducing costs.
Patent Information
- Application Number
- CN202210135345.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-02-14
- Publication Date
- 2025-11-04
- Estimated Expiration
- 2042-02-14
AI Technical Summary
In a shared storage environment, when multiple front-end computing nodes access the same storage space simultaneously, it may lead to I/O conflicts or even data corruption. Existing technologies require hardware modifications, which increases costs.
A distributed lock is implemented on the storage-side management node. The lock shares information to determine whether to allow related commands, thus avoiding conflicts, and no hardware modification is required.
This approach avoids storage space operation conflicts without changing the hardware, reduces implementation costs, and improves system reliability and data security.
Smart Images

Figure CN114579321B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of computer technology, and in particular to a method and apparatus for implementing a distributed lock. Background Technology
[0002] As competition intensifies in the hybrid cloud and on-premises enterprise customer markets, it is becoming increasingly important for public cloud storage products to meet the storage needs of on-premises enterprise customers. Meeting these specific enterprise requirements is gradually becoming a key factor in on-premises customer bidding. Shared storage space features, such as shared LUNs (Logical Unit Numbers), are among the most core and fundamental features in enterprise storage.
[0003] In a shared storage environment, multiple front-end computing nodes may simultaneously access the same storage space through a distributed architecture. If multiple front-end computing nodes perform I / O (input / output) operations on the same storage space (e.g., the same LUN) at the same time, I / O conflicts will occur, potentially leading to data corruption. Therefore, a distributed lock implementation is urgently needed to achieve reasonable sharing of storage space. Summary of the Invention
[0004] In view of this, this application provides a method and apparatus for implementing a distributed lock, so as to achieve reasonable sharing of storage space through a distributed lock.
[0005] This application provides the following solution:
[0006] According to the first aspect, a method for implementing a distributed lock is provided, applicable to a system comprising multiple hosts, multiple storage-side management nodes, and shared storage space; the method includes:
[0007] The storage-side management node receives commands from the host requesting the shared storage space.
[0008] Based on the lock sharing information local to the storage-side management node, determine whether to allow the relevant command;
[0009] If the relevant command is allowed, the lock sharing information is updated, and the information allowing the relevant command is shared with other storage-side management nodes;
[0010] The lock sharing information maintains the status information of the shared storage space, including information on registration, reservation, preemption, or idle status.
[0011] According to the implementation method of Embodiment 1 of this application, before determining whether to allow the relevant command based on the lock sharing information local to the storage-side management node, the method further includes:
[0012] Identify the type of the relevant command;
[0013] If the relevant command is a reserved command or a preemption command, then continue to execute the step of determining whether to allow the relevant command based on the lock sharing information of the local storage management node;
[0014] If the relevant command is a registration command, then the registration command is allowed, and the registration information is shared with other storage-side management nodes.
[0015] According to the implementation method of Embodiment 1 of this application, determining whether to allow the relevant commands based on the lock sharing information local to the storage-side management node includes:
[0016] If the lock sharing information contains information that the shared storage space has been reserved or preempted by other hosts, then the reservation command or preemption command is not allowed; otherwise, the reservation command or preemption command is allowed.
[0017] Updating the lock sharing information includes updating the information in the lock sharing information regarding whether the shared storage space was reserved or preempted by the host that sent the relevant command.
[0018] According to the embodiment of this application, the method further includes:
[0019] If, based on the responses from other storage-side management nodes regarding the information allowing the relevant commands, it is determined that a storage-side management node is out of contact or that there is a command conflict, then the reserved command or preemption command is not allowed.
[0020] According to the embodiment of this application, the method further includes:
[0021] If the reservation command or preemption command is allowed, a lock key is assigned to the host.
[0022] According to the embodiment of this application, the method further includes:
[0023] If an I / O command is received, it is determined whether the I / O command carries the assigned lock key. If so, the I / O command is allowed; otherwise, the I / O command is rejected.
[0024] According to the embodiment of this application, the method further includes:
[0025] If information allowing related commands is received from other storage-side management nodes, a response is returned;
[0026] If the information of the relevant command does not conflict with the lock sharing information stored locally, the lock sharing information stored locally is updated; otherwise, the response carries information indicating a command conflict.
[0027] According to the embodiment of this application, the method further includes:
[0028] If the relevant command is not allowed, a message denying the relevant command is returned to the host.
[0029] According to the embodiment of this application, the system further includes a distributed device, and the method further includes:
[0030] The storage-side management node sends the lock sharing information stored locally to the distributed device for serialization and storage, so that the newly added storage-side management node can obtain the lock sharing information from the distributed device and store it locally.
[0031] According to a second aspect, a distributed lock implementation device is provided, applied to a system comprising multiple hosts, multiple storage-side management nodes, distributed devices, and shared storage space, wherein the device is disposed at the storage-side management node, comprising:
[0032] The first communication unit is configured to receive commands from the host requesting the shared storage space.
[0033] The decision unit is configured to determine whether to allow the relevant command based on the lock sharing information local to the storage-side management node;
[0034] A lock information maintenance unit is configured to update the lock sharing information if the decision unit determines that the relevant command is allowed, wherein the lock sharing information maintains the status information of the shared storage space, and the status information includes registered information, reserved information, preempted information, or idle information;
[0035] The second communication unit is configured to share information that allows the relevant commands with other storage-side management nodes.
[0036] According to the third aspect, a distributed lock implementation system is provided, including multiple hosts, multiple storage-side management nodes, and shared storage space;
[0037] The host is configured to send commands to the storage-side management node requesting the shared storage space;
[0038] The storage-side management node is configured to receive commands from the host requesting the shared storage space; determine whether to allow the commands based on the lock sharing information on the local storage-side management node; if the commands are allowed, update the lock sharing information and share the information allowing the commands with other storage-side management nodes; wherein, the lock sharing information maintains the status information of the shared storage space, including information on registration, reservation, preemption, or idle status.
[0039] According to a fourth aspect, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in any one of the first aspects.
[0040] According to a fifth aspect, an electronic device is provided, characterized in that it comprises:
[0041] One or more processors; and
[0042] A memory associated with the one or more processors, the memory being used to store program instructions that, when read and executed by the one or more processors, perform the steps of the method described in any one of the first aspects above.
[0043] According to the specific embodiments provided in this application, the technical solution of this application can have the following advantages:
[0044] 1) After receiving a command from the host requesting shared storage space, this application determines whether to allow the command by using lock sharing information maintained by the storage-side management node. This lock sharing information is maintained through a lock information sharing mechanism with each storage-side management node, thereby ensuring that the host's operation on the shared storage space does not conflict.
[0045] 2) The distributed lock in this application is not implemented on the storage device, but on the first layer (transmission bearer layer) on the storage side. It can be adapted to existing host and storage device hardware without hardware modification, thus reducing implementation costs.
[0046] 3) This application can further reject relevant commands when the passive node is disconnected or when a command conflict is determined to exist in the passive node, thereby more reliably avoiding data corruption caused by the conflict.
[0047] Of course, any product implementing this application does not necessarily need to achieve all of the advantages described above at the same time. Attached Figure Description
[0048] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0049] Figure 1 An exemplary system architecture that can be applied to embodiments of this application is shown;
[0050] Figure 2 The main method flowchart provided for the embodiments of this application;
[0051] Figure 3 This application provides an implementation architecture for a distributed lock on a storage management node.
[0052] Figure 4 A flowchart for processing the register command provided in this application embodiment;
[0053] Figure 5 A flowchart for processing the reserve command provided in an embodiment of this application;
[0054] Figure 6 A flowchart for processing preemption commands provided in embodiments of this application;
[0055] Figure 7 A flowchart of processing I / O commands provided in the embodiments of this application;
[0056] Figure 8 A schematic block diagram of an implementation apparatus for a distributed lock according to one embodiment is shown;
[0057] Figure 9 An example is shown of the architecture of an electronic device. Detailed Implementation
[0058] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.
[0059] The terminology used in the embodiments of this invention is for the purpose of describing particular embodiments only and is not intended to limit the invention. The singular forms “a,” “the,” and “the” as used in the embodiments of this invention and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise.
[0060] It should be understood that the term "and / or" used in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.
[0061] Depending on the context, the word "if" as used here can be interpreted as "when," "when," "in response to determination," or "in response to detection." Similarly, depending on the context, the phrase "if determination" or "if detection (of the stated condition or event)" can be interpreted as "when determination," "in response to determination," "when detection (of the stated condition or event)," or "in response to detection (of the stated condition or event)."
[0062] Some companies have implemented distributed locks in their storage products, but this is achieved by setting up dual controllers in the storage device. Two controllers need to exist simultaneously on the access path within the same storage space, and this requires special hardware support, meaning hardware modifications to the storage device are necessary, increasing costs.
[0063] This application, however, adopts a completely different implementation approach. Instead of implementing it on the storage device, it is implemented at the first layer of command reception, namely the transport layer. For ease of understanding, please refer to [link to relevant documentation]. Figure 1 , Figure 1 An exemplary system architecture that can be applied to embodiments of this application is shown.
[0064] This system architecture primarily includes multiple initiators, multiple storage-side management nodes, and shared storage space, and may further include distributed devices. The initiators are devices that access the shared storage space and can be various computing nodes, either physical or virtual. The initiators provide iSCSI initiator capabilities, for example, through libiscsi (a static data linking library for iSCSI). The iSCSI initiator has the ability to discover backend iSCSI target servers and scan the corresponding LUNs for the storage space. iSCSI (Internet Small Computer System Interface) is an IPSAN (Storage Area Network) technology that combines existing SCSI interfaces with Ethernet technology, connecting iSCSI target servers and iSCSI initiators based on the TCP / IP protocol, enabling encapsulated SCSI data packets to be transmitted over the general internet. The iSCSI initiator and iSCSI target server are client and server devices for the SCSI protocol. The iSCSI initiator connects to the iSCSI target server, which then responds to the request.
[0065] The storage-side management node is the node responsible for managing the shared storage space on the storage device side, residing in the transport layer of the iSCSI TargetServer. In this application, the storage-side management node is responsible for implementing the distributed lock. Based on its local lock-sharing information, the storage-side management node makes decisions regarding relevant commands from the host, determining whether to allow the command, and shares this information with other storage-side management nodes, thereby ensuring the consistency of lock-sharing information across all storage-side management nodes.
[0066] Distributed devices are used to serialize and store lock-sharing information locally on each storage-side management node.
[0067] In this system, there are typically multiple storage-side management nodes. Multiple hosts can access the shared storage space through different storage-side management nodes. Commands sent by a host are usually intercepted by one of the storage-side management nodes. For example, commands sent by a host may be routed to one of the storage-side management nodes using load balancing. Alternatively, commands may be routed to one of the storage-side management nodes based on proximity, and so on.
[0068] The host and storage-side management node, storage-side network nodes, and storage-side network nodes and distributed devices can interact through various network connections, such as wired and wireless communication links or fiber optic cables.
[0069] Shared storage space refers to the space in a storage device, such as a hard drive, that can be used by multiple hosts to store data. Since lock control is typically implemented based on the smallest storage unit to avoid data read / write conflicts, the shared storage space targeted in the distributed lock implementation of this application is usually the smallest storage unit pointed to by a LUN. Each storage-side management node connects to a common shared storage space, but each storage-side logical node also has a LUN pointing to this shared storage space.
[0070] It should be understood that Figure 1 The number of hosts, storage-side management nodes, distributed devices, and shared storage spaces shown is merely illustrative. Depending on implementation needs, any number of hosts, storage-side management nodes, and shared storage spaces can be included.
[0071] Figure 2 This is a flowchart of the main method provided in the embodiments of this application. The method is composed of... Figure 1 The storage-side management node in the illustrated architecture executes. For example... Figure 2 As shown, the method may include the following steps:
[0072] Step 201: Receive the command sent by the active node requesting shared storage space from the host.
[0073] It should be noted that, in this embodiment, the storage-side management node that receives the host's command requesting shared storage space is designated as the active node, while other storage-side management nodes are designated as passive nodes. For a single storage-side management node, it may act as an active node in some situations and as a passive node in others.
[0074] Step 202: Based on the lock sharing information on the local storage management node, determine whether the relevant command is allowed. If allowed, proceed to step 203; otherwise, proceed to step 204.
[0075] Step 203: Update the above lock sharing information, and share the information that allows the relevant command with other storage-side management nodes so that other storage-side management nodes can update their local lock sharing information.
[0076] Step 204: Return a message to the host rejecting the relevant command.
[0077] As can be seen, after receiving a host requesting a shared storage space command, this application determines whether to allow the command by using the lock sharing information maintained by the storage-side management node. This lock sharing information is maintained through a lock information sharing mechanism with each storage-side management node, thereby ensuring that the host's operation on the shared storage space does not conflict.
[0078] In addition, the distributed lock in this application is not implemented on the storage device, but on the first layer (transmission bearer layer) on the storage side, which can be adapted to existing host and storage device hardware without hardware modification, thus reducing implementation costs.
[0079] The following is a detailed description of each step in the above process. First, step 201, namely "receiving the host request for shared storage space sent by the active node", will be described in detail with reference to the embodiment.
[0080] The distributed lock involved in this application can be applied to various protocols such as SCSI and NVMe (non-volatile memory express, host controller interface specification). In this embodiment, the SCSI protocol is used as an example. The commands related to the host requesting shared storage space mainly include register, reserve, preemption, and I / O commands. Originally, these commands were sent to the storage device; in this application, they are intercepted by the storage-side management node (active node).
[0081] Since the control of distributed locks such as SCSIPR (Persistent Reservation) locks mainly relies on the implementation of reserve and preemption commands, this step can first parse the type of the command received from the host requesting shared storage space. If the command is a reserve or preemption command, then proceed to the next step 202.
[0082] However, if the parsing determines that the relevant command is a registration command, since host registration is usually allowed and unaffected by conflicts, the registration command can be directly allowed, and the registration information can be shared with each passive node. The purpose of sharing the registration information with the passive nodes is to ensure that other storage-side management nodes also have the host's registration information. This allows other storage-side management nodes to process the host's reservation or preemption commands promptly. The specific process of the registration command will be described in detail in subsequent embodiments.
[0083] For I / O commands, a different logic is used to determine whether an I / O command is allowed. This involves analyzing whether the I / O command carries the assigned lock key. Only if the correct lock key is carried is the I / O command allowed; otherwise, it is rejected. The lock key is assigned after the host reserves or successfully preempts the lock, which will be detailed in subsequent embodiments.
[0084] The following describes step 202, namely "determining whether to allow the relevant command based on the lock sharing information of the local storage management node", in conjunction with the embodiments.
[0085] The lock sharing information (e.g., represented as PRinfo) involved in the embodiments of this application may include lock sharing information stored locally by each storage-side management node and lock sharing information stored locally by the distributed device.
[0086] When a storage-side management node receives a command from a host requesting shared storage space (such as a `reserve` or `preempt` command), it needs to determine the current lock status of that shared storage space. Each storage-side management node locally maintains lock sharing information, which includes the status information of the shared storage space. This status information can primarily include information such as whether it is registered, reserved, preempted, or available. Of course, other finer-grained or coarser-grained representations of the status information can also be used, which will not be listed here.
[0087] More specifically, the status information of shared storage space can be such as host id - LUN id - register / reserve / preemption information, or it can be such as LUN id - idle information. That is, it can record the identifier of the shared storage space, its status as registered, reserved, or preempted by the host, and the host identifier. If it is in an idle state, only the identifier of the shared storage space and its idle status need to be recorded. Other recording methods can also be used, which will not be listed here.
[0088] As one possible approach for the reserve or preemption command, if the shared storage space requested by the host has already been reserved or preempted by another host, the reserve or preemption command is not allowed; otherwise, the reserve or preemption command is allowed.
[0089] For example, host 1 sends a reserve command to node 1 requesting to reserve LUN1, but node 1 determines from its local PRinfo that LUN1 has already been reserved by host 2 or preempted, and therefore rejects the reserve command.
[0090] Since there are multiple storage-side management nodes in the system, on the one hand, the lock sharing information maintained by all storage-side management nodes needs to be unified, and on the other hand, each storage-side management node needs to know the current accurate lock status. Each time a host is allowed to request a command related to the shared storage space, the storage-side management node, as the active node, needs to share the latest lock information about that shared storage space with all other storage-side management nodes, so that each storage-side management node knows the latest lock information. Therefore, step 203 will be executed.
[0091] The following describes step 203, namely, "updating the above lock sharing information and sharing the information that allows the relevant command with each passive node," in conjunction with an embodiment.
[0092] If the active node determines to allow the aforementioned commands, in addition to updating its local lock sharing information, it also needs to share this permission information with each passive node. The active node can share this information by calling the interfaces of each passive node. Each passive node can then return a response message to the distributed system via callbacks. Each storage-side management node can be configured with a distributed module to facilitate communication between them. Communication between storage-side management nodes can be implemented using existing systems such as the Nuwa system, which will not be detailed here.
[0093] In addition, each storage-side management node can periodically provide the lock sharing information stored locally to the distributed device, either after being triggered or after receiving a request from the distributed device, so that the distributed device can serialize and save the lock sharing information.
[0094] As one feasible approach, when a new storage-side management node is added, it can obtain lock sharing information from the distributed device and store it locally. Alternatively, if a storage-side management node malfunctions and loses its locally stored lock sharing information or fails to update the lock sharing information in a timely manner, it can obtain the lock sharing information from the distributed device and store it locally.
[0095] Furthermore, the communication between storage-side management nodes in this step also simultaneously enables the detection of storage-side management node disconnection. For example, if the active node sends information allowing a relevant command to each passive node within a set timeframe and does not receive a response from that passive node, then the passive node is determined to be disconnected. Since storage-side management nodes can be disconnected, the accuracy of the current status information of the shared storage space cannot be determined. As a possible approach, if a storage-side management node is disconnected, the execution of the relevant command cannot be prohibited, and the active node can return a message rejecting the relevant command.
[0096] If the active node allows the host to execute the `reserve` or `preempt` command, the active node will assign a lock key to the host. Only the host holding the lock key can perform I / O operations on the shared storage space. The lock key can be a random string or a string generated according to certain rules. This application does not restrict the method of generating the lock key, as long as the uniqueness of the lock key is guaranteed.
[0097] The active node can return a message to the host that allows the relevant command, carrying the lock key that was assigned to the host.
[0098] Besides the methods of assigning lock keys described above, it is also possible not to assign a lock key to the host, but simply record locally on the active node that the host has successfully reserved or preempted the shared storage space. Upon receiving an I / O command from that host, the I / O command of host 1 is permitted.
[0099] Furthermore, after receiving the information that allows related commands, other storage-side management nodes, which are passive nodes, can update their local lock sharing information using this information as a possible approach.
[0100] However, as an alternative approach, after receiving information allowing a relevant command, if the relevant command is a reserve or preemption command, the passive node can first determine whether there is a conflict in the reserve or preemption command based on its local lock sharing information. For example, if allowing a reserve or preemption command means allowing host1 to reserve or preempt LUN1, but the lock sharing information stored locally by the passive node indicates that another host, such as host2, has already reserved or preempted LUN1, then the local lock sharing information does not need to be updated, and the response returned to the active node indicates that there is a command conflict.
[0101] If the relevant command is registration information, the passive node usually does not need to make a judgment and can directly store the registration information in the local lock sharing information.
[0102] To facilitate understanding of the distributed lock implementation in the entire system, Figure 3 Let's take an example to introduce the overall architecture. Figure 3 Taking three storage-side management nodes, node-A, node-B, and node-C, as an example, each node includes a request queue (RequestQueue) and a response queue (ResponseQueue). Taking node-A as an example, after receiving a command from a host requesting access to shared storage space, it sends the command to the request queue. Messages that deny or allow the command are sent to the host and then returned to the host in sequence.
[0103] The modules implementing distributed locks in the storage-side management nodes are represented by TS, specifically TS-A, TS-B, and TS-C in the diagram. Taking TS-A as an example, the Distribute module is used to enable communication between the various storage-side management nodes. The PR module is used to maintain local PR info. Local PR info can be implemented in RAM (Random Access Memory), enabling fast reading and updating of lock information.
[0104] The distributed device is responsible for serializing and saving the PR info of the storage-side management node.
[0105] The implementation processes of the register command, reserve command, preemption command, and I / O command are described below with reference to specific examples.
[0106] Figure 4 This is a flowchart illustrating the processing of the register command according to an embodiment of this application. Due to space limitations, this flowchart uses two storage-side management nodes, node1 and node2, as an example; other nodes are not shown. Figure 5 As shown, the process may include the following steps:
[0107] Step 401: Host 1 sends a register command to node 1. Assume that the register command is a command from host 1 to request the registration of LUN1.
[0108] Step 402: node1 calls the interface of the distributed module to send the register information to other storage-side management nodes, namely node2.
[0109] Step 403: Node2 saves the register information of host1 locally. For example, node2 can save the host1-LUN1-register information locally.
[0110] Step 404: node2 returns a response message. This response message indicates that the register information has been received.
[0111] Step 405: node1 saves the register information locally. For example, node2 can save the host1-LUN1-register information locally.
[0112] Step 406: node1 returns a message to host1 allowing registration.
[0113] It should be noted that steps 402 to 404 and step 405 can be performed simultaneously, or step 405 can be performed first and then steps 402 to 404. This application does not impose any restrictions on this.
[0114] After host 1 completes the registration process described above, it can be accessed via... Figure 5 The process shown requests reserved shared storage space. Figure 5 This is a flowchart illustrating the processing of the `reserve` command according to an embodiment of this application. Due to space limitations, this flowchart uses two storage-side management nodes, node1 and node2, as an example; other nodes are not shown. Figure 5 As shown, the process may include the following steps:
[0115] Step 501: Host 1 sends a reserve command to node 1, assuming that the reserve command contains information that Host 1 requests to reserve LUN1.
[0116] Step 502: node1 determines the status of LUN1 based on the lock sharing information maintained locally. If the lock sharing information indicates that LUN1 has been reserved or preempted by other hosts, the reserve command should be rejected and step 509 should be executed; otherwise, the reserve command should be allowed and step 503 should be executed.
[0117] Step 503: node1 updates the locally maintained lock sharing information.
[0118] That is, in the lock sharing information maintained locally, the host1-LUN1-reserve status is recorded, and information reserved through node1 can also be further recorded.
[0119] Step 504: node1 calls the interface of the distributed module to send the information that reserves is allowed to other storage-side management nodes, namely node2.
[0120] Step 505: Node2 updates the lock sharing information of the local storage, that is, updates the status of LUN1 in the lock sharing information of the local storage to the host1-LUN1-reserve status.
[0121] Normally, the lock sharing information between node2 and the distributed device is consistent. However, under certain abnormal circumstances, the state of LUN1 in the lock sharing information stored locally on node2 may have been reserved or preempted by other hosts, indicating a conflict with the current reserve command. In this case, it is not necessary to update the state of local LUN1.
[0122] Step 506: node2 returns a response message.
[0123] If node2 determines that there is no command conflict based on the lock sharing information stored locally, it can return a normal response message indicating that the shared information has been received.
[0124] If node2 determines that a command conflict exists based on the lock sharing information stored locally, then the command conflict can be detected in the response message.
[0125] If all passive nodes return normal response messages, a 507 error is executed. If any passive node returns a response indicating a command conflict, the reserve command is rejected, i.e., a 509 error is executed.
[0126] Step 507: node1 assigns a lock key to host 1.
[0127] It should be noted that the step of updating lock sharing information involved in step 503 above can also be performed in step 507.
[0128] Step 508: Node1 returns information to Host1 that allows the reserve command, which includes the lock key assigned to Host1.
[0129] The following are the steps when the reserve command is not allowed:
[0130] Step 509: node1 returns a message to host 1 rejecting the reserve command.
[0131] In the event of a command conflict, an exception handling mechanism can be activated to determine the current state of LUN1 and update the state of LUN1 on each node. This application does not limit the specific content of the exception handling mechanism; for example, manual intervention may be allowed.
[0132] If host 1 is denied reserve through the above process, it can attempt to preempt LUN1 through the preemption process. Alternatively, if host 1 passes the preemption process... Figure 4 After successful registration of the process shown, you can also directly attempt to preempt the process. The specific method used depends on the processing logic of host 1. Figure 6 This is a flowchart illustrating the preemption command processing provided in an embodiment of this application. Due to space limitations, this flowchart uses two storage-side management nodes, node1 and node2, as an example; other nodes are not shown. Figure 6 As shown, the process may include the following steps:
[0133] Step 601: Host 1 sends a preemption command to node 1, assuming that the preemption command contains information that Host 1 requests to reserve LUN1.
[0134] Step 602: node1 determines the status of LUN1 based on the lock sharing information maintained locally. If the lock sharing information indicates that LUN1 has been reserved or preempted by other hosts, the preemption command should be rejected and step 609 should be executed; otherwise, the preemption command should be allowed and step 603 should be executed.
[0135] In practical applications, other preemption logic can also be designed. For example, only one host can successfully preempt at a time based on its priority level, and which host succeeds in preemption is determined by the priority level of the preempting host. These are not listed here.
[0136] Step 603: node1 updates the locally maintained lock sharing information.
[0137] That is, in the lock sharing information maintained locally, the host1-LUN1-preemption status is recorded, and further information on whether the preemption was carried out through node1 can also be recorded.
[0138] Step 604: node1 calls the interface of the distributed module to send the information that preemption is allowed to other storage-side management nodes, namely node2.
[0139] Step 605: node2 updates the lock sharing information of the local storage, that is, updates the status of LUN1 in the lock sharing information of the local storage to host1-LUN1-preemption status.
[0140] Under normal circumstances, the lock sharing information between node2 and the distributed device is consistent. However, under certain abnormal conditions, the state of LUN1 in the lock sharing information stored locally on node2 may have been reserved or preempted by other hosts, indicating a conflict in the current preemption command. In this case, it is not necessary to update the state of local LUN1.
[0141] Step 606: node2 returns a response message.
[0142] If node2 determines that there is no command conflict based on the lock sharing information stored locally, it can return a normal response message indicating that the shared information has been received.
[0143] If node2 determines that a command conflict exists based on the lock sharing information stored locally, it can indicate the existence of a command conflict in the response message.
[0144] If all passive nodes return normal response messages, then execute 607. If any passive node returns a response indicating a command conflict, then the preemption command is rejected, i.e., execute 609.
[0145] Step 607: node1 assigns a lock key to host 1.
[0146] It should be noted that the step of updating lock sharing information involved in step 603 above can also be performed in step 607.
[0147] Step 608: Node1 returns information to Host1 that allows the preemption command, which carries the lock key assigned to Host1.
[0148] The following are the steps when preemption is not allowed:
[0149] Step 609: node1 returns a message to host 1 rejecting the preemption command.
[0150] After successfully reserving or preempting LUN1, host 1 can process I / O commands on LUN1. The I / O commands mainly include read and write commands. Figure 7 This is a flowchart of I / O command processing provided in an embodiment of this application. Due to space limitations, this flowchart uses two storage-side management nodes, node1 and node2, as an example; other nodes are not shown. Figure 7 As shown, the process may include the following steps:
[0151] Step 701: Host 1 sends a write command to node 1.
[0152] Step 702: node1 determines whether the write command carries the lock key allocated to host 1. If so, proceed to step 803; otherwise, proceed to step 805.
[0153] Step 703: node1 allows the write command and can forward the write command to the storage device for write operation.
[0154] Step 704: node1 returns a message to host 1 indicating that the write was successful.
[0155] Step 705: node1 returns a write failure message to host 1.
[0156] Additionally, if an error occurs when forwarding the write command to the storage device for write operation in step 703, a write failure message can also be returned to host 1.
[0157] Figure 7 The example used here is the write command; the process for reading commands is similar and will not be repeated here.
[0158] As can be seen from the above embodiments, the method provided in this application does not require any changes to the host and storage device hardware, can be fully adapted, reduces the cost of hardware modifications, and achieves complete transparency to the backend storage.
[0159] The aforementioned system and method can be applied to the core feature requirements of shared LUNs for enterprise storage users in hybrid cloud scenarios. Through software-level innovation, SCSIPR locks are supported at the storage-side transport layer using distributed locks to enable shared LUNs.
[0160] It should be noted that this application embodiment does not restrict the method of releasing shared storage space after it has been preempted or reserved. The shared storage space can transition to an idle state after being preempted or reserved.
[0161] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.
[0162] According to another embodiment, an implementation apparatus for a distributed lock is provided. Figure 8 A schematic block diagram of an implementation apparatus for a distributed lock according to one embodiment is shown. Figure 8 As shown, the device 800 includes: a first communication unit 801, a decision-making unit 802, a lock information maintenance unit 803, and a second communication unit 804, and may further include a type identification unit 805. The main functions of each component are as follows:
[0163] The first communication unit 801 is configured to receive commands from the host requesting shared storage space.
[0164] Decision unit 802 is configured to determine whether to allow relevant commands based on the lock sharing information local to the storage-side management node.
[0165] The lock information maintenance unit 803 is configured to update the lock sharing information if the decision unit 802 determines that the relevant command is allowed. The lock sharing information maintains the status information of the shared storage space, including the information of being registered, reserved, preempted, or idle.
[0166] The second communication unit 804 is configured to share information about enabling relevant commands with other storage-side management nodes.
[0167] The first communication unit 801 is further configured to return a message rejecting the relevant command to the host if the decision unit 802 does not allow the relevant command.
[0168] As one possible implementation, the type identification unit 805 is configured to identify the type of the relevant command. If the relevant command is a reserved command or a preemption command, the decision unit 802 is triggered to determine whether to allow the processing of the relevant command based on the lock sharing information local to the storage-side management node.
[0169] If the relevant command is a registration command, the decision unit 802 is triggered to determine whether to allow the registration command, and the second communication unit 804 shares the registration information with other storage-side management nodes.
[0170] As one possible implementation method, the decision unit 802 is specifically configured to: if the lock sharing information contains information that the shared storage space has been reserved or preempted by other hosts, then the reservation command or preemption command is not allowed; otherwise, the reservation command or preemption command is allowed.
[0171] Accordingly, when the lock information maintenance unit 803 updates the lock sharing information, it can update the information in the lock sharing information about the shared storage space reserved or preempted by the host that sent the relevant command.
[0172] Furthermore, if the decision unit 802 determines, based on the responses returned by other storage-side management nodes regarding the information allowing related commands, that a storage-side management node is disconnected or there is a command conflict, then it will not allow the reservation of commands or the preemption of commands.
[0173] As one possible implementation, the lock sharing information maintenance unit 803 is also used to allocate a lock key to the host if a reservation command or a preemption command is allowed, and return it to the host by the first communication unit 801 in the reservation or preemption information.
[0174] In this case, if an I / O command is received, the decision unit 802 determines whether the I / O command carries the assigned lock key. If so, the I / O command is allowed; otherwise, the I / O command is rejected.
[0175] As an alternative approach, the storage-side management node can also act as a passive node. In this case, if the second communication unit 804 receives information about enabling related commands shared by other storage-side management nodes, it will return a response.
[0176] If the information of the relevant command does not conflict with the lock sharing information stored locally, the lock information maintenance unit 803 can update the lock sharing information stored locally; otherwise, the second communication unit 804 will carry information about command conflicts in the response.
[0177] As an alternative approach, the system may also include a distributed device. In this case, the lock information maintenance unit 803 can send the locally stored lock sharing information to the distributed device for serialization and storage through the third communication unit 806, so that the newly added storage-side management node can obtain the lock sharing information from the distributed device and store it locally.
[0178] It should be noted that the embodiments of this application may involve the use of user data. In practical applications, user-specific personal data may be used in the scheme described herein within the scope permitted by applicable laws and regulations, provided that it complies with the applicable laws and regulations of the country (e.g., with the user's explicit consent, with the user being properly notified, etc.).
[0179] In addition, embodiments of this application also provide a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the method described in any of the foregoing method embodiments.
[0180] And an electronic device, comprising:
[0181] One or more processors; and
[0182] A memory associated with the one or more processors, the memory being used to store program instructions that, when read and executed by the one or more processors, perform the steps of the method described in any of the foregoing method embodiments.
[0183] in, Figure 9 An exemplary architecture of an electronic device is shown, which may include a processor 910, a video display adapter 911, a disk drive 912, an input / output interface 913, a network interface 914, and a memory 920. The processor 910, video display adapter 911, disk drive 912, input / output interface 913, network interface 914, and memory 920 can communicate with each other via a communication bus 930.
[0184] The processor 910 can be implemented using a general-purpose CPU, microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits to execute relevant programs and implement the technical solution provided in this application.
[0185] The memory 920 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 920 can store the operating system 921 for controlling the operation of the electronic device 900, and the basic input / output system (BIOS) 922 for controlling the low-level operations of the electronic device 900. Additionally, it can store a web browser 923, a data storage management system 924, and a distributed device / storage-side management node 925, etc. The aforementioned distributed device / storage-side management node 925 can be the application program that specifically implements the aforementioned steps in this embodiment. In summary, when implementing the technical solution provided in this application through software or firmware, the relevant program code is stored in the memory 920 and executed by the processor 910.
[0186] Input / output interface 913 is used to connect input / output modules to realize information input and output. Input / output modules can be configured as components in the device (not shown in the figure) or externally connected to the device to provide corresponding functions. Input devices may include keyboards, mice, touch screens, microphones, various sensors, etc., and output devices may include displays, speakers, vibrators, indicator lights, etc.
[0187] Network interface 914 is used to connect a communication module (not shown in the figure) to enable communication between this device and other devices. The communication module can communicate via wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).
[0188] The bus includes a pathway for transmitting information between various components of the device (e.g., processor 910, video display adapter 911, disk drive 912, input / output interface 913, network interface 914, and memory 920).
[0189] It should be noted that although the above-described device only shows the processor 910, video display adapter 911, disk drive 99, input / output interface 913, network interface 914, memory 920, bus 930, etc., in specific implementations, the device may also include other components necessary for normal operation. Furthermore, those skilled in the art will understand that the above-described device may only include the components necessary for implementing the solution of this application, and does not necessarily include all the components shown in the figures.
[0190] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments of this application.
[0191] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for system or system embodiments, since they are basically similar to method embodiments, the description is relatively simple, and relevant parts can be referred to the descriptions in the method embodiments. The systems and system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.
[0192] The technical solutions provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.
Claims
1. A method for implementing a distributed lock, applied to a system comprising multiple hosts, multiple storage-side management nodes, and shared storage space; the method includes: The storage-side management node receives commands from the host requesting the shared storage space. Based on the lock sharing information local to the storage-side management node, determine whether to allow the relevant command; If the relevant command is allowed, the lock sharing information is updated, and the information allowing the relevant command is shared with other storage-side management nodes so that the lock sharing information of other storage-side management nodes can be updated locally. The lock sharing information maintains the status information of the shared storage space, including information on registration, reservation, preemption, or idle status. Before determining whether to allow the relevant command based on the lock sharing information local to the storage-side management node, the process also includes: Identify the type of the relevant command; If the relevant command is a reserved command or a preemption command, then continue to execute the step of determining whether to allow the relevant command based on the lock sharing information of the local storage management node; If the relevant command is a registration command, then the registration command is allowed, and the registration information is shared with other storage-side management nodes; If, based on the responses from other storage-side management nodes regarding the information allowing the relevant commands, it is determined that a storage-side management node is out of contact or that there is a command conflict, then the reserved command or preemption command is not allowed.
2. The method according to claim 1, wherein, Based on the lock sharing information local to the storage-side management node, determine whether to allow the relevant commands, including: If the lock sharing information contains information that the shared storage space has been reserved or preempted by other hosts, then the reservation command or preemption command is not allowed; otherwise, the reservation command or preemption command is allowed. Updating the lock sharing information includes updating the information in the lock sharing information regarding whether the shared storage space was reserved or preempted by the host that sent the relevant command.
3. The method according to claim 1, further comprising: If the reservation command or preemption command is allowed, a lock key is assigned to the host.
4. The method according to claim 3, wherein, The method also includes: If an I / O command is received, it is determined whether the I / O command carries the assigned lock key. If so, the I / O command is allowed; otherwise, the I / O command is rejected.
5. The method according to claim 1, further comprising: If information allowing related commands is received from other storage-side management nodes, a response is returned; If the information of the relevant command does not conflict with the lock sharing information stored locally, the lock sharing information stored locally is updated; otherwise, the response carries information indicating a command conflict.
6. The method according to any one of claims 1 to 5, further comprising: If the relevant command is not allowed, a message denying the relevant command is returned to the host.
7. The method according to any one of claims 1 to 5, wherein the system further comprises a distributed device, and the method further comprises: The storage-side management node sends the lock-sharing information from its local storage to the distributed device for serialization and storage, so as to facilitate... The newly added storage-side management node in the system obtains lock sharing information from the distributed device and stores it locally.
8. A distributed lock implementation device, applied to a system comprising multiple hosts, multiple storage-side management nodes, distributed devices, and shared storage space, wherein the device is disposed at the storage-side management node, comprising: The first communication unit is configured to receive commands from the host requesting the shared storage space. The decision unit is configured to determine whether to allow the relevant command based on the lock sharing information local to the storage-side management node; A lock information maintenance unit is configured to update the lock sharing information if the decision unit determines that the relevant command is allowed, wherein the lock sharing information maintains the status information of the shared storage space, and the status information includes registered information, reserved information, preempted information, or idle information; The second communication unit is configured to share information that allows the relevant commands with other storage-side management nodes; Before determining whether to allow the relevant command based on the lock sharing information local to the storage-side management node, the decision unit is further configured to: Identify the type of the relevant command; If the relevant command is a reserved command or a preemption command, then continue to execute the step of determining whether to allow the relevant command based on the lock sharing information of the local storage management node; If the relevant command is a registration command, then the registration command is allowed, and the registration information is shared with other storage-side management nodes; If, based on the responses from other storage-side management nodes regarding the information allowing the relevant commands, it is determined that a storage-side management node is out of contact or that there is a command conflict, then the reserved command or preemption command is not allowed.
9. A distributed lock implementation system, comprising multiple hosts, multiple storage-side management nodes, and shared storage space; The host is configured to send commands to the storage-side management node requesting the shared storage space; The storage-side management node is configured to receive commands from the host requesting the shared storage space. Based on the lock sharing information local to the storage-side management node, determine whether to allow the relevant command; If the relevant command is allowed, the lock sharing information is updated, and the information allowing the relevant command is shared with other storage-side management nodes; wherein, the lock sharing information maintains the status information of the shared storage space, and the status information includes information on registration, reservation, preemption, or idle status; Before determining whether to allow the relevant command based on the lock sharing information local to the storage-side management node, the storage-side management node is further configured as follows: Identify the type of the relevant command; If the relevant command is a reserved command or a preemption command, then continue to execute the step of determining whether to allow the relevant command based on the lock sharing information of the local storage management node; If the relevant command is a registration command, then the registration command is allowed, and the registration information is shared with other storage-side management nodes; If, based on the responses from other storage-side management nodes regarding the information allowing the relevant commands, it is determined that a storage-side management node is out of contact or that there is a command conflict, then the reserved command or preemption command is not allowed.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps of the method according to any one of claims 1 to 7.
11. An electronic device, characterized in that, include: One or more processors; as well as A memory associated with the one or more processors, the memory being used to store program instructions that, when read and executed by the one or more processors, perform the steps of the method according to any one of claims 1 to 7.
Citation Information
Patent Citations
Distributed lock management in a cloud computing environment
CN105283838A
Concurrent processing method and device for distributed cluster
CN111737020A