Partition allocation method and apparatus, and distributed system and electronic device

Abstract

Provided in the embodiments of the present disclosure are a partition allocation method and apparatus, and a distributed system and an electronic device. The method comprises: receiving a correspondence between a first partition of a distributed system and a first slave node thereof that is sent by a master node; after a lock of the first partition is acquired from a lock server, sending a lease acquisition request to the master node, wherein the lease acquisition request comprises an identifier of the first partition, a first lease identifier, and a first timeout period; and in response to a lease acquisition success response returned by the master node, determining a lease term of a lease of the first partition as the current moment plus the first timeout period, which lease corresponds to the first lease identifier, and loading the first partition, wherein either of the lock of the first partition and the lease of the first partition is used by the first slave node to execute an IO operation on the first partition. Therefore, the reliability and stability of a distributed system are improved.

Description

Partitioning methods, devices, distributed systems and electronic equipment

[0001] This disclosure claims priority to Chinese Patent Application No. 202411187712.9, filed with the China Patent Office on August 27, 2024, entitled “Partition Allocation Method, Apparatus, Distributed System and Electronic Equipment”, the entire contents of which are incorporated herein by reference. Technical Field

[0002] This disclosure relates to the field of cloud computing, and more specifically, to partitioning methods, apparatus, distributed systems, and electronic devices. Background Technology

[0003] The master-slave model is a common distributed system architecture. In this model, the master node coordinates and manages the allocation of resources across the entire system, while the slave nodes (servers) load the resources allocated by the master node. A common form of resource allocation in distributed systems is partitioning. Partitioning divides the data space into smaller blocks to facilitate the distribution and processing of tasks on that data within the distributed system. Partitions can be loaded by slave nodes, enabling parallel processing of tasks.

[0004] To ensure the mutual exclusion of partitions and the security of data during scheduling, a distributed system must guarantee that a single partition is loaded and processed by only one slave node at a time to avoid conflicts during data writing. Typically, a distributed lock can achieve this. The distributed lock service ensures that the same lock is not issued to multiple slave nodes; a slave node must first "grab" the lock for the corresponding partition from the external distributed lock service before it can load the partition or perform input / output (I / O) operations.

[0005] While distributed locks are an effective mechanism, they also introduce external dependencies. If the distributed lock service fails, all operations that depend on the lock will be interrupted, resulting in poor reliability and stability of the distributed system. Summary of the Invention

[0006] This disclosure provides partitioning allocation methods, apparatus, distributed systems, and electronic devices that improve the reliability and stability of distributed systems.

[0007] In a first aspect, this disclosure provides a memory processing method applied to a first slave node of a distributed system, the distributed system including a master node and multiple slave nodes, wherein the first slave node is any one of the multiple slave nodes, the method comprising:

[0008] Receive the correspondence between the first partition and the first slave node of the distributed system sent by the master node;

[0009] After obtaining the lock of the first partition from the lock server, a lease acquisition request is sent to the master node. The lease acquisition request includes the identifier of the first partition, the first lease identifier, and the first timeout time.

[0010] In response to the lease acquisition success response returned by the master node, the lease term of the first partition corresponding to the first lease identifier is determined as the current time plus the first timeout time, and the first partition is loaded. Either the lock of the first partition or the lease of the first partition is used by the first slave node to perform input / output I / O operations on the first partition.

[0011] Secondly, this disclosure provides a partitioning allocation method applied to the master node of a distributed system, the distributed system including the master node and multiple slave nodes, the method comprising:

[0012] Send the correspondence between the first partition and the first slave node of the distributed system to the first slave node;

[0013] Receive a lease acquisition request sent by the first slave node, wherein the lease acquisition request includes the identifier of the first partition, the first lease identifier, and the first timeout period;

[0014] If the first partition is not bound to any other lease identifier besides the first lease identifier, or if the first partition is bound to any other lease identifier besides the first lease identifier but the lease corresponding to the other lease identifier has expired, then a lease acquisition success response is returned to the first slave node, the first partition and the first lease identifier are bound, and the lease term of the lease corresponding to the first lease identifier stored by the master node is determined according to the first timeout time.

[0015] Thirdly, this disclosure provides a partitioning allocation device applied to a first slave node of a distributed system, the distributed system including a master node and multiple slave nodes, the device comprising:

[0016] The receiving module is used to receive the correspondence between the first partition and the first slave node of the distributed system sent by the master node;

[0017] The request module is used to send a lease acquisition request to the master node after acquiring the lock of the first partition from the lock server. The lease acquisition request includes the identifier of the first partition, the first lease identifier, and the first timeout time.

[0018] The update module is used to respond to the lease acquisition success response returned by the master node, determine the lease period of the first partition corresponding to the first lease identifier as the current time plus the first timeout time, and load the first partition, wherein either the lock of the first partition or the lease of the first partition is used by the first slave node to perform input / output I / O operations on the first partition.

[0019] Fourthly, this disclosure provides a partitioning allocation device applied to the master node of a distributed system, the distributed system including the master node and multiple slave nodes, the device comprising:

[0020] The sending module is used to send the correspondence between the first partition and the first slave node of the distributed system to the first slave node;

[0021] The receiving module is configured to receive a lease acquisition request sent by the first slave node, wherein the lease acquisition request includes the identifier of the first partition, the first lease identifier, and the first timeout period;

[0022] The response module is configured to return a lease acquisition success response to the first slave node if the first partition is not bound to any other lease identifier besides the first lease identifier, or if the first partition is bound to any other lease identifier besides the first lease identifier but the lease corresponding to the other lease identifier has expired, bind the first partition and the first lease identifier, and determine the lease term of the lease corresponding to the first lease identifier stored by the master node based on the first timeout time.

[0023] Fifthly, this disclosure provides a distributed system, including: a master node and a plurality of slave nodes; wherein the master node is used to perform the method as described in the second aspect, and the slave nodes are used to perform the method as described in the first aspect.

[0024] In a sixth aspect, this disclosure provides an electronic device, including: a memory and a processor;

[0025] The memory is used to store computer programs;

[0026] The processor is configured to execute a computer program stored in the memory, wherein the computer program, when executed, causes the processor to perform the method described in the first aspect or the second aspect.

[0027] In a seventh aspect, this disclosure provides a computer-readable storage medium storing a computer program that, when executed by a processor, causes the processor to perform the method as described in the first or second aspect.

[0028] Eighthly, this disclosure provides a computer program product including a computer program that, when executed by a processor, implements the method described in the first aspect.

[0029] The partition allocation method, apparatus, distributed system, and electronic device provided in this disclosure add a lease mechanism in addition to locks to the distributed system. The lease has a similar function to the lock, which can ensure the mutual exclusion of partitions. At the same time, the lease mechanism serves as a buffer guarantee for the lock service. As long as either the lock or the lease has not expired, the slave node can maintain control over the partition. The lease is maintained by the master node of the distributed system and does not depend on the outside, thus avoiding the interruption of lock-dependent operations due to the failure of the external lock service, thereby improving the reliability and stability of the distributed system. Attached Figure Description

[0030] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0031] Figure 1 is a schematic flowchart of a partition allocation method provided in an embodiment of this disclosure;

[0032] Figure 2 is a schematic diagram of a partition loading and unloading process provided in an embodiment of this disclosure;

[0033] Figure 3 is a schematic diagram of a lease refresh provided in an embodiment of this disclosure;

[0034] Figure 4 is a schematic diagram of a lease refresh and acquisition provided in an embodiment of this disclosure;

[0035] Figure 5 is a schematic diagram of the structure of a partition allocation device provided in an embodiment of this disclosure;

[0036] Figure 6 is a schematic diagram of the structure of a partition allocation device provided in an embodiment of this disclosure;

[0037] Figure 7 is a schematic block diagram of an electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0038] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0039] To reduce the dependence of the distributed system on external distributed lock services, this embodiment adds a lease mechanism that collaborates with the lock service to the distributed system architecture. The lease has similar functions to the lock, and it can also ensure the mutual exclusion of partitions. Each partition can only be held by one slave node at a time. At the same time, the lease mechanism serves as a buffer guarantee for the lock service. Even if the lock service is unavailable, the lease can maintain the slave node's control over the partition, thereby ensuring the reliability and stability of the distributed system.

[0040] Figure 1 is a flowchart illustrating a partition allocation method provided in an embodiment of this disclosure. This method is applied to a master node and a first slave node in a distributed system. The distributed system includes the master node and multiple slave nodes, where the first slave node is any one of the multiple slave nodes. As shown in Figure 1, the method includes:

[0041] S101. The master node sends the correspondence between the first partition and the first slave node of the distributed system to the first slave node.

[0042] In a distributed system, partition allocation is determined by the master node. The master node establishes the mapping between each partition and each slave node. For the first slave node, if the master node assigns the first partition to it, it sends the mapping between the first partition and the first slave node to that node. Partition allocation by the master node can be dynamic. Regardless of the allocation result, the master node only needs to send the mapping between partitions and slave nodes to the corresponding slave nodes. For example, if the master node previously assigned the second partition to the first slave node, and in this allocation, it assigns the first partition to the first slave node, then the master node only needs to send the mapping between the first partition and the first slave node to confirm that it has been assigned the first partition.

[0043] S102. After the first slave node acquires the lock of the first partition from the lock server, it sends a lease acquisition request to the master node. The lease acquisition request includes the identifier of the first partition, the identifier of the first lease, and the first timeout period.

[0044] After receiving the correspondence between the first partition of the distributed system and the first slave node sent by the master node, the first slave node determines that the partition that can be used later is the first partition. Therefore, the first slave node needs to acquire the lock and lease of the first partition. The first slave node first requests the lock of the first partition from the lock server. The lock server can be any distributed lock service server. This embodiment of the disclosure is not limited to this. The distributed lock can adopt any of the related technologies, which will not be elaborated here.

[0045] After acquiring the lock on the first partition, the first slave node sends a lease acquisition request to the master node to obtain a lease for the first partition. In this embodiment of the disclosure, the leases of the partitions in the distributed system are maintained by the master node, not by other external services. The identifier of the first partition is used to indicate to the master node that a lease for the first partition is required, while the first lease identifier indicates the identifier of the lease for the first partition generated by the first slave node. For example, the first lease identifier can be composed of a unique identifier generated by the first slave node and the address of the first slave node. The first timeout indicates the timeout period of the lease requested by the first slave node.

[0046] S103. After receiving the lease acquisition request sent by the first slave node, if the first partition is not bound to any other lease identifier besides the first lease identifier, or if the first partition is bound to any other lease identifier besides the first lease identifier but the lease corresponding to the other lease identifier has expired, the master node returns a lease acquisition success response to the first slave node, binds the first partition to the first lease identifier, and determines the lease term of the lease corresponding to the first lease identifier stored by the master node according to the first timeout time.

[0047] After receiving the lease acquisition request sent by the first slave node, the master node needs to determine whether the first partition has been allocated based on the identifier of the first partition carried in the request.

[0048] If the first partition is not bound to any other lease identifier besides the first lease identifier, or if the first partition is bound to any other lease identifier besides the first lease identifier but the leases corresponding to those other lease identifiers have expired, then the first partition is currently in a state where it can be allocated. Therefore, the master node returns a lease acquisition success response to the first slave node, and the master node binds the first partition to the first lease identifier to indicate that the first partition has been allocated a lease. At the same time, the master node also maintains the lease term of the lease corresponding to the first lease identifier. The duration of this lease term is determined based on the first timeout time, which will be further explained in subsequent embodiments.

[0049] In addition, if the first partition has been bound to the first lease identifier, or if the first partition has been bound to other lease identifiers besides the first lease identifier and the leases corresponding to the other lease identifiers have not expired, or if the master node is in a quiet period, then a lease acquisition failure response is returned to the first slave node. Here, the quiet period of the master node refers to the preset time period after the master node of the distributed system is switched.

[0050] If the first partition is already bound to the first lease identifier, it means that the first partition has already been assigned to the first slave node. The slave node's lease acquisition request is a duplicate request, therefore the master node returns a lease acquisition failure response. If the first partition is bound to a lease identifier other than the first lease identifier, and the leases corresponding to those other lease identifiers have not expired, it means that the first partition is currently still occupied by another slave node. In this case, the first partition cannot be assigned to the first slave node. For example, if the first partition was assigned to another slave node before the master node assigned it, and that the first slave node's request to acquire the first partition's lease has not yet been completed by that slave node, the first slave node's request will fail. The first slave node can resend the lease acquisition request until the other slave node successfully unloads the first partition, at which point the first slave node will receive a lease acquisition success response from the master node. The master node being in a silent period occurs when a master node switch happens. To prevent the new master node from violating the old master node's lease, the new master node needs to maintain a silent period after startup. During this silent period, the master node does not allocate or schedule partitions, and returns a failure response for all lease acquisition requests. In this situation, the first slave node can also send lease acquisition requests again until the master node's silent period expires, at which point the master node can process lease acquisition requests normally.

[0051] S104. The first slave node responds to the lease acquisition success response returned by the master node, determines the lease period of the first partition corresponding to the first lease identifier as the current time plus the first timeout time, and loads the first partition. Either the lock of the first partition or the lease of the first partition is used by the first slave node to perform IO operations on the first partition.

[0052] If the master node returns a successful lease acquisition response, meaning the first slave node successfully acquires the lease for the first partition, the first slave node determines the lease duration of the first partition corresponding to the first lease identifier as the current time plus the first timeout period. In other words, the lease timeout period is obtained by adding the first timeout period to the current time. After acquiring the lease for the first partition, the first slave node can load the first partition. It can be understood that after loading the first partition, the first slave node can also request a refresh of the lock timeout period and the lease timeout period, thereby maintaining control over the first partition.

[0053] After the first slave node acquires the lock and lease of the first partition and loads the first partition, it can perform I / O operations on the first partition. Either the lock or the lease of the first partition can guarantee that the first slave node has control over the first partition. In other words, if either the lock or the lease of the first partition has not expired, the first slave node can perform I / O operations on the first partition.

[0054] In the method of this disclosure embodiment, a lease mechanism in addition to locks is added to the distributed system. The lease has a similar function to the lock. The lease can also ensure the mutual exclusion of partitions. Each partition can only be held by one slave node at a time. At the same time, the lease mechanism serves as a buffer guarantee for the lock service. As long as either the lock or the lease has not expired, the slave node can maintain control over the partition. The lease is maintained by the master node of the distributed system and does not depend on the outside. This avoids the interruption of lock-dependent operations due to the failure of the external lock service, thereby improving the reliability and stability of the distributed system.

[0055] Based on the above embodiment, before the first slave node performs input / output I / O operations on the first partition, it checks whether the lock and lease of the first partition have timed out. If either the lock or the lease of the first partition has not timed out, the I / O operation is performed.

[0056] Before performing an I / O operation on the first partition, the first slave node needs to check whether its lock and lease on the first partition have timed out. As long as either the lock or the lease has not timed out, the first slave node can perform the I / O operation. In other words, the lease mechanism serves as a buffer guarantee for the lock service. Thus, even if the lock service is unavailable, as long as the first slave node's lease on the first partition still exists, the first slave node can maintain control over the first partition, thereby ensuring that the I / O operation can be performed normally.

[0057] The master node's partition allocation has been described in the above embodiments. Assuming that before this partition allocation, the master node allocated the second partition to the first slave node (i.e., the second partition was already loaded by the first slave node), and in this allocation, the master node allocates the first partition to the first slave node, then the first slave node, in addition to requesting the lock and lease for the first partition and loading the first partition, also needs to unload the second partition. That is, the first slave node unloads the second partition of the distributed system and sends a second lease release request to the master node. The second lease release request includes the identifier of the second partition and the second lease identifier. The second lease identifier corresponds to the lease of the second partition and is generated by the first slave node when requesting the lease for the second partition from the master node. Based on the identifier of the second partition, the master node determines that the second partition and the second lease identifier are bound, and then returns a lease release success response to the first slave node, releasing the binding between the second partition and the second lease identifier, i.e., releasing the lease between the second partition and the first slave node. After receiving the lease release success response from the master node, the first slave node requests the lock server to release the lock of the second partition, thereby releasing its occupation of the second partition.

[0058] Similarly, assuming the master node allocated the first partition to other slave nodes before this partition allocation, after this allocation, the other slave nodes need to unload the first partition. Only after the other slave nodes successfully unload the first partition and release their leases and locks can the first slave node successfully acquire the lock and lease for the first partition and load it. As illustrated in Figure 2, the master node sends the mapping relationship between partitions and slave nodes to the first slave node and other slave nodes respectively. While the other slave nodes are unloading the first partition and releasing their leases and locks, the first slave node will attempt to acquire the lock for the first partition. Only after the other slave nodes release their locks can the first slave node successfully acquire the lock for the first partition, further acquire the lease for the first partition, and load it.

[0059] Based on the above embodiments, the first slave node can apply for a lease extension through a lease refresh request.

[0060] The first slave node sends a lease refresh request to the master node at preset time intervals. The lease refresh request includes the identifier of the first partition, the first lease identifier, and a second timeout period. Upon receiving the lease refresh request from the first slave node, if the master node is in a silent period and the lease refresh request is the first for the first partition within that period, or if the first partition is bound to the first lease identifier, the master node returns a lease refresh success response to the first slave node and updates the lease duration corresponding to the first lease identifier stored by the master node according to the second timeout period. If the first partition is not bound to any lease identifier, or if the first partition is bound to a lease identifier other than the first lease identifier, the master node returns a lease refresh failure response to the first slave node. In response to the master node's successful lease refresh response, the first slave node updates the lease duration of the first partition corresponding to the first lease identifier to the current time plus the second timeout period. Through the lease refresh request, the first slave node can extend its control over the first partition, ensuring the normal execution of I / O operations.

[0061] During the quiet period, the master node returns a success response for the first lease refresh request for any partition. Subsequent lease refresh requests for that partition are determined based on whether the first partition is bound to another partition. If the first partition is bound to a first lease identifier, the first slave node can refresh the lease corresponding to the first lease identifier, and therefore, the master node returns a successful lease refresh response to the first slave node. If the first partition is not bound to any lease identifier, or if the first partition is bound to a lease identifier other than the first lease identifier, the first slave node cannot refresh the lease corresponding to the first lease identifier, and therefore, the master node returns a failed lease refresh response to the first slave node. For example, if the first slave node acquires a lease for the first partition, and after a period of time, the lease times out, and then the master node reassigns the first partition to another slave node, the first slave node will receive a failed response from the master node when it sends a lease refresh request again.

[0062] Based on the above embodiments, the first slave node can also release the lease through a lease release request.

[0063] The first slave node sends a first lease release request to the master node. The first lease release request includes the identifier of the first partition and the first lease identifier. After receiving the first lease release request from the first slave node, if the first partition is bound to the first lease identifier, the master node returns a lease release success response to the first slave node. If the first partition is not bound to any lease identifier, or if the first partition is bound to a lease identifier other than the first lease identifier, the master node returns a lease release failure response to the first slave node. The first slave node receives either the lease release success response or the lease release failure response sent by the master node.

[0064] If the first partition is bound to the first lease identifier, the first slave node can release the lease corresponding to the first lease identifier. Therefore, the master node returns a lease release success response to the first slave node. If the first partition is not bound to any lease identifier, or if the first partition is bound to a lease identifier other than the first lease identifier, the first slave node cannot release the lease corresponding to the first lease identifier. Therefore, the master node returns a lease release failure response to the first slave node. For example, after the first slave node acquires the lease for the first partition, the lease times out after a period of time. After that, the master node assigns the first partition to another slave node. If the first slave node then sends a lease release request, it will receive a failure response from the master node.

[0065] Based on the above embodiments, the various times in the embodiments of this disclosure will be described.

[0066] Referring to Figure 3, every preset time interval T i The first slave node initiates a lease refresh request. The response to this request is only valid if it returns within the timeout period of the previous lease term. Conversely, if the response does not return after the timeout period, it means the first slave node has lost the lease. Upon receiving the lease refresh request, the master node extends the lease term. If no new lease refresh request is received after the timeout period, the master node considers the lease invalid and can delete it. Assuming the second timeout period is the same as the first timeout period, the following relationship can be obtained from Figure 3:

[0067] T2′>T1′>T0′;

[0068] 0 <T2′-T0′≤T L ;

[0069] Among them, T L This represents the first timeout period, which can be obtained from the above formula:

[0070] 0 <T2′-T1′≤T L ;

[0071] To maintain mutual exclusion among partitions, the timeout period of the lease maintained by the master node must be greater than the timeout period of the lease maintained by the first slave node. Therefore:

[0072] T2′+T L ≤T1′+T L ′

[0073] Among them, T L ′ represents the timeout period for the lease maintained by the master node, which can be obtained from the above:

[0074] T2′-T1′≤T L ′-T L

[0075] Since the maximum value of T2′-T1′ tends to T L Therefore, T L ≥2T L In other words, the lease term corresponding to the first lease identifier stored on the master node is greater than or equal to the current time plus twice the first timeout period.

[0076] The preset time interval for refreshing the lease is still shown in Figure 3.

[0077] T L ≥T2′-T2=T i -(T2-T0)+(T2′-T0′);

[0078] Among them, T i This represents the preset time interval for refreshing the lease. Since T2-T0 is at least close to 0, and T2′-T0′ is at most the timeout period of the Remote Procedure Call (RPC) of the first slave node, therefore...

[0079] T L >T i +T R ;

[0080] Among them, T R This indicates the remote procedure call timeout period for the first slave node. In other words, the preset time interval is less than the first timeout period minus the remote procedure call timeout period for the first slave node.

[0081] Regarding the master node's quiet period, refer to Figure 4 and consider the extreme case: At time T1, the master node switches over, and a new master node is successfully elected at time T′, after which the master node enters a quiet period. Furthermore, the master node successfully processes the first slave node's lease refresh request precisely at time T1. Assume that the first slave node remains network-disconnected from the master node after T2, and initiates its final lease refresh request at time T3, just before its lease expires. Other slave nodes continuously initiate lease acquisition requests for the first partition. To ensure partition mutual exclusion, as shown in the diagram:

[0082] T S =T4-T′=T2-T′+T L +T R ;

[0083] Among them, T S This represents the silent period of the master node. Since T′ approaches T0 at its minimum, therefore:

[0084] T S ≥T2-T0+T L +T R ;

[0085] Since the maximum value of T2-T0 is T R ,therefore:

[0086] T S ≥T L +2T R ;

[0087] That is, the duration of the master node's quiet period is greater than or equal to the sum of the first timeout time and twice the remote procedure call timeout time of the first slave node.

[0088] Based on the above embodiments, the method of this disclosure can be further optimized for the availability of distributed systems.

[0089] In actual operation, slave nodes may enter a suspended state due to high load, hardware failure, or other reasons, but their lease refresh threads remain active, continuing to maintain the leases for their partitions. In this situation, although the slave node is actually unable to provide normal service, other slave nodes cannot acquire the lease for that partition, causing the partition to become unusable. To solve this problem, this embodiment adds an operation and maintenance interface for the master node, providing a lease blacklisting function. This interface performs a barrier operation on the partition's lease to prevent its renewal. When the lease expires naturally, other slave nodes can safely compete for and acquire the partition's lease. For the master node, if it receives a lease barrier request, which includes the identifier of the first partition and the identifier of the first lease, it returns a lease refresh failure response to the first slave node after receiving the lease refresh request sent by the first slave node, until the lease corresponding to the first lease identifier times out. By barriering the lease corresponding to the first lease identifier of the first slave node, the occupation of the first partition can be avoided in the event of a failure of the first slave node, improving the availability of the system.

[0090] In distributed system upgrade scenarios, after a slave node upgrades and restarts, it needs to reacquire leases for the partition. However, if the lease from before the upgrade has not yet expired, it needs to wait for it to expire, which also affects system availability. To solve this problem, this embodiment introduces an operation and maintenance interface for the master node, providing a lease deletion function. The interface operates as follows: When a slave node successfully acquires a lease, it stores a file containing the identifier of the process corresponding to the lease in the slave node. This file should also contain lease information. After the slave node restarts, it first checks whether the process corresponding to the file still exists. If it does not exist, it reads the lease information from the file and sends a lease deletion request to the master node with the lease information.

[0091] For the first slave node, in response to the master node's successful lease acquisition response, it stores the identifier of the process corresponding to the lease for the first partition. If the first slave node restarts and the process corresponding to the lease for the first partition does not exist, it sends a lease deletion request to the master node, including the identifier of the first partition and the identifier of the first lease. These measures improve system failover efficiency and ensure the effective management and efficient utilization of mutually exclusive resources.

[0092] Figure 5 is a schematic diagram of a partition allocation device provided in an embodiment of this disclosure. As shown in Figure 5, the partition allocation device 500 includes:

[0093] The receiving module 501 is used to receive the correspondence between the first partition and the first slave node of the distributed system sent by the master node.

[0094] The request module 502 is used to send a lease acquisition request to the master node after acquiring the lock of the first partition from the lock server. The lease acquisition request includes the identifier of the first partition, the first lease identifier, and the first timeout time.

[0095] The update module 503 is used to respond to the lease acquisition success response returned by the master node, determine the lease period of the first partition corresponding to the first lease identifier as the current time plus the first timeout time, and load the first partition. Either the lock of the first partition or the lease of the first partition is used by the first slave node to perform IO operations on the first partition.

[0096] One implementation also includes:

[0097] The execution module is used to check whether the locks and leases of the first partition have timed out before the first slave node performs input / output I / O operations on the first partition. If either the lock or the lease of the first partition has not timed out, the I / O operation is performed.

[0098] One implementation also includes:

[0099] The refresh module is used to send lease refresh requests to the master node at preset time intervals. The lease refresh request includes the identifier of the first partition, the identifier of the first lease, and the second timeout.

[0100] In response to the lease refresh success message returned by the master node, the lease term of the lease in the first partition corresponding to the first lease identifier is updated to the current time plus the second timeout period.

[0101] In one implementation, the preset time interval is less than the first timeout time minus the remote procedure call timeout time of the first slave node.

[0102] One implementation also includes:

[0103] The release module is used to send a first lease release request to the master node. The first lease release request includes the identifier of the first partition and the identifier of the first lease.

[0104] Received a successful lease release response from the master node.

[0105] One implementation also includes:

[0106] The unloading module is used to unload the second partition of the distributed system and send a second lease release request to the master node. The second lease release request includes the identifier of the second partition and the identifier of the second lease. The second partition is the partition that has been loaded by the first slave node, and the second lease identifier corresponds to the lease of the second partition.

[0107] After receiving the lease release success response from the master node, it requests the lock server to release the lock on the second partition.

[0108] One implementation also includes:

[0109] The deletion module is used to respond to the lease acquisition success response returned by the master node and store the identifier of the process corresponding to the lease in the first partition;

[0110] If the process corresponding to the lease for the first partition does not exist when the first slave node restarts, a lease deletion request is sent to the master node. The lease deletion request includes the identifier of the first partition and the identifier of the first lease.

[0111] The memory processing apparatus of this disclosure can be used to implement the partition allocation method of any of the foregoing embodiments. Its implementation principle and technical effect are similar, and will not be repeated here.

[0112] Figure 6 is a second structural schematic diagram of a partition allocation device provided in an embodiment of this disclosure. As shown in Figure 6, the partition allocation device 600 includes:

[0113] The sending module 601 is used to send the correspondence between the first partition and the first slave node of the distributed system to the first slave node.

[0114] The receiving module 602 is used to receive a lease acquisition request sent by the first slave node. The lease acquisition request includes the identifier of the first partition, the first lease identifier, and the first timeout time.

[0115] The response module 603 is used to return a lease acquisition success response to the first slave node if the first partition is not bound to any other lease identifier besides the first lease identifier, or if the first partition is bound to any other lease identifier besides the first lease identifier, but the lease corresponding to the other lease identifier has expired. The response module binds the first partition and the first lease identifier, and determines the lease term of the lease corresponding to the first lease identifier stored by the master node based on the first timeout time.

[0116] In one implementation, the response module 603 is further used for:

[0117] If the first partition is already bound to the first lease identifier, or if the first partition is already bound to another lease identifier besides the first lease identifier and the lease corresponding to the other lease identifier has not expired, or if the master node is in a quiet period, then a lease acquisition failure response is returned to the first slave node. Here, the quiet period of the master node refers to the preset time period after the master node of the distributed system is switched.

[0118] In one implementation, the lease term corresponding to the first lease identifier stored by the master node is greater than or equal to the current time plus twice the first timeout time.

[0119] One implementation also includes:

[0120] The refresh module is used to receive a lease refresh request sent by the first slave node. The lease refresh request includes the identifier of the first partition, the first lease identifier, and the second timeout period.

[0121] If the master node is in a quiet period and the lease refresh request is the first lease refresh request for the first partition within the quiet period, or if the first partition is bound to the first lease identifier, then return a lease refresh success response to the first slave node and update the lease term of the lease corresponding to the first lease identifier stored by the master node according to the second timeout period.

[0122] If the first partition is not bound to any lease identifier, or if the first partition is bound to a lease identifier other than the first lease identifier, a lease refresh failure response is returned to the first slave node.

[0123] One implementation also includes:

[0124] The release module is used to receive a first lease release request sent by the first slave node. The first lease release request includes the identifier of the first partition and the identifier of the first lease.

[0125] If the first partition is bound to the first lease identifier, then return a lease release success response to the first slave node;

[0126] If the first partition is not bound to any lease identifier, or if the first partition is bound to a lease identifier other than the first lease identifier, a lease release failure response is returned to the first slave node.

[0127] In one implementation, the duration of the master node's quiet period is greater than or equal to the sum of the first timeout time and twice the remote procedure call timeout time of the first slave node.

[0128] One implementation also includes:

[0129] The barrier module is used to return a lease refresh failure response to the first slave node after receiving a lease refresh request, which includes the identifier of the first partition and the first lease identifier, if a lease barrier request is received. This response continues until the lease corresponding to the first lease identifier times out.

[0130] The memory processing apparatus of this disclosure can be used to implement the partition allocation method of any of the foregoing embodiments. Its implementation principle and technical effect are similar, and will not be repeated here.

[0131] Figure 7 is a schematic block diagram of an electronic device provided in an embodiment of this disclosure. As shown in Figure 7, the electronic device 700 may include at least one processor 701 for implementing the partition allocation method provided in an embodiment of this disclosure.

[0132] Optionally, the electronic device 700 further includes at least one memory 702 for storing program instructions and / or data. The memory 702 is coupled to the processor 701. The coupling in this embodiment is an indirect coupling or communication connection between devices, units, or modules, and can be electrical, mechanical, or other forms, for information exchange between devices, units, or modules. The processor 701 may operate in conjunction with the memory 702. The processor 701 may execute program instructions stored in the memory 702. At least one of the at least one memory may be included in the processor.

[0133] Optionally, the electronic device 700 further includes a communication interface 703 for communicating with other devices via a transmission medium, thereby enabling the electronic device 700 to communicate with other devices. The communication interface 703 may be, for example, a transceiver, interface, bus, circuit, or a device capable of transmitting and receiving functions. The processor 701 can utilize the communication interface 703 to transmit and receive data and / or information, and to implement the methods provided in the embodiments of this disclosure. See the detailed description in the foregoing embodiments for further details, which will not be repeated here.

[0134] This embodiment of the invention does not limit the specific connection medium between the processor 701, memory 702, and communication interface 703. In FIG. 7, the processor 701, memory 702, and communication interface 703 are connected via a bus 704. The bus 704 is represented by a thick line in FIG. 7. The connection methods between other components are for illustrative purposes only and are not intended to be limiting. This bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used in FIG. 7, but this does not indicate that there is only one bus or one type of bus.

[0135] It should be understood that the processor in the embodiments of this disclosure can be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method embodiments can be completed by the integrated logic circuitry in the processor's hardware or by instructions in software form. The processor can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this disclosure. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this disclosure can be directly embodied in the execution of a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software modules can be located in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory, and the processor reads the information in the memory and, in conjunction with its hardware, completes the steps of the above method.

[0136] It should also be understood that the memory in the embodiments of this disclosure may be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory may be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDR SDRAM), Enhanced Synchronous DRAM (ESDRAM), Sync Link DRAM (SLDRAM), and Direct Rambus RAM (DR RAM). It should be noted that the memory used in the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0137] This disclosure also provides a distributed system including a master node and a plurality of slave nodes, wherein the master node and slave nodes are used to perform the methods as described in any of the foregoing embodiments.

[0138] This disclosure also provides a computer-readable storage medium storing a computer program (also referred to as code or instructions). When the computer program is run by a processor, it causes the computer to perform the methods as described in any of the foregoing embodiments.

[0139] This disclosure also provides a computer program product, including a computer program that, when executed by a processor, implements the methods in any of the foregoing embodiments.

[0140] As used in this publication, the terms “unit”, “module”, etc., can be used to refer to computer-related entities, hardware, firmware, combinations of hardware and software, software, or software in execution.

[0141] Those skilled in the art will recognize that the various illustrative logical blocks and steps described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this disclosure. In the several embodiments provided in this disclosure, it should be understood that the disclosed apparatus, devices, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the shown or discussed mutual couplings or direct couplings or communication connections may be through some interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0142] The unit described as a separate component may or may not be physically separate. The component shown as a unit may or may not be a physical unit; that is, it may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0143] In addition, the functional units in the various embodiments of this disclosure can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0144] In the above embodiments, the functions of each functional unit can be implemented entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. This computer program product includes one or more computer instructions (programs). When the computer program instructions (programs) are loaded and executed on a computer, all or part of the flow or function according to the embodiments of this disclosure is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line, DSL) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., Digital Video Discs, DVDs), or semiconductor media (e.g., Solid State Disks, SSDs), etc.

[0145] If this function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this disclosure, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this disclosure. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.

[0146] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this disclosure are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data shall comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points shall be provided for users to choose to authorize or refuse.

[0147] The above description is merely a specific embodiment of this disclosure, but the scope of protection of this disclosure is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this disclosure should be included within the scope of protection of this disclosure. Therefore, the scope of protection of this disclosure should be determined by the scope of the claims.

Claims

1. A partition allocation method, wherein, The method is applied to a first slave node in a distributed system, the distributed system including a master node and multiple slave nodes, wherein the first slave node is any one of the multiple slave nodes, and the method includes: Receive the correspondence between the first partition and the first slave node of the distributed system sent by the master node; After obtaining the lock of the first partition from the lock server, a lease acquisition request is sent to the master node. The lease acquisition request includes the identifier of the first partition, the first lease identifier, and the first timeout time. In response to the lease acquisition success response returned by the master node, the lease term of the first partition corresponding to the first lease identifier is determined as the current time plus the first timeout time, and the first partition is loaded. Either the lock of the first partition or the lease of the first partition is used by the first slave node to perform input / output I / O operations on the first partition.

2. The method according to claim 1, wherein, Also includes: Before the first slave node performs an I / O operation on the first partition, it checks whether the lock and lease of the first partition have timed out. If either the lock or the lease of the first partition has not timed out, the I / O operation is performed.

3. The method according to claim 1 or 2, wherein, Also includes: A lease refresh request is sent to the master node at a preset time interval. The lease refresh request includes the identifier of the first partition, the first lease identifier, and the second timeout period. In response to the lease refresh success response returned by the master node, the lease term of the lease in the first partition corresponding to the first lease identifier is updated to the current time plus the second timeout time.

4. The method according to claim 3, wherein, The preset time interval is less than the first timeout time minus the remote procedure call timeout time of the first slave node.

5. The method according to any one of claims 1-4, wherein, Also includes: Send a first lease release request to the master node, the first lease release request including the identifier of the first partition and the first lease identifier; Received a lease release success response from the master node.

6. The method according to any one of claims 1-5, wherein, After receiving the correspondence between the first partition and the first slave node of the distributed system sent by the master node, the method further includes: The second partition of the distributed system is unloaded, and a second lease release request is sent to the master node. The second lease release request includes the identifier of the second partition and the second lease identifier. The second partition is the partition that has been loaded by the first slave node, and the second lease identifier corresponds to the lease of the second partition. After receiving the lease release success response from the master node, it requests the lock server to release the lock of the second partition.

7. The method according to any one of claims 1-6, wherein, Also includes: In response to the lease acquisition success response returned by the master node, the identifier of the process corresponding to the lease of the first partition is stored; If the process corresponding to the lease of the first partition does not exist when the first slave node restarts, a lease deletion request is sent to the master node. The lease deletion request includes the identifier of the first partition and the identifier of the first lease.

8. A partition allocation method, wherein, The method is applied to the master node of a distributed system, wherein the distributed system includes the master node and multiple slave nodes, and includes: Send the correspondence between the first partition and the first slave node of the distributed system to the first slave node; Receive a lease acquisition request sent by the first slave node, wherein the lease acquisition request includes the identifier of the first partition, the first lease identifier, and the first timeout period; If the first partition is not bound to any other lease identifier besides the first lease identifier, or if the first partition is bound to any other lease identifier besides the first lease identifier but the lease corresponding to the other lease identifier has expired, then a lease acquisition success response is returned to the first slave node, the first partition and the first lease identifier are bound, and the lease term of the lease corresponding to the first lease identifier stored by the master node is determined according to the first timeout time.

9. The method according to claim 8, wherein, Also includes: If the first partition is already bound to the first lease identifier, or if the first partition is already bound to another lease identifier besides the first lease identifier and the lease corresponding to the other lease identifier has not expired, or if the master node is in a quiet period, then a lease acquisition failure response is returned to the first slave node. The quiet period of the master node refers to a preset time period after the master node of the distributed system is switched.

10. The method according to claim 8 or 9, wherein, The lease term corresponding to the first lease identifier stored in the master node is greater than or equal to the current time plus twice the first timeout time.

11. The method according to any one of claims 8-10, wherein, Also includes: Receive a lease refresh request sent by the first slave node, wherein the lease refresh request includes the identifier of the first partition, the first lease identifier, and the second timeout period; If the master node is in a quiet period, and the lease refresh request is the first lease refresh request for the first partition within the quiet period, or if the first partition is bound to the first lease identifier, then a lease refresh success response is returned to the first slave node, and the lease term of the lease corresponding to the first lease identifier stored by the master node is updated and determined according to the second timeout period. If the first partition is not bound to any lease identifier, or if the first partition is bound to a lease identifier other than the first lease identifier, a lease refresh failure response is returned to the first slave node.

12. The method according to any one of claims 8-11, wherein, Also includes: Receive a first lease release request sent by the first slave node, wherein the first lease release request includes the identifier of the first partition and the first lease identifier; If the first partition is bound to the first lease identifier, then return a lease release success response to the first slave node; If the first partition is not bound to any lease identifier, or if the first partition is bound to a lease identifier other than the first lease identifier, a lease release failure response is returned to the first slave node.

13. The method according to any one of claims 9-12, wherein, The duration of the master node's quiet period is greater than or equal to the sum of the first timeout time and twice the remote procedure call timeout time of the first slave node.

14. The method according to any one of claims 9-13, wherein, Also includes: If a lease barrier request is received from the terminal, and the lease barrier request includes the identifier of the first partition and the first lease identifier, and the lease barrier request is used to indicate that the lease of the first partition should be stopped, then after receiving the lease refresh request sent by the first slave node, a lease refresh failure response is returned to the first slave node until the lease corresponding to the first lease identifier times out.

15. A partition allocation device, wherein, A first slave node applied in a distributed system, the distributed system including a master node and multiple slave nodes, the device comprising: The receiving module is used to receive the correspondence between the first partition and the first slave node of the distributed system sent by the master node; The request module is used to send a lease acquisition request to the master node after acquiring the lock of the first partition from the lock server. The lease acquisition request includes the identifier of the first partition, the first lease identifier, and the first timeout time. The update module is used to respond to the lease acquisition success response returned by the master node, determine the lease period of the first partition corresponding to the first lease identifier as the current time plus the first timeout time, and load the first partition, wherein either the lock of the first partition or the lease of the first partition is used by the first slave node to perform input / output I / O operations on the first partition.

16. A partition allocation device, wherein, An apparatus for use as a master node in a distributed system, the distributed system including the master node and multiple slave nodes, the apparatus comprising: The sending module is used to send the correspondence between the first partition and the first slave node of the distributed system to the first slave node; The receiving module is configured to receive a lease acquisition request sent by the first slave node, wherein the lease acquisition request includes the identifier of the first partition, the first lease identifier, and the first timeout period; The response module is configured to return a lease acquisition success response to the first slave node if the first partition is not bound to any other lease identifier besides the first lease identifier, or if the first partition is bound to any other lease identifier besides the first lease identifier but the lease corresponding to the other lease identifier has expired, bind the first partition and the first lease identifier, and determine the lease term of the lease corresponding to the first lease identifier stored by the master node based on the first timeout time.

17. A distributed system, wherein, include: A master node and multiple slave nodes; wherein the master node is used to perform the method as described in any one of claims 8-14, and the slave nodes are used to perform the method as described in any one of claims 1-7.

18. An electronic device, wherein, include: Memory and processor; The memory is used to store computer programs; The processor is configured to execute a computer program stored in the memory, wherein the computer program, when executed, causes the processor to perform the method according to any one of claims 1-14.

19. A computer-readable storage medium, wherein, The computer-readable storage medium stores a computer program that, when executed by a processor, causes the processor to perform the method as described in any one of claims 1-14.

20. A computer program product, wherein, Includes a computer program that, when executed by a processor, implements the method as described in any one of claims 1-14.

Images