Distributed queue lock execution method based on key-value database

Abstract

The present invention provides a distributed queue lock execution method based on a key-value database, which can be applied to the field of database technology. The method includes: in response to detecting a data access request of a first client for a target resource in a key-value database, reading first value information according to key information of a distributed lock corresponding to the target resource, the first value information including lock queue information of the distributed lock; determining the occupation result of the distributed lock according to the reading result of the first value information; when the occupation result indicates that the distributed lock is occupied by a second client, adding the client identifier of the first client to the lock queue information to obtain the updated first value information; writing the updated first value information in the key-value database to control the first client to access the target resource after the second client accesses the target resource.

Description

Technical Field

[0001] The present invention relates to the technical field of databases, and in particular to a distributed queue lock execution method based on a key-value database. Background Art

[0002] In a multitasking operating system, multiple tasks running at the same time may need to use the same resource. In order to ensure the rational use of resources, when a task needs to use a resource, it needs to be locked and protected. This lock is also called a mutex lock. With lock protection, concurrent access to resources between systems can become correct and reasonable.

[0003] In the traditional stand-alone programming mode, the lock programming interface provided by the database operating system is generally used to lock the resources that need to be protected. However, in a distributed database, since database tasks run on multiple different physical machines, the original lock control strategy in the stand-alone environment is not available in a distributed scenario. Therefore, a locking mechanism suitable for distributed scenarios is needed to protect shared resources in the distributed system. In addition, in the process of implementing the locking mechanism, it is also necessary to ensure the normal processing of tasks to avoid business starvation due to long-term failure to grab locks. Summary of the invention

[0004] In view of the above problems, the present invention provides a distributed queue lock execution method based on a key-value database.

[0005] According to a first aspect of the present invention, a distributed queue lock execution method based on a key-value database is provided, comprising: in response to detecting a data access request of a first client for a target resource in a key-value database, reading first value information according to key information of a distributed lock corresponding to the target resource, wherein the first value information includes lock queue information of the distributed lock; determining an occupation result of the distributed lock according to a reading result of the first value information; when the occupation result indicates that the distributed lock is occupied by a second client, starting a queue transaction, adding a client identifier of the first client to the lock queue information, and obtaining an updated first value information; and writing the updated first value information in the key-value database, so as to control the first client to access the target resource after the second client completes access to the target resource.

[0006] According to an embodiment of the present invention, the method further includes: in response to detecting a commit conflict in a write operation for the updated first value information, reopening the queued transaction after randomly sleeping for a preset period of time to re-add the client identifier of the first client to the lock queue information.

[0007] According to an embodiment of the present invention, the method also includes: in response to receiving lock change notification information for a distributed lock, obtaining second value information of the distributed lock, wherein the lock change notification information indicates that the value information of the distributed lock has changed; extracting a client identifier located at the first position of the lock queue information from the lock queue information in the second value information; when the client identifier matches the client identifier of the first client, writing the client address of the first client into the second value information, modifying the lock status information in the second value information to occupied, and deleting the client identifier of the first client from the lock queue information; and writing the updated second value information into the key-value database.

[0008] According to an embodiment of the present invention, the method also includes: determining the change type of the distributed lock based on the lock status information in the second value information; and when it is determined that the change type is to release the distributed lock, extracting the client identifier located at the first position of the lock queue information from the lock queue information in the second value information.

[0009] According to an embodiment of the present invention, the method also includes: when the occupation result is unoccupied, writing the client address of the first client into the first value information, modifying the lock status information in the first value information to occupied, and obtaining the updated first value information; and writing the updated first value information into the key-value database.

[0010] According to an embodiment of the present invention, the method further includes: in response to detecting that the first client ends access to the target resource, acquiring third value information of the distributed lock;

[0011] When it is determined that the client address in the third value information matches the client address of the first client, the lock queue information in the third value information is determined; when the lock queue information is not empty, the lock state information in the third value information is modified to unoccupied, and the client address in the third value information is deleted to obtain the updated third value information; when the lock queue information is empty, the third value information is deleted.

[0012] According to an embodiment of the present invention, the occupancy result of the distributed lock is determined based on the reading result of the first value information, including: when the reading result is a successful read, determining the occupancy result of the distributed lock is occupied; when the reading result is a read failure, determining the occupancy result of the distributed lock is not occupied.

[0013] According to an embodiment of the present invention, the first value information also includes the lock keep alive time and the lock write time. According to the reading result of the first value information, determining the occupation result of the distributed lock also includes: if the reading result is a successful read, obtaining the current time; calculating the time difference between the current time and the lock write time; if the time difference and the lock keep alive time meet a predetermined condition, determining the occupation result of the distributed lock as an occupation timeout; if the time difference and the lock keep alive time do not meet the predetermined condition, determining the occupation result of the distributed lock as being occupied.

[0014] According to an embodiment of the present invention, the method further includes: when it is determined that the occupancy result is an occupancy timeout, modifying the lock status information in the first value information to unoccupied, deleting the client address in the first value information, and obtaining the updated first value information; and starting a queuing transaction, adding the client identifier of the first client to the lock queue information, and obtaining the updated first value information.

[0015] The second aspect of the present invention provides a distributed queue lock execution device based on a key-value database, including: a reading module, which is used to respond to detecting a data access request of a first client for a target resource in the key-value database, and read the first value information according to the key information of the distributed lock corresponding to the target resource, wherein the first value information includes the lock queue information of the distributed lock; an occupation module, which is used to determine the occupation result of the distributed lock according to the reading result of the first value information; a queuing module, which is used to start a queuing transaction when the occupation result indicates that the distributed lock is occupied by the second client, add the client identifier of the first client to the lock queue information, and obtain the updated first value information; a writing module, which is used to write the updated first value information in the key-value database, so as to control the first client to access the target resource after the second client completes the access to the target resource.

[0016] The third aspect of the present invention provides an electronic device, comprising: one or more processors; a memory for storing one or more computer programs, and when the one or more processors execute the one or more computer programs, the steps of the distributed queue lock execution method based on the key-value database are implemented.

[0017] The fourth aspect of the present invention also provides a computer program product, including a computer program or instructions, which, when executed by a processor, implements the steps of the distributed queue lock execution method based on the key-value database.

[0018] The embodiment of the present invention not only implements a distributed lock mechanism in a distributed scenario through a key-value mechanism, but also adds a queuing mechanism under the mechanism, so that the traditional concurrent lock grabbing is transformed into a sequential queuing lock, avoiding invalid grabbing between clients and greatly reducing the system overhead caused by lock grabbing. At the same time, the embodiment of the present invention can also avoid the phenomenon of business starvation due to failure to grab locks, making a disordered database system orderly. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The above contents and other objects, features and advantages of the present invention will become more apparent through the following description of the embodiments of the present invention with reference to the accompanying drawings, in which:

[0020] Figure 1 The application scenario of the distributed queue lock execution method based on the key-value database according to an embodiment of the present invention is shown.

[0021] Figure 2 A flow chart of a distributed queue lock execution method based on a key-value database according to an embodiment of the present invention is shown.

[0022] Figure 3 A schematic diagram of the timing interaction between occupied transactions and queued transactions according to an embodiment of the present invention is shown.

[0023] Figure 4 A schematic diagram of an interaction for releasing a distributed lock according to a specific embodiment of the present invention is shown.

[0024] Figure 5 A schematic diagram of lock occupation interaction according to an embodiment of the present invention is shown.

[0025] Figure 6 A flow chart of a distributed lock queue mechanism according to an embodiment of the present invention is shown.

[0026] Figure 7 A flow chart of a distributed lock occupation mechanism according to an embodiment of the present invention is shown.

[0027] Figure 8 A structural block diagram of a distributed queue lock execution device based on a key-value database according to an embodiment of the present invention is shown.

[0028] Fig. 9 A block diagram of an electronic device suitable for a distributed queue lock execution method based on a key-value database according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0029] Below, embodiments of the present invention will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present invention. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of embodiments of the present invention. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of concepts of the present invention.

[0030] The terms used herein are only for describing specific embodiments and are not intended to limit the present invention. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0031] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0032] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0033] In the traditional stand-alone environment programming mode, the lock programming interface provided by the database operating system is generally used to lock the resources that need to be protected. The following will take the initialization of the mutex variable, calling the lock method to lock, using the shared resource after the lock is successfully locked, and calling the unlock method to unlock as an example to explain. The method for initializing the mutex variable is generally: int pthread_ mutex_init (pthread_mutex_t *mutex, const pthread_mutexattr_t *attr).

[0034] The lock method is generally called as: int pthread_mutex_lock (pthread_ mutex_t *mutex), which is used to lock the mutex. If the mutex is already locked, the caller will be blocked in this call interface to achieve the purpose of lock protection. When the shared resource is used after the lock is successful, the use of the shared resource is safe because the lock is successful. The unlock method is generally called as: int pthread_mutex_lock (pthread_mutex_t *mutex). From this, we can see that the mutex lock implemented through the programming interface is generally suitable for a single machine environment.

[0035] In addition, in the current distributed lock implementation mechanism, distributed locks generally simply form mutually exclusive operations, and access to shared resources is achieved through concurrent lock grabbing operations. For example, the distributed lock function based on a single-machine database is mainly achieved by writing a unique index to achieve the purpose of locking; the distributed lock based on redis is mainly achieved by concurrently writing a key and ensuring that only one client can write successfully; the distributed lock based on zookeeper is mainly achieved by concurrently writing a path and ensuring that only one write can succeed. It can be seen that the current distributed lock implementation mechanism is concurrent lock grabbing, without a queuing mechanism. This leads to a large contention overhead for the database system, and there may also be a situation where the business is dead due to failure to grab the lock.

[0036] To this end, an embodiment of the present invention provides a distributed queue lock implementation method based on a key-value database, which not only implements a distributed lock, but also adds a queuing mechanism to the distributed lock implementation mechanism. The distributed queue lock execution method based on the key-value database includes: in response to detecting a data access request from a first client for a target resource in the key-value database, reading first value information according to the key information of the distributed lock corresponding to the target resource, wherein the first value information includes the lock queue information of the distributed lock; determining the occupation result of the distributed lock according to the reading result of the first value information; when the occupation result indicates that the distributed lock is occupied by the second client, starting a queuing transaction, adding the client identifier of the first client to the lock queue information, and obtaining the updated first value information; and writing the updated first value information in the key-value database, so as to control the first client to access the target resource after the second client completes the access to the target resource.

[0037] Figure 1 The application scenario of the distributed queue lock execution method based on the key-value database according to an embodiment of the present invention is shown.

[0038] like Figure 1 As shown, the application scenario 100 according to this embodiment may include a first client 101 , a second client 102 , a third client 103 and a database operating system 104 .

[0039] The first client 101, the second client 102, and the third client 103 may be servers or database nodes for executing tasks. The database operating system 104 may include a server cluster for unified management of multiple databases and servers.

[0040] The network is used to provide a medium for communication links between the first client 101, the second client 102, the third client 103 and the database operating system 104. The network may include various connection types, such as wired, wireless communication links or optical fiber cables.

[0041] It should be noted that the distributed queue lock execution method based on the key-value database provided in the embodiment of the present invention can generally be executed by one of the first client 101, the second client 102, and the third client 103. Correspondingly, the distributed queue lock execution device based on the key-value database provided in the embodiment of the present invention can generally be set in one of the first client 101, the second client 102, and the third client 103.

[0042] The following will be based on Figure 1 The scene described by Figure 2~Figure 7 The distributed queue lock execution method based on the key-value database of an embodiment of the invention is described in detail.

[0043] Figure 2 A flow chart of a distributed queue lock execution method based on a key-value database according to an embodiment of the present invention is shown.

[0044] like Figure 2 As shown, the method 200 includes operations S210 to S240.

[0045] In operation S210, in response to detecting a data access request from a first client for a target resource in a key-value database, first value information is read according to key information of a distributed lock corresponding to the target resource, wherein the first value information includes lock queue information of the distributed lock.

[0046] In operation S220, an occupation result of the distributed lock is determined according to a reading result of the first value information.

[0047] In operation S230, when the occupation result indicates that the distributed lock is occupied by the second client, a queue transaction is started, and the client identifier of the first client is added to the lock queue information to obtain updated first value information.

[0048] Operation S240: writing the updated first value information into the key-value database, so as to control the first client to access the target resource after the second client completes accessing the target resource.

[0049] According to an embodiment of the present invention, the first client may be a server, a database node, etc. of a target resource to be accessed.

[0050] A key-value database is a database that stores data in the form of key-value pairs. It can be understood as a distributed hash map. The key (primary key) in the table is used to locate the value (key value), that is, to store and retrieve specific values. In a key-value database, a key uniquely corresponds to a value. Value can be used to store any type of data, including integers, characters, arrays, objects, etc.

[0051] The key-value database used in the embodiment of the present invention supports multi-node distributed deployment to meet high availability requirements; supports transaction atomicity, consistency, isolation, and durability of data reading and writing, that is, ACID characteristics; and supports key-value change notification mechanisms. For example, a distributed database foundationdb can be used as a key-value database.

[0052] The target resource may be information pre-stored in a key-value database. For example, the target resource may be a shared resource that can be accessed by multiple clients.

[0053] The embodiment of the present invention uses a key-value mechanism to implement a distributed lock. For a distributed lock for a target resource, the key can be defined using the lock name of the distributed lock, and the value can be defined using a variety of information, such as lock queue information. The lock queue information is used to indicate whether there are other clients waiting in line to occupy the distributed lock.

[0054] When the first client accesses the target resource in the key-value database, the first client can uniquely read the first value information corresponding to the target resource from the key-value database through the key information of the distributed lock corresponding to the target resource. It should be noted that when accessing the target resource, the first client can construct the key information of the distributed lock based on the target resource. For example, the key information can be the lock name, "path access lock".

[0055] Since the embodiment of the present invention implements a distributed lock based on a key-value mechanism, it is possible to determine whether the distributed lock has been occupied based on whether the first value information can be read from the key-value database. It should be noted that due to the unique correspondence between key and value in the key-value mechanism, if the first value information is read, it indicates that a second client or other client has occupied the distributed lock, and the occupation information is written into the first value information. If the first value information is not read, it indicates that no client has occupied the distributed lock yet.

[0056] According to an embodiment of the present invention, operations S210 and S220 may be implemented through occupied transactions of a key-value database, and operations S230 and S240 may be implemented through queued transactions of a key-value database.

[0057] If it is determined in operation S230 that the distributed lock is occupied by the second client, the occupation transaction may be terminated, and the distributed lock occupation operation of the first client may be terminated; thereafter, a queuing transaction may be started to sequentially access the target resource through a queuing mechanism.

[0058] According to an embodiment of the present invention, since the first value information includes the lock queue information of the distributed lock, the client identifier of the first client can be added to the lock queue information according to the time data in the queue transaction. So far, the queue transaction updates the specific value in the first value information.

[0059] For example, when the first client accesses the target resource of the key-value database, the second client is accessing the target resource, and there is a third client waiting to access the target resource. Therefore, the lock queue information of the first value information before being updated includes the client identifier of the third client, and after the queuing transaction, the lock queue information in the updated first value information includes the client identifiers of the third client and the first client in sequence, so as to realize the sequential access of the second client, the third client and the first client to the target resource.

[0060] Afterwards, in order to inform the key-value database of the occupancy or queuing status of the distributed lock, the updated first value information needs to be rewritten into the key-value database. Thus, the queuing mechanism of the first client is completed.

[0061] The embodiment of the present invention not only implements a distributed lock mechanism in a distributed scenario through a key-value mechanism, but also adds a queuing mechanism under the mechanism, so that the traditional concurrent lock grabbing is transformed into a sequential queuing lock, avoiding invalid grabbing between clients and greatly reducing the system overhead caused by lock grabbing. At the same time, the embodiment of the present invention can also avoid the phenomenon of business starvation due to failure to grab locks, making a disordered database system orderly.

[0062] It should be noted that the first value information, the second value information and the third value information appearing in the present invention can be the value values ​​read in real time for the same target resource in different access scenarios; they can also be the value values ​​obtained by performing read operations on the same target resource at different time sequences in the same access scenario.

[0063] According to an embodiment of the present invention, the value information of a distributed lock may include lock queue information, client address, and lock status information. The client address is used to represent the address of the client currently occupying the distributed lock, such as an Internet Protocol Address (IP address). The lock status information is used to represent whether the distributed lock is currently occupied.

[0064] According to an embodiment of the present invention, the method further includes: in response to detecting a commit conflict in a write operation for the updated first value information, reopening the queued transaction after randomly sleeping for a preset period of time to re-add the client identifier of the first client to the lock queue information.

[0065] For the write operation of operation S240, since other clients, such as the third client, may also synchronously perform the occupation and queuing operations in a distributed environment, that is, perform operations S210 to S240, the write operation of the first client for the updated first value information will have a commit conflict.

[0066] In the event of a commit conflict, in order to ensure the reliability of data in the key-value database, the queued transaction of the first client needs to be retried. In order to avoid the commit conflict during requeuing, the queue conflict of the third client is avoided through random sleep operation.

[0067] According to an embodiment of the present invention, the random sleep preset duration may be sleep according to a predefined preset duration, or may be sleep randomly selected from a preset duration interval. For example, sleep may be performed randomly for 1ms to 10ms, and then queue transactions may be reopened to queue, thereby effectively avoiding another queue conflict.

[0068] Figure 3 A schematic diagram of the timing interaction between occupied transactions and queued transactions according to an embodiment of the present invention is shown.

[0069] like Figure 3 As shown, the sequential interaction diagram of the occupied transaction and the queued transaction includes operations S300 to S308. Operation S300 indicates that the distributed lock is currently occupied by the second client. The occupied transaction includes operations S301 to S303, the queued transaction includes operations S304 to S306, and operations S307 and S308 are requeuing operations in the case of submission conflicts.

[0070] In operation S300, the distributed lock is currently occupied by the second client.

[0071] In operation S301, first value information is read.

[0072] In operation S302, the read is successful. In the case where the first value information is read, the read is successful, indicating that the distributed lock has been added by other clients, the current lock fails, and the occupied transaction ends. In operation S303, it ends.

[0073] In operation S304, the first value information is read. It should be noted that, since there are other operations between operation S301 and S304, the first value information read in operation S304 may be the same as the first value information read in S301, or may be different.

[0074] In operation S305, the client identifier of the first client is added to the lock queue information.

[0075] In operation S306, the updated first value information is written.

[0076] In operation S307, the conflict is committed.

[0077] In operation S308, the system randomly sleeps for a preset time period.

[0078] It should be noted that before occupying transactions and queuing transactions, the operations of starting transactions using the database interface of the key-value database, building key information, and passing in key information using the data reading interface are also included. After that, the first client can read the first value information from the key-value database.

[0079] According to an embodiment of the present invention, after executing operation S308 and randomly sleeping for a preset time, the queued transaction is restarted, that is, operations S304 to S306 are re-executed, which will not be described in detail here.

[0080] According to an embodiment of the present invention, after the first client completes the queued transaction, the first client can monitor in real time whether the value information of the distributed lock has changed. Alternatively, the key-value database can send lock change notification information to the first client through a notification mechanism when the value information of the distributed lock changes.

[0081] Considering that there can be multiple clients queuing for a certain distributed lock at the same time. Since multiple clients queue up to lock, when the distributed lock is released, the distributed lock will be occupied by the client in the first place. Therefore, considering the system consumption caused by the lock change notification, the key-value database can send lock change notification information only to the first or first few clients based on the lock queue information, so as to reduce the number of clients notified and the number of notifications, thereby reducing system consumption.

[0082] According to an embodiment of the present invention, when the value information of a distributed lock changes, the method further includes: in response to receiving lock change notification information for the distributed lock, obtaining second value information of the distributed lock, wherein the lock change notification information indicates that the value information of the distributed lock has changed; extracting a client identifier located at the first position of the lock queue information from the lock queue information in the second value information; when the client identifier matches the client identifier of the first client, writing the client address of the first client into the second value information, modifying the lock status information in the second value information to occupied, deleting the client identifier of the first client from the lock queue information; and writing the updated second value information into the key-value database.

[0083] According to an embodiment of the present invention, after the first client receives the lock change notification information for the distributed lock, the first client is awakened and can start a new transaction to determine whether the distributed lock can be occupied, such as starting a lock awakening transaction.

[0084] The execution logic of the lock wake-up transaction includes: queue verification and lock occupation operation. Specifically, the queue verification includes extracting the client identifier located at the first position of the lock queue information from the lock queue information in the second value information, and determining whether the client identifier matches the client identifier of the first client. The lock occupation operation includes: updating the second value information when the client identifier matches the client identifier of the first client; and writing the updated second value information.

[0085] It should be noted that no matter whether the lock change notification information is sent to the client at the head of the queue or the clients at the front of the queue, the client needs to verify again when receiving the lock change notification information to avoid lock grabbing due to erroneous notification.

[0086] For example, when the first client is the first client in the lock queue information, the first client can receive lock change notification information for the distributed lock. When the client identifier matches the client identifier of the first client, it indicates that the first client is the first in the lock queue information, that is, the first in the queue. At this time, the first client can occupy the distributed lock by modifying the second value information.

[0087] Specifically, when releasing the distributed lock, the client address in the second value information obtained by the first client is cleared and the lock status information is unoccupied. At this time, the first client can modify the client address by writing the IP address of the first client and change the lock status information from unoccupied to occupied. In addition, since the first client is going to occupy the distributed lock, the client identifier of the first client needs to be deleted from the lock queue information.

[0088] According to an embodiment of the present invention, after writing the updated second value information, a transaction is also submitted to complete the lock wake-up transaction. At this time, since there is a queue first information matching mechanism in the lock wake-up transaction, no submission conflict will be caused.

[0089] According to an embodiment of the present invention, the method further includes: determining the change type of the distributed lock based on the lock status information in the second value information; and when it is determined that the change type is to release the distributed lock, extracting the client identifier at the first position of the lock queue information from the lock queue information in the second value information.

[0090] According to an embodiment of the present invention, changes in the value information of a distributed lock may include multiple change types, such as releasing the distributed lock and queuing changes.

[0091] When the value information of the distributed lock changes, if the lock status information is unoccupied, the change type is determined to be releasing the distributed lock; if the lock status information is occupied, the change type is determined to be a queued change.

[0092] When it is determined that the change type is a queued change, the lock wakeup transaction is ended.

[0093] Figure 4 A schematic diagram of an interaction for releasing a distributed lock according to a specific embodiment of the present invention is shown.

[0094] In operation S400, release.

[0095] In operation S401, the value information of the distributed lock changes.

[0096] In operation S402, lock change notification information is issued.

[0097] In operation S403, the lock state information is unoccupied.

[0098] In operation S404, second value information is requested.

[0099] In operation S405, second value information is read.

[0100] In operation S406, the client identifier located at the first position in the lock queue information is extracted from the lock queue information in the second value information.

[0101] In operation S407, when the client identifier matches, the IP address and lock status information are modified.

[0102] In operation S408, the updated second value information is written.

[0103] It should be noted that after the second client actively releases the distributed lock and writes the released value information in operation S400, the key-value database can detect that the value information of the distributed lock has changed. Then, the key-value database sends a lock change notification to the first client at the top of the lock queue information.

[0104] According to an embodiment of the present invention, after the occupancy result of the distributed lock is determined in operation S220, it also includes: when the occupancy result is not occupied, writing the client address of the first client into the first value information, modifying the lock status information in the first value information to occupied, and obtaining the updated first value information; and writing the updated first value information into the key-value database.

[0105] According to an embodiment of the present invention, the occupation result is unoccupied, indicating that the distributed lock is not currently occupied. Therefore, the first client can directly occupy it without queuing, that is, continue to execute the occupation transaction.

[0106] According to an embodiment of the present invention, since the occupation result is unoccupied and the first value information is empty, the IP address of the first client can be directly written into the first value information, and the lock state information in the first value information can be modified to occupied to obtain the updated first value information. Afterwards, the updated first value information is written into the key-value database to indicate that the distributed lock is occupied by the first client.

[0107] According to an embodiment of the present invention, after the first client ends accessing the target resource, the distributed lock may be actively released.

[0108] According to an embodiment of the present invention, the method also includes: in response to detecting that the first client ends access to the target resource, obtaining the third value information of the distributed lock; when it is determined that the client address in the third value information matches the client address of the first client, determining the lock queue information in the third value information; when the lock queue information is not empty, modifying the lock status information in the third value information to unoccupied, deleting the client address in the third value information, and obtaining the updated third value information; when the lock queue information is empty, deleting the third value information.

[0109] According to an embodiment of the present invention, in response to detecting that the first client ends access to the target resource, a lock release transaction may be started to implement active release of the distributed lock.

[0110] Since operations such as client queuing that cause value information to change may occur during the process of the first client occupying the distributed lock, the third value information of the current key information still needs to be read after the lock release transaction is started.

[0111] In actual application, when the first client occupies the distributed lock, if the first client fails, other clients will release the distributed lock to avoid deadlock; afterwards, if the first client continues to actively release the distributed lock after recovery, it may cause an erroneous release. Therefore, in order to avoid erroneous release, it is necessary to first verify the identity of the client, that is, to determine whether the client address in the third value information matches the client address that executes the lock release transaction.

[0112] In the case where the first client actively releases the distributed lock, the identity authentication of the first client is achieved by determining whether the client address in the third value information matches the client address of the first client. In the case where it is determined that the client address in the third value information matches the client address of the first client, the lock queue information in the third value information is determined. At this time, the lock queue information in the third value information is determined to determine whether there are other clients in the queue. If the lock queue information is not empty, there are other clients in the queue; if the lock queue information is empty, there are no other clients in the queue.

[0113] If the lock queue information is not empty, the lock state information in the third value information is modified to unoccupied, and the client address in the third value information is deleted to obtain the updated third value information, so that after the first client releases the distributed lock, the distributed lock can be occupied by other clients in the queue. If the lock queue information is empty, the third value information is directly deleted.

[0114] According to an embodiment of the present invention, a commit conflict may occur when committing a lock release transaction, indicating that there may be queued client requests at this time and lock queue information has been written, and the lock release transaction can be re-executed.

[0115] Regarding commit conflicts, the commit conflict of an occupied transaction may be caused by the fact that when client A is executing the occupied transaction, another client, such as client B, has occupied the distributed lock first. In this case, client A can re-execute the occupied transaction.

[0116] Queued transactions can avoid commit conflicts by notifying only the client that is first in the lock queue information.

[0117] Figure 5 A schematic diagram of lock occupation interaction according to an embodiment of the present invention is shown.

[0118] When the first client accesses the key-value database, no other client occupies the distributed lock. Therefore, the first client can occupy the distributed lock through operations S501 to S504, and after completing the access to the target resource, actively release the distributed lock through operations S505 to S510, and then the second client occupies the distributed lock released by the first client through operation S511. It should be noted that the operation of the second client occupying the distributed lock is similar to that of the first client, and will not be repeated here.

[0119] In operation S501, first value information is read.

[0120] In operation S502 , reading fails.

[0121] In operation S503, the IP address, current time, and lock status information are written into the first value information.

[0122] In operation S504, the updated first value information is written.

[0123] In operation S505, third value information is read.

[0124] In operation S506, the reading is successful.

[0125] In operation S507, the IP address in the third value information is consistent with the IP address of the first client.

[0126] In operation S508, the lock queue information in the third value information is not empty.

[0127] In operation S509, the lock status information is modified to be unoccupied, and the IP address in the third value information is deleted.

[0128] In operation S510, updated third value information is written.

[0129] In operation S511, occupy.

[0130] Through the embodiments of the present invention, the distributed lock can achieve the following conditions: 1. Mutual exclusivity. At any time, only one client obtains the distributed lock. When a client obtains and occupies the distributed lock, the remaining clients can only wait. 2. High availability. In a distributed environment, the function of the distributed lock requires a higher level of availability, that is, when some machines in the system are abnormal, the lock service can continue to be provided to ensure that the lock service is not interrupted. 3. Queue mechanism: If the distributed lock is held by another client at this time, the client that needs the distributed lock will wait. When the distributed lock is released, the client at the top of the lock queue information will be notified.

[0131] According to an embodiment of the present invention, the occupancy result of the distributed lock is determined based on the reading result of the first value information, including: when the reading result is a successful read, determining the occupancy result of the distributed lock is occupied; when the reading result is a read failure, determining the occupancy result of the distributed lock is not occupied.

[0132] In the embodiment of the present invention, when the occupation result of the distributed lock is determined directly based on the reading result, if the client does not release the distributed lock due to an exception, other clients will be unable to obtain the distributed lock, resulting in a deadlock. Therefore, a lock validity judgment can be added to the locking logic to deal with invalid distributed locks.

[0133] According to an embodiment of the present invention, the first value information also includes a lock keep alive time and a lock write time. The lock keep alive time is used to determine whether the currently occupied distributed lock is valid. For example, the lock keep alive time can be predefined. The lock write time refers to the time when the client occupies the distributed lock, which can be determined by the system timestamp.

[0134] According to an embodiment of the present invention, in an occupied transaction, each time the IP address of the value information is updated, the lock keep alive time and the lock write time may be synchronously updated.

[0135] Therefore, the distributed queue lock execution based on the key-value database also includes: when the reading result is a successful read, obtaining the current time; calculating the time difference between the current time and the lock write time; when the time difference and the lock keep alive time meet the predetermined conditions, determining that the occupation result of the distributed lock is an occupation timeout; when the time difference and the lock keep alive time do not meet the predetermined conditions, determining that the occupation result of the distributed lock is occupied. Therefore, deadlock is avoided by the above method.

[0136] Specifically, the time difference between the current time and the lock write time and the lock keep alive time can be compared to perform the validity judgment of the distributed lock. If the distributed lock is valid, the occupation result of the distributed lock is determined to be occupied, the occupation transaction is terminated, and the queued transaction is started; if the distributed lock is invalid, the occupation result of the distributed lock is determined to be occupied timeout, and the distributed lock is passively released.

[0137] In an embodiment of the present invention, the time difference may be obtained by subtracting the lock keep alive time from the current time.

[0138] According to an embodiment of the present invention, the predetermined condition may be that the time difference is greater than a predetermined multiple of the lock keep alive time, such as 2. For example, if a client is abnormal, the lock keep alive time of the distributed lock occupied by the client will not be updated. At this time, after the time difference between the current time and the lock write time exceeds 2 times the lock keep alive time, that is, 10 seconds, it indicates that the lock is invalid.

[0139] The embodiment of the present invention can handle the problem of deadlock in distributed locks by adding lock failure judgment in the process of adding distributed locks, thereby promoting the lock occupation progress and reducing resource waste of the database system.

[0140] According to an embodiment of the present invention, in addition to adding lock validity judgment to the occupied transaction, a lock release operation can also be added to handle the invalid distributed lock.

[0141] According to an embodiment of the present invention, the method further includes: when it is determined that the occupancy result is an occupancy timeout, modifying the lock status information in the first value information to unoccupied, deleting the client address in the first value information, and obtaining the updated first value information; and starting a queuing transaction, adding the client identifier of the first client to the lock queue information, and obtaining the updated first value information.

[0142] According to the embodiment of the present invention, since the modification operation of the first value information is newly added in the occupied transaction, that is, the lock state information and the client address are modified, the invalid distributed lock can be cleared when the occupied transaction ends. Therefore, the first client can execute the queued transaction normally.

[0143] In another embodiment, when it is determined that the occupation result is an occupation timeout, it is also possible to determine whether there are other clients in the queue according to the lock queue information in the first value information. If there are no other clients in the queue, the IP address and lock writing time of the first client are directly written into the first value information, that is, the distributed lock is directly occupied without starting a queue transaction. If there are other clients in the queue, the queue transaction is started.

[0144] According to an embodiment of the present invention, the lock keep alive time can be updated through a keep alive process. Specifically, the first client can periodically interact with the database system to continuously update the lock keep alive time.

[0145] For example, when the first client occupies a distributed lock, the lock keep alive time is the default 5 seconds. If the first client needs to continue to occupy the distributed lock, the lock keep alive time can be extended every 5 seconds through interactive operations. For example, through an interactive operation, the lock keep alive time is extended to 10s or 15s, etc., so that when other clients execute the occupied transaction, the distributed lock occupied by the first client will not be mistakenly cleared due to invalidity.

[0146] Figure 6 A flow chart of a distributed lock queue mechanism according to an embodiment of the present invention is shown.

[0147] like Figure 6 As shown, for a client to access the target resource, such as client 1, after starting the occupation transaction, the key information of the lock of the target resource to be accessed can be constructed. The lock is a distributed lock, which is referred to as the lock below. After constructing the key information, the value information is read according to the key information. After reading the value information, if it is determined that the lock is currently occupied by other clients, such as client 2, the client identifier of client 1 needs to be written into the lock queue information in the value information (not shown in the figure). After that, the key and value information are written in the key-value database and submitted.

[0148] After the occupied transaction is successfully submitted, client 1 is in the waiting state. When awakened, the value information is read according to the key information, and it is determined whether client 1 is the first in the queue. If client 1 is the first in the queue, the lock information is modified to occupy the distributed lock. The lock ends after the lock is successfully added. If client 1 is not the first in the queue, client 1 resumes the waiting state.

[0149] When the occupied transaction is not successfully committed, that is, the commit fails, client 1 fails to lock and retries after random sleep.

[0150] Figure 7A flow chart of a distributed lock occupation mechanism according to an embodiment of the present invention is shown.

[0151] like Figure 7 As shown, for a client to access the target resource, such as client 2, after starting the occupation transaction, the key information of the lock can be constructed for the target resource to be accessed. The lock is a distributed lock, hereinafter referred to as a lock. After the key information is constructed, the value information is read according to the key information.

[0152] If the read is successful, continue to determine whether the lock keep alive time in the value information is valid. If the lock keep alive time is valid, that is, the lock is valid, client 2 fails to lock and waits for retry; if the lock keep alive time is invalid, that is, the lock fails, client 2 releases the current lock and constructs the lock value information and lock write time based on its own information. For example, construct the value information based on the IP address of client 2, and construct the lock write time based on the system timestamp. After that, write the key and value information in the key-value database and submit.

[0153] Furthermore, for the commit operation, it is necessary to determine whether the commit is successful. After the occupied transaction is successfully committed, client 2 clears the currently invalid lock and reoccupies the lock, and ends after the lock is successfully locked. If the occupied transaction is not successfully committed, that is, the commit fails, client 2 fails to lock and waits for retry.

[0154] If the read fails, it indicates that the current lock is not occupied. You can then build the lock value information, build the lock write time, write the key and value information, and submit.

[0155] Compared with the existing distributed lock technology, the distributed queue lock execution method based on the key-value database provided by the present invention has the following advantages: 1. It can meet the distributed locking requirements in the distributed database system. 2. It has failed lock detection. 3. It is implemented based on the distributed key-value database system, and the logic is simple. 4. It meets the high availability characteristics and can continue to provide services even if there are node abnormalities in the database system. 5. It has a queue mechanism to avoid the system overhead caused by the polling and preemptive lock, and avoid starvation of the business layer due to the inability to add locks.

[0156] Figure 8 A structural block diagram of a distributed queue lock execution device based on a key-value database according to an embodiment of the present invention is shown.

[0157] like Figure 8As shown, the distributed queue lock execution device 800 based on the key-value database of this embodiment includes a reading module 810, an occupation module 820, a queuing module 830, and a writing module 840.

[0158] The reading module 810 is used to respond to detecting a data access request from a first client for a target resource in a key-value database, and read the first value information according to the key information of the distributed lock corresponding to the target resource, wherein the first value information includes the lock queue information of the distributed lock.

[0159] The occupation module 820 is used to determine the occupation result of the distributed lock according to the reading result of the first value information.

[0160] The queuing module 830 is used to start a queuing transaction when the occupation result indicates that the distributed lock is occupied by the second client, add the client identifier of the first client to the lock queue information, and obtain the updated first value information.

[0161] The writing module 840 is used to write the updated first value information into the key-value database so as to control the first client to access the target resource after the second client completes accessing the target resource.

[0162] It should be noted that the device part in the embodiment of the present invention corresponds to the method part in the embodiment of the present invention. The description of the device part specifically refers to the method part, which will not be repeated here.

[0163] According to an embodiment of the present invention, any multiple modules among the reading module 810, the occupying module 820, the queuing module 830, and the writing module 840 can be combined into one module for implementation, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module.

[0164] According to an embodiment of the present invention, at least one of the read module 810, the occupied module 820, the queue module 830, and the write module 840 may be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application-specific integrated circuit (ASIC), or may be implemented by hardware or firmware in any other reasonable manner of integrating or packaging the circuit, or in any one of the three implementation methods of software, hardware, and firmware, or in any appropriate combination of any of them. Alternatively, at least one of the read module 810, the occupied module 820, the queue module 830, and the write module 840 may be at least partially implemented as a computer program module, and when the computer program module is run, the corresponding function may be executed.

[0165] Fig. 9 A block diagram of an electronic device suitable for a distributed queue lock execution method based on a key-value database according to an embodiment of the present invention is shown.

[0166] like Fig. 9 As shown, the electronic device 900 according to an embodiment of the present invention includes a processor 901, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage part 908 to a random access memory (RAM) 903. The processor 901 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 901 may also include an onboard memory for caching purposes. The processor 901 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present invention.

[0167] In RAM 903, various programs and data required for the operation of electronic device 900 are stored. Processor 901, ROM 902 and RAM 903 are connected to each other via bus 904. Processor 901 performs various operations of the method flow according to the embodiment of the present invention by executing the program in ROM 902 and / or RAM 903. It should be noted that the program can also be stored in one or more memories other than ROM 902 and RAM 903. Processor 901 can also perform various operations of the method flow according to the embodiment of the present invention by executing the program stored in the one or more memories.

[0168] According to an embodiment of the present invention, the electronic device 900 may further include an input / output (I / O) interface 905, which is also connected to the bus 904. The electronic device 900 may further include one or more of the following components connected to the input / output I / O interface 905: an input portion 906 including a keyboard, a mouse, etc.; an output portion 907 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 908 including a hard disk, etc.; and a communication portion 909 including a network interface card such as a LAN card, a modem, etc. The communication portion 909 performs communication processing via a network such as the Internet. A drive 910 is also connected to the I / O interface 905 as needed. A removable medium 911, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 910 as needed, so that a computer program read therefrom is installed into the storage portion 908 as needed.

[0169] The embodiment of the present invention also includes a computer program product, which includes a computer program, and the computer program includes a program code for executing the method shown in the flowchart. When the computer program product is run in a computer system, the program code is used to enable the computer system to implement the above method provided by the embodiment of the present invention. When the computer program is executed by the processor 901, the above functions defined in the system / device of the embodiment of the present invention are executed. According to an embodiment of the present invention, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0170] In one embodiment, the computer program may rely on tangible storage media such as optical storage devices and magnetic storage devices. In another embodiment, the computer program may also be transmitted and distributed in the form of a signal on a network medium, and downloaded and installed through the communication part 909, and / or installed from a removable medium 911. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above. In such an embodiment, the computer program may be downloaded and installed from the network through the communication part 909, and / or installed from a removable medium 911. When the computer program is executed by the processor 901, the above functions defined in the system of the embodiment of the present invention are executed. According to an embodiment of the present invention, the system, device, means, module, unit, etc. described above may be implemented by a computer program module.

[0171] According to an embodiment of the present invention, the program code for executing the computer program provided by the embodiment of the present invention can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (e.g., using an Internet service provider to connect through the Internet).

[0172] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0173] It will be appreciated by those skilled in the art that the features described in the various embodiments of the present invention may be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in the present invention. In particular, without departing from the spirit and teachings of the present invention, the features described in the various embodiments of the present invention may be combined and / or combined in various ways. All of these combinations and / or combinations fall within the scope of the present invention.

[0174] The specific embodiments described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A distributed queue lock execution method based on a key-value database, characterized in that: The method comprises: In response to detecting a data access request from a first client for a target resource in a key-value database, reading first value information according to key information of a distributed lock corresponding to the target resource, wherein the first value information includes lock queue information of the distributed lock; Determine the occupation result of the distributed lock according to the reading result of the first value information; When the occupation result indicates that the distributed lock is occupied by the second client, start a queue transaction, add the client identifier of the first client to the lock queue information, and obtain updated first value information; and Writing the updated first value information into the key-value database, so as to control the first client to access the target resource after the second client completes accessing the target resource; Also includes: In response to receiving lock change notification information for the distributed lock, obtaining second value information of the distributed lock, wherein the lock change notification information indicates that the value information of the distributed lock has changed; extracting a client identifier located at the first position of the lock queue information from the lock queue information in the second value information; when the client identifier matches the client identifier of the first client, writing the client address of the first client into the second value information, modifying the lock state information in the second value information to occupied, deleting the client identifier of the first client from the lock queue information; and writing the updated second value information into the key-value database; In response to detecting that the first client ends access to the target resource, the third value information of the distributed lock is obtained; when it is determined that the client address in the third value information matches the client address of the first client, the lock queue information in the third value information is determined; when the lock queue information is not empty, the lock state information in the third value information is modified to be unoccupied, and the client address in the third value information is deleted to obtain the updated third value information; when the lock queue information is empty, the third value information is deleted.

2. The method according to claim 1, characterized in that Also includes: In response to detecting a commit conflict in the write operation for the updated first value information, the queued transaction is reopened after randomly sleeping for a preset period of time to re-add the client identifier of the first client to the lock queue information.

3. The method according to claim 1, characterized in that Also includes: Determining a change type of the distributed lock according to the lock state information in the second value information; as well as When it is determined that the change type is releasing the distributed lock, the client identifier located at the first position in the lock queue information is extracted from the lock queue information in the second value information.

4. The method according to claim 1, characterized in that: Also includes: When the occupation result is unoccupied, writing the client address of the first client into the first value information, modifying the lock state information in the first value information to occupied, and obtaining updated first value information; as well as The updated first value information is written into the key-value database.

5. The method according to claim 1, characterized in that The determining, according to the reading result of the first value information, the occupation result of the distributed lock includes: When the reading result is a successful reading, determining that the occupation result of the distributed lock is occupied; When the reading result is a reading failure, it is determined that the occupation result of the distributed lock is not occupied.

6. The method according to claim 1, characterized in that The first value information also includes a lock keep alive time and a lock write time; The determining the occupation result of the distributed lock according to the reading result of the first value information further includes: When the reading result is that the reading is successful, obtaining the current time; Calculate the time difference between the current time and the lock write time; When the time difference and the lock keep alive time satisfy a predetermined condition, determining that the occupation result of the distributed lock is an occupation timeout; When the time difference and the lock keep alive time do not satisfy a predetermined condition, it is determined that the occupation result of the distributed lock is occupied.

7. The method according to claim 6, characterized in that Also includes: When it is determined that the occupation result is an occupation timeout, the lock state information in the first value information is modified to unoccupied, and the client address in the first value information is deleted to obtain an updated first value information; as well as The queue transaction is started, and the client identifier of the first client is added to the lock queue information to obtain updated first value information.

8. An electronic device, characterized in that: The electronic device comprises: one or more processors; a memory for storing one or more computer programs, It is characterized in that the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1 to 7.

Images