Distributed lock implementation method and system based on Redis

Abstract

The present invention discloses a method and system for implementing a distributed lock based on Redis, belonging to the technical field of Java under the microservice architecture. The technical problem to be solved by the present invention is how to handle the concurrent access of data with Redis distributed locks and ensure the ultimate consistency of data. The technical solution is as follows: The method is specifically as follows: Obtain the current time in milliseconds; Request to acquire a lock from the Redis master node in sequence; The client sets the network connection and response timeout, and the response timeout is less than the expiration time of the lock; If the response timeout expires, skip this Redis master node and try the next Redis master node; The client uses the current time minus the time when the lock acquisition started to obtain the time used for lock acquisition: Only when more than half of the Redis master nodes have acquired the lock and the time used for lock acquisition is less than the lock expiration time, the lock acquisition is successful, and the true effective time of the acquired lock needs to subtract the time used for lock acquisition; When the lock acquisition fails, the client unlocks on all Redis master nodes.

Description

Technical Field

[0001] The present invention relates to the field of Java technology under a microservice architecture, and specifically to a distributed lock implementation method and system based on Redis. Background Art

[0002] In many scenarios, ensuring eventual data consistency requires a variety of technical solutions, such as distributed locks. When developing an application, if concurrent access to a shared variable is required, distributed locks can be used to handle this.

[0003] To ensure that a method or property can only be executed by one accessor at a time under high concurrency conditions, traditional monolithic applications deployed on a single machine can use concurrency-related features for mutual exclusion control. However, as business needs evolve, the original monolithic, single-machine system evolves into a distributed cluster system. Because distributed systems are multi-threaded, multi-process, and distributed across different machines, this renders the concurrency control locking strategy used in the original single-machine deployment ineffective. Simple applications cannot provide distributed locking capabilities. To address this problem, a cross-machine mutual exclusion mechanism is needed to control access to shared resources. This is the problem that distributed locks address.

[0004] Many internet applications require locking, such as flash sales, globally incremented IDs, and floor generation. Distributed locks are a method for controlling synchronized access to shared resources across distributed systems. If different systems, or even different hosts within the same system, share a resource or set of resources, access to these resources often requires mutual exclusion to prevent interference and ensure consistency. In this case, distributed locks are essential. Most solutions rely on databases, but common relational databases face bottlenecks in read and write performance.

[0005] Therefore, how to use Redis distributed locks to handle concurrent access to data and ensure the eventual consistency of data is a technical problem that needs to be solved urgently. Summary of the Invention

[0006] The technical task of the present invention is to provide a distributed lock implementation method and system based on Redis to solve the problem of how to use Redis distributed lock to handle concurrent access to data and ensure the ultimate consistency of data.

[0007] The technical task of the present invention is achieved in the following way: a distributed lock implementation method based on Redis, the method is as follows:

[0008] Get the current time in milliseconds;

[0009] Request locks from the Redis master node in sequence;

[0010] The client sets the network connection and response timeout, and the response timeout is less than the lock expiration time. If the response timeout exceeds the timeout, the Redis master node is skipped and the next Redis master node is tried.

[0011] The client uses the current time minus the time when the lock was acquired to obtain the usage time of the lock:

[0012] The lock is acquired successfully if and only if more than half of the Redis master nodes have acquired the lock and the lock usage time is less than the lock expiration time;

[0013] When the lock is acquired successfully, the actual effective time of acquiring the lock (key) needs to be subtracted from the time used to acquire the lock;

[0014] When the lock acquisition fails, the client unlocks the lock on all Redis master nodes.

[0015] As a preference, when acquiring a lock, use SETNX to lock it, and use the EXPIRE command to add a timeout for the lock. If the timeout is exceeded, the lock is automatically released. The value of the lock is a randomly generated UUID, and the lock value is used to determine when to release the lock.

[0016] SETNX(SETNX key val) is specifically: if and only if key does not exist, set a string with key val and return 1; if key exists, do nothing and return 0;

[0017] EXPIRE (EXPIRE key timeout) specifically: set a timeout for the key in seconds. After the timeout, the lock will be automatically released to avoid deadlock;

[0018] DELETE (DELETE key) specifically means: delete key.

[0019] More preferably, when acquiring a lock, a timeout is set. If the timeout is exceeded, the lock acquisition is abandoned.

[0020] More preferably, when releasing a lock, the UUID is used to determine whether it is the lock. If it is the lock, DELETE is executed to release the lock.

[0021] As a preference, the locking is specifically as follows:

[0022] Try to lock it through the set method:

[0023] If the current lock does not exist, return lock success

[0024] If the current lock already exists, get the lock's expiration time and compare the acquired lock's expiration time with the current time:

[0025] If the lock has expired, set a new expiration time and return a lock success message.

[0026] More preferably, the set method is:

[0027] set(String key,String value,String nxxx,String expx,int time);

[0028] Among them, the parameter key: use the key as the lock, the key is unique;

[0029] Parameter value: The value passed here is the requestId. To ensure the reliability of distributed locks, the value is assigned the requestId to understand the corresponding lock request and provide a basis for unlocking. That is, we know which request added the lock and have a basis for unlocking. The requestId is generated using the UUID.randomUUID().toString() method.

[0030] Parameter nxxx: This uses NX, which means SET IF NOT EXIST. That is, when the key does not exist, the set operation is performed; if the key already exists, no operation is performed.

[0031] Parameter expx: This parameter is passed as PX, which means to add an expiration setting for the key. The specific time is determined by the parameter time;

[0032] Parameter time: corresponds to the parameter Expx and represents the expiration time of the key.

[0033] As a preference, unlocking is as follows:

[0034] Get the value corresponding to the lock and check whether it is equal to the requestId:

[0035] If they are equal, delete the lock, that is, unlock it;

[0036] If they are not equal, the execution ends immediately.

[0037] Among these distributed locks, exclusive locks (X locks, also known as write locks or exclusive locks) are a basic type of lock. If transaction T1 applies an exclusive lock to data object O1, only transaction T1 is allowed to read and update O1 during the entire lock period. No other transactions can perform any operations on this data object until T1 releases the exclusive lock. The core of exclusive locks is how to ensure that only one transaction currently holds the lock and that all transactions waiting to acquire the lock are notified when the lock is released.

[0038] Distributed locks should meet the following conditions:

[0039] ① In a distributed system environment, a method can only be executed by one thread on one machine at a time;

[0040] ② Highly available lock acquisition and lock release;

[0041] ③High-performance lock acquisition and lock release;

[0042] ④It has reentrant characteristics;

[0043] ⑤Equipped with lock failure mechanism to prevent deadlock;

[0044] ⑥ It has a non-blocking lock feature, that is, if the lock is not acquired, it will directly return a lock acquisition failure.

[0045] The present invention adopts Redis combined with springboot as the implementation of distributed lock.

[0046] A distributed lock implementation system based on redis, the system includes:

[0047] Get module, used to get the current time in milliseconds;

[0048] The request module is used to request locks from the Redis master node in sequence;

[0049] The setting module is used to set the network connection and response timeout through the client. The response timeout is less than the lock expiration time. If the response timeout is exceeded, the redis master node is skipped and the next redis master node is tried.

[0050] The calculation module is used to subtract the time when the lock is acquired from the current time used by the client to obtain the usage time of the lock:

[0051] The judgment module is used to determine whether more than half of the Redis master nodes have obtained the lock and the lock usage time is less than the lock expiration time:

[0052] If yes, the lock is acquired successfully, and the actual effective time of acquiring the lock (key) needs to be subtracted from the time used to acquire the lock;

[0053] If not, the lock acquisition fails and the client unlocks the lock on all Redis master nodes.

[0054] An electronic device comprising: a memory and at least one processor;

[0055] wherein the memory stores computer-executable instructions;

[0056] The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes the above-mentioned distributed lock implementation method based on Redis.

[0057] A computer-readable storage medium is characterized in that the computer-readable storage medium stores computer-executable instructions, and when a processor executes the computer-executable instructions, the above-mentioned distributed lock implementation method based on Redis is implemented.

[0058] The distributed lock implementation method and system based on Redis of the present invention have the following advantages:

[0059] (1) The present invention utilizes the characteristics of Redis and combines it with Spring Boot to implement a distributed lock application, simplifying the development of back-end applications, improving the system performance of traditional methods, and thus improving system stability. The switch mode makes it more flexible. Among them, Redis is a single-process single-threaded mode, using a queue mode to convert concurrent access into serial access, and there is no competition between multiple clients for Redis connections. Secondly, Redis provides some commands SETNX and GETSET, which can facilitate the implementation of a distributed lock mechanism.

[0060] (2) The distributed lock implemented by Redis in the present invention has good scalability and can easily respond to changes in demand and business expansion. However, for simple scenarios, it is easier to directly use scheduled tasks. When there are a large number of distributed lock requests, you can consider using the cluster feature of Redis to deploy more Redis instances to make the system more scalable.

[0061] (3) The present invention is highly efficient. Redis is a well-known open source key-value database. Official test data shows that the read speed is 110,000 times / s and the write speed is 81,000 times / s, and this has been verified in many practical scenarios.

[0062] (4) The present invention has high availability, and Redis supports multi-instance deployment. After one instance is shut down, there are backup instances to continue providing services;

[0063] (5) The present invention is real-time, and the redis master-slave synchronization mechanism ensures that lock information can be synchronized between users in a timely manner. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] The present invention will be further described below with reference to the accompanying drawings.

[0065] Attachment Figure 1 This is a schematic diagram of the distributed lock implementation method based on Redis. DETAILED DESCRIPTION

[0066] The following detailed description of the distributed lock implementation method and system based on Redis of the present invention is made with reference to the accompanying drawings and specific embodiments.

[0067] Example 1:

[0068] As attached Figure 1 As shown, the distributed lock implementation method based on redis of the present invention is as follows:

[0069] S1. Get the current time in milliseconds;

[0070] S2, request locks from the Redis master node in sequence;

[0071] S3. The client sets the network connection and response timeout. The response timeout should be less than the lock expiration time (assuming the lock automatically expires in 10 seconds, the timeout is generally between 5 and 50 milliseconds. Let's assume the timeout is 50 milliseconds). If the response timeout is exceeded, the Redis master node is skipped and the next one is tried.

[0072] S4. The client subtracts the time when the lock was acquired from the current time to obtain the usage time of the lock:

[0073] The lock is acquired successfully if and only if more than half (N / 2+1, here 5 / 2+1=3 nodes) of the Redis master nodes have acquired the lock, and the lock acquisition time is less than the lock expiration time (10s>30ms+40ms+50ms+4m0s+50ms);

[0074] When the lock is acquired successfully, the actual effective time of acquiring the lock (key) needs to be subtracted from the time used to acquire the lock;

[0075] When the lock acquisition fails (the lock is not acquired on at least N / 2+1 master instances, or the lock acquisition time has exceeded the valid time), the client unlocks the lock on all Redis master nodes (even if some master nodes do not lock successfully at all, they still need to be unlocked to prevent some from slipping through the net).

[0076] In this embodiment, when acquiring a lock, SETNX is used to lock it, and the EXPIRE command is used to add a timeout period for the lock. When the timeout period is exceeded, the lock is automatically released. The value of the lock is a randomly generated UUID, and the lock value is used to determine when to release the lock.

[0077] SETNX(SETNX key val) is specifically: if and only if key does not exist, set a string with key val and return 1; if key exists, do nothing and return 0;

[0078] EXPIRE (EXPIRE key timeout) specifically: set a timeout for the key in seconds. After the timeout, the lock will be automatically released to avoid deadlock;

[0079] DELETE (DELETE key) specifically means: delete key.

[0080] When acquiring a lock in this embodiment, a timeout period is set for acquisition. If the timeout period is exceeded, the acquisition of the lock is abandoned.

[0081] When releasing the lock in this embodiment, the UUID is used to determine whether it is the lock. If it is the lock, DELETE is executed to release the lock.

[0082] The locking in this embodiment is specifically as follows:

[0083] Try to lock it through the set method:

[0084] If the current lock does not exist, return lock success

[0085] If the current lock already exists, get the lock's expiration time and compare the acquired lock's expiration time with the current time:

[0086] If the lock has expired, set a new expiration time and return a lock success message.

[0087] The set method in this embodiment is specifically:

[0088] set(String key,String value,String nxxx,String expx,int time);

[0089] Among them, the parameter key: use the key as the lock, the key is unique;

[0090] Parameter value: The value passed here is the requestId. To ensure the reliability of distributed locks, the value is assigned the requestId to understand the corresponding lock request and provide a basis for unlocking. That is, we know which request added the lock and have a basis for unlocking. The requestId is generated using the UUID.randomUUID().toString() method.

[0091] Parameter nxxx: This uses NX, which means SET IF NOT EXIST. That is, when the key does not exist, the set operation is performed; if the key already exists, no operation is performed.

[0092] Parameter expx: This parameter is passed as PX, which means to add an expiration setting for the key. The specific time is determined by the parameter time;

[0093] Parameter time: corresponds to the parameter Expx and represents the expiration time of the key.

[0094] The unlocking in this embodiment is accomplished through a script implemented in LUA. Redis can execute Lua scripts atomically. This part of the code must be atomic. Non-atomic execution may cause the following problems:

[0095] The problem with this code is that during a DELETE, the lock held by another client is released when the lock no longer belongs to the current client. Is this scenario really possible? The answer is yes. For example, if client A locks the lock and then unlocks it some time later, before executing the DELETE method, the lock suddenly expires. At this point, client B attempts to lock the lock successfully. If client A then executes the DELETE method, client B's lock is released.

[0096] Unlock details are as follows:

[0097] Get the value corresponding to the lock and check whether it is equal to the requestId:

[0098] If they are equal, delete the lock, that is, unlock it;

[0099] If they are not equal, the execution ends immediately.

[0100] Example 2:

[0101] The distributed lock implementation system based on redis of the present invention includes:

[0102] Get module, used to get the current time in milliseconds;

[0103] The request module is used to request locks from the Redis master node in sequence;

[0104] The setting module is used to set the network connection and response timeout through the client. The response timeout is less than the lock expiration time. If the response timeout is exceeded, the redis master node is skipped and the next redis master node is tried.

[0105] The calculation module is used to subtract the time when the lock is acquired from the current time used by the client to obtain the usage time of the lock:

[0106] The judgment module is used to determine whether more than half of the Redis master nodes have obtained the lock and the lock usage time is less than the lock expiration time:

[0107] If yes, the lock is acquired successfully, and the actual effective time of acquiring the lock (key) needs to be subtracted from the time used to acquire the lock;

[0108] If not, the lock acquisition fails and the client unlocks the lock on all Redis master nodes.

[0109] Example 3:

[0110] The present invention also provides an electronic device, comprising: a memory and at least one processor;

[0111] wherein the memory stores computer-executable instructions;

[0112] The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes the distributed lock implementation method based on Redis according to any one of the present inventions.

[0113] The processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The processor may be a microprocessor or any conventional processor, etc.

[0114] The memory can be used to store computer programs and / or modules. The processor implements various functions of the electronic device by running or executing the computer programs and / or modules stored in the memory, and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area. The program storage area can store an operating system, at least one application required for a function, etc.; the data storage area can store data created based on the use of the terminal, etc. In addition, the memory can also include high-speed random access memory and non-volatile memory, such as a hard disk, internal memory, a plug-in hard disk, a smart memory card (SMC), a secure digital (SD) card, a flash memory card, at least one disk storage period, a flash memory device, or other volatile solid-state memory devices.

[0115] Example 4:

[0116] The present invention also provides a computer-readable storage medium storing a plurality of instructions, which are loaded by a processor and cause the processor to execute the Redis-based distributed lock implementation method according to any embodiment of the present invention. Specifically, a system or device equipped with a storage medium can be provided, wherein the storage medium stores software program code that implements the functions of any of the above-described embodiments, and causes a computer (or CPU or MPU) of the system or device to read and execute the program code stored in the storage medium.

[0117] In this case, the program code itself read from the storage medium can realize the function of any one of the above-mentioned embodiments, and thus the program code and the storage medium storing the program code constitute part of the present invention.

[0118] Examples of storage media for providing program code include floppy disks, hard disks, magneto-optical disks, optical disks (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD+RW), magnetic tapes, non-volatile memory cards, and ROMs. Alternatively, the program code can be downloaded from a server computer via a communication network.

[0119] In addition, it should be clear that the functions of any of the above embodiments can be achieved not only by executing the program code read by the computer, but also by enabling the operating system operating on the computer to complete part or all of the actual operations based on the instructions of the program code.

[0120] In addition, it can be understood that the program code read from the storage medium is written into the memory provided in the expansion board inserted into the computer or into the memory provided in the expansion unit connected to the computer, and then based on the instructions of the program code, the CPU installed on the expansion board or expansion unit is enabled to perform part or all of the actual operations, thereby realizing the functions of any of the above embodiments.

[0121] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A distributed lock implementation method based on redis, characterized in that: The method is as follows: Get the current time in milliseconds; Request locks from the Redis master node in sequence; The client sets the network connection and response timeout, and the response timeout is less than the lock expiration time. If the response timeout exceeds the timeout, the Redis master node is skipped and the next Redis master node is tried. The client uses the current time minus the time when the lock was acquired to obtain the usage time of the lock: The lock is acquired successfully if and only if more than half of the Redis master nodes have acquired the lock and the lock usage time is less than the lock expiration time; When the lock is acquired successfully, the actual effective time of the lock acquisition needs to be subtracted from the time used to acquire the lock; When the lock acquisition fails, the client unlocks it on all Redis master nodes; When acquiring a lock, use SETNX to lock it, and use the EXPIRE command to add a timeout for the lock. If the timeout is exceeded, the lock is automatically released. The lock value is a randomly generated UUID, and the lock value is used to determine when to release the lock. SETNX is specifically: if and only if key does not exist, set a string with key val and return 1; if key exists, return 0; EXPIRE specifically sets a timeout for the key in seconds. After the timeout, the lock will be automatically released to avoid deadlock. DELETE specifically means: delete key; When acquiring a lock, a timeout is set. If the timeout is exceeded, the lock acquisition is abandoned. When releasing a lock, the UUID is used to determine whether it is the lock. If it is the lock, DELETE is executed to release the lock. The locking details are as follows: Try to lock it through the set method: If the current lock does not exist, return lock success If the current lock already exists, get the lock's expiration time and compare the acquired lock's expiration time with the current time: If the lock has expired, set a new expiration time and return a lock success message; The set method is as follows: set(String key, String value, String nxxx, String expx, int time); Among them, the parameter key: use the key as the lock, the key is unique; Parameter value: By assigning value to requestId, we can understand the corresponding lock request and provide a basis for unlocking. The requestId is generated using the UUID.randomUUID().toString() method. Parameter nxxx: SET IF NOT EXIST, that is, when the key does not exist, perform the set operation; if the key already exists, do nothing; Parameter expx: adds an expiration setting to the key. The specific time is determined by the parameter time. Parameter time: corresponds to the parameter Expx, representing the expiration time of the key; Unlock details are as follows: Get the value corresponding to the lock and check whether it is equal to the requestId: If they are equal, delete the lock, that is, unlock it; If they are not equal, the execution ends immediately.

2. A distributed lock implementation system based on redis, characterized in that: The system is used to implement the distributed lock implementation method based on redis as claimed in claim 1; the system includes: Get module, used to get the current time in milliseconds; The request module is used to request locks from the Redis master node in sequence; The setting module is used to set the network connection and response timeout through the client. The response timeout is less than the lock expiration time. If the response timeout is exceeded, the redis master node is skipped and the next redis master node is tried. The calculation module is used to obtain the lock usage time by subtracting the lock acquisition start time from the current time used by the client: The judgment module is used to determine whether more than half of the Redis master nodes have obtained the lock and the lock usage time is less than the lock expiration time: If yes, the lock is acquired successfully, and the actual effective time of the lock acquisition needs to be subtracted from the time used to acquire the lock; If not, the lock acquisition fails and the client unlocks the lock on all Redis master nodes.

3. An electronic device, characterized in that: include: memory and at least one processor; wherein the memory stores computer-executable instructions; The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes the Redis-based distributed lock implementation method according to claim 1.

4. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer-executable instructions. When the processor executes the computer-executable instructions, the distributed lock implementation method based on Redis as claimed in claim 1 is implemented.

Images