Distributed lock batch processing method and system based on atomic script and medium

Abstract

The application provides a distributed lock batch processing method and system based on atomic scripts and a medium, the method comprising: batch atomic locking, a client initiates a single atomic request to a distributed storage server, calls a first atomic script, the script atomically locks multiple resources, and internally builds a failure automatic rollback mechanism; batch atomic renewal, after successful locking, an asynchronous daemon thread is started, a single atomic request is periodically initiated, a second atomic script is called, and batch renewal of multiple locks is realized; batch atomic unlocking, after the completion of the business, a single atomic request is initiated, a third atomic script is called, and batch unlocking is realized based on a reentrant counting mechanism. The application optimizes locking, renewal and unlocking into a single network request, realizes high performance of the whole process, guarantees strong atomicity and consistency through atomic scripts and a rollback mechanism, and enhances the robustness of the system.

Description

Technical Field

[0001] This invention relates to the field of distributed computing technology, and more specifically, to a distributed lock batch processing method, system, and medium based on atomic scripts. Background Technology

[0002] In distributed computing systems, distributed lock mechanisms are typically introduced to ensure data consistency and atomicity of operations when multiple nodes concurrently access shared resources. Existing distributed lock implementations, such as those based on distributed caching databases or coordination services, often encounter performance bottlenecks when handling scenarios requiring the simultaneous locking of a large number of resources.

[0003] A common implementation involves the client sending a separate lock request to the distributed storage service for each resource that needs to be locked. This serial request approach results in the total network overhead being proportional to the number of resources to be locked. When the number of resources to be locked is large, frequent network round trips can cause significant latency, severely reducing system throughput and response speed, making it difficult to meet the needs of high-performance applications.

[0004] To address these issues, some technical solutions propose using atomic commands provided by storage services to achieve batch locking and unlocking of multiple resources. For example, a client can send a single atomic write command, which the server then locks multiple resources involved in that command at once. While this approach optimizes the network overhead of the locking and unlocking phases from linear complexity to constant complexity, it still has several drawbacks. First, this solution does not adequately consider the scenario where locking some resources fails during batch locking, lacking effective atomicity guarantees and rollback mechanisms. This could lead to an inconsistent system state due to partial resource locking failures, or even the creation of persistent locks that are difficult to clean up. Second, this solution does not provide effective support for long-running services. If the service execution time exceeds the lock's preset lifespan, the lock will be automatically released, thus violating the atomicity of the service execution. Finally, even if a lock renewal mechanism is introduced, existing renewal methods typically poll and renew for individual locks. This makes the renewal operation itself a new performance bottleneck when holding a large number of locks, and its network overhead degenerates back to linear complexity, contradicting the original intention of batch operations. Summary of the Invention

[0005] In view of the deficiencies in the prior art, the purpose of this invention is to provide a distributed lock batch processing method, system and medium based on atomic scripts.

[0006] According to the present invention, a batch processing method for distributed locks based on atomic scripts is executed by a client. The method is applied to a system consisting of the client and a distributed storage service client, and includes: batch atomic locking: initiating a single atomic request to the distributed storage service client to call a first atomic script pre-installed on the server, and passing in a locking instruction containing a list of resource keys for multiple resources, a lock holder identifier, and a lock lifetime; so that the first atomic script is executed atomically on the server: traversing the resource key list, for each resource, if the resource is not locked, creating a lock record containing the lock holder identifier and a reentry count initialized to 1; if the resource is already locked by the currently requested lock holder, incrementing the reentry count of the corresponding lock record by 1; wherein, during the execution of the first atomic script, if locking of any resource fails, an automatic return is triggered. Delete the lock records of resources successfully locked in this request; Batch atomic renewal: After successful batch atomic locking, start an asynchronous daemon thread. The asynchronous daemon thread periodically constructs renewal instructions containing a list of resource keys to be renewed and sends a single atomic request to the distributed storage service client to call a second atomic script pre-installed on the server to perform batch renewal of locks in the list of resource keys to be renewed; Batch atomic unlock: Send a single atomic request to the distributed storage service client to call a third atomic script pre-installed on the server and pass in the list of resource keys to be unlocked and the lock holder identifier; so that the third atomic script is executed atomically on the server: traverse the list of resource keys, for each resource, if its lock holder is the same as the requester, decrement the re-entry count of its lock record by 1, and delete the lock record of the resource when the re-entry count reaches zero.

[0007] Preferably, the client maintains a local state table to record lock information that has been successfully acquired, and the asynchronous daemon thread operates based on the local state table.

[0008] Preferably, the renewal period of the asynchronous daemon thread is set to be shorter than the lifetime of the lock.

[0009] Preferably, the asynchronous daemon thread scans the local state table to select a set of lock resources whose next renewal time is within a preset time threshold, and initiates the batch atomic renewal request only for lock resources in that set.

[0010] Preferably, when the batch atomic locking fails, the first atomic script, after executing the rollback, also returns to the client the location or identifier of the first conflicting resource that caused the locking failure.

[0011] Preferably, when processing an unlock request, if the third atomic script finds that the lock holder of the resource to be unlocked does not match the requester, it records the resource as an abnormal resource and returns it to the client.

[0012] This invention also provides a distributed lock batch processing system, including a client and a distributed storage service client. The client includes: a lock request module, used to construct a lock instruction containing a list of resource keys for multiple resources, a lock holder identifier, and a lock lifetime, and to initiate a single atomic request to the distributed storage service client to call a first atomic script to implement batch atomic locking; and a renewal management module, used to start and manage an asynchronous daemon thread after successful batch atomic locking. The asynchronous daemon thread periodically constructs a renewal instruction containing a list of resource keys to be renewed and initiates a single atomic request to the distributed storage service client to... The system calls a second atomic script to perform batch renewals of locks in the list of resource keys awaiting renewal. An unlock request module is used to construct an unlock instruction containing a list of resource keys to be unlocked and the lock holder's identifier, and initiate a single atomic request to the distributed storage service client to call a third atomic script to perform batch atomic unlocking. The distributed storage service client includes a script execution engine for atomically executing the first, second, and third atomic scripts. When executing the first atomic script, the script execution engine supports lock reentrancy counting and automatic rollback on failure; when executing the third atomic script, it supports unlocking based on reentrancy counting.

[0013] Preferably, the renewal management module of the client is specifically used to: scan the local status table, filter out the set of lock resources whose next renewal time is within a preset time threshold, and initiate batch atomic renewal requests only for the lock resources in the set.

[0014] The present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, it implements the above-described distributed lock batch processing method based on atomic scripts.

[0015] Compared with the prior art, the present invention has the following beneficial effects:

[0016] 1. This invention features high performance throughout the entire process. It optimizes the operations of the three key stages of locking, renewal, and unlocking into batch atomic operations of a single network request, so that the network overhead is a constant complexity regardless of the number of locks processed, which significantly improves the system throughput and response speed in large-scale chain scenarios.

[0017] 2. This invention has strong atomicity and consistency. It utilizes the atomic script characteristics of the storage engine, especially the built-in automatic rollback mechanism in the locking script, to ensure the "all or nothing" nature of batch locking operations. This fundamentally avoids the problem of inconsistent states or resource deadlocks caused by the failure of locking some resources, thus ensuring the reliability of the system.

[0018] 3. This invention enhances robustness and flexibility. The batch atomic renewal mechanism of this invention ensures lock security for long-running tasks and effectively prevents locks from being released unexpectedly due to business timeouts. At the same time, by introducing a reentrancy counting mechanism, it provides native support for complex nested locking business logic, avoids self-deadlock problems of the same client, and improves the versatility and robustness of the solution. Attached Figure Description

[0019] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0020] Figure 1 This is a schematic diagram of the architecture of a distributed lock batch processing system provided in an embodiment of the present invention;

[0021] Figure 2 A flowchart illustrating a distributed lock batch processing method based on atomic scripts, provided as an embodiment of the present invention;

[0022] Figure 3 This is a timing diagram of signaling interaction during the batch atomic processing provided in an embodiment of the present invention.

[0023] Explanation of reference numerals in the attached figures:

[0024] 10. Client; 11. Business Application; 12. Distributed Lock Manager; 13. Asynchronous Daemon Thread; 14. Local State Table; 20. Distributed Storage Service Client; 21. Atomic Script Collection; 22. Lock Data Storage Area; 30. Network; S100. Start Step; S110. Batch Atomic Locking Step; S120. Check if Locking is Successful Step; S130. Execute Business Logic Step; S140. Batch Atomic Unlocking Step; S150. End Step; S210. Start Asynchronous Daemon Thread Step; S220. Periodic Batch Atomic Renewal Step. Detailed Implementation

[0025] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that those skilled in the art can make several changes and improvements without departing from the concept of the present invention. These all fall within the protection scope of the present invention.

[0026] Example 1

[0027] This embodiment provides a basic implementation scheme for a distributed lock batch processing method and system based on atomic scripts. This scheme aims to solve the problems of low performance and insufficient atomicity guarantees caused by multiple network interactions when locking, renewing, and unlocking a large number of shared resources in existing technologies.

[0028] Please see Figure 1 The figure illustrates a schematic diagram of the architecture of a distributed lock batch processing system provided by an embodiment of the present invention. As shown in the figure, the system mainly includes a client 10 that communicates with the system via a network 30 and a distributed storage service client 20.

[0029] In one embodiment of the present invention, client 10 may be a business node in a distributed application, such as a microservice instance or a web application server. Client 10 deploys logical units that implement the method of the present invention. Specifically, client 10 may include a business application 11, a distributed lock manager 12, an asynchronous daemon thread 13, and a local state table 14. The business application 11, as the end user of the distributed lock service, is responsible for handling specific business logic. To coordinate access to shared resources, business application 11 calls distributed lock manager 12. Distributed lock manager 12, as the core implementation of the method of the present invention on the client 10 side, encapsulates all the details of interaction with the distributed storage service client 20, including constructing atomic script invocation instructions, sending requests, processing responses, and managing the entire lifecycle of the lock. To support long-running tasks and prevent locks from expiring due to timeout, distributed lock manager 12 creates and manages an asynchronous daemon thread 13 after successfully acquiring the lock. This thread runs independently of the main flow of business application 11, and its core responsibility is to periodically perform batch renewal operations on the locks it already holds. Meanwhile, the distributed lock manager 12 also maintains a local state table 14, which is used to cache information related to successfully acquired locks locally on the client 10, such as resource keys and lock expiration times, thereby providing data support for the work of the asynchronous daemon thread 13. Understandably, the local state table 14 can be implemented using an in-memory data structure such as a hash table to support efficient querying and updating.

[0030] The distributed storage service client 20 can be a high-performance key-value storage system that provides atomic operation capabilities. In this embodiment, it can specifically be a cluster of one or more Redis servers. The distributed storage service client 20 contains two key parts: an atomic script set 21 and a lock data storage area 22. The atomic script set 21 is a set of scripts pre-loaded onto the server, such as scripts written in languages ​​like Lua. These scripts define the specific atomic operation logic for batch locking, batch renewal, and batch unlocking. Since these scripts are executed atomically by the server's own script execution engine, it is ensured that they are not interrupted by other commands during execution, thus guaranteeing the indivisibility of batch operations. The lock data storage area 22 is the area in the server's memory used to actually store lock records. Each lock record typically exists in the form of a key-value pair, where the key is the identifier of the locked resource (i.e., the resource key), and the value is a data structure containing lock holder information and reentrancy counts, such as a hash data structure in Redis.

[0031] Please refer to the following: Figure 2 The diagram illustrates the overall process of the method of the present invention. The entire process starts from the initial step S100 and mainly includes three stages: batch atomic locking, batch atomic renewal, and batch atomic unlocking.

[0032] The process first enters the batch atomic locking step S110. In this step, the business application 11 of the client 10 sends a locking request to the distributed lock manager 12 for multiple resources. The distributed lock manager 12 then sends an atomic request to the distributed storage service client 20 containing all the resources to be locked.

[0033] Subsequently, in step S120, which determines whether the locking was successful, client 10 makes a judgment based on the result returned by the server. If the locking fails (for example, because the resource has been locked by another client), the process will directly jump to the end step S150. At this time, the upper-layer business application 11 can execute the corresponding exception handling logic based on the failure result, such as retrying later or abandoning the operation. If the locking is successful, the main process continues to the business logic execution step S130. At the same time, a parallel renewal process is triggered, namely, the asynchronous daemon thread is started in step S210.

[0034] After starting the asynchronous daemon thread in step S210, the thread enters the periodic batch atomic renewal step S220. Asynchronous daemon thread 13 will continuously send batch renewal requests to the server according to a preset period, independently of the main business process, to maintain the validity of the lock. This renewal process will continue until the main business logic is completed.

[0035] Once application 11 completes the task in business logic step S130, the main process enters the batch atomic unlocking step S140. In this step, client 10 sends an atomic request to the server to release all previously held locks. After the unlocking operation is completed, the process reaches the end step S150. Accordingly, the distributed lock manager 12 will also notify and terminate the corresponding asynchronous daemon thread 13.

[0036] Please refer to the following: Figure 3 The high-performance characteristics of the proposed solution are intuitively demonstrated through a signaling interaction sequence diagram. Whether it's the client application initiating a "batch locking script" or a "batch unlocking script," or the asynchronous daemon thread initiating a "batch renewal script," the interaction with the distributed storage service client follows a "single request - atomic execution - single response" pattern. This pattern contrasts sharply with the traditional approach that requires a network interaction for each lock, significantly reducing network round-trip time and optimizing network communication overhead from a linear complexity proportional to the number of locks to a constant complexity.

[0037] The specific execution process of the method in this embodiment will be described in detail below:

[0038] 1. Batch atomic locking stage (corresponding to) Figure 2 (S110 and S120 in the middle)

[0039] When client 10's business application 11 needs to lock a set of resources simultaneously (e.g., ['resource_A', 'resource_B', 'resource_C']), it will call an interface provided by the distributed lock manager 12, such as acquireLocks(keys, ttl).

[0040] First, the distributed lock manager 12 generates a globally unique lock holder identifier, which is used to uniquely identify the current lock holder throughout the distributed system. One possible generation method is to combine the client's unique identifier and the current thread's ID, in the form of client-uuid-1:thread-id-5. Assume the generated lock holder identifier is holder-001. Simultaneously, the lock's time-to-live parameter, such as 30000 milliseconds, is passed as a parameter during method calls.

[0041] Next, the distributed lock manager 12 constructs a locking instruction, which contains all the information required to perform batch locking, specifically: the list of resource keys to be locked ['resource_A', 'resource_B', 'resource_C'], the lock holder identifier holder-001, and the lock's time-to-live of 30000.

[0042] Then, the distributed lock manager 12 sends a single atomic request to the distributed storage service client 20. In a scenario where Redis is used as the server, this request is an EVAL command, used to execute the first atomic script (locking script) pre-installed on the server, passing the aforementioned locking instruction as a parameter. For example, the command format could be: EVAL<sha_of_lock_script> 3 'resource_A' 'resource_B' 'resource_C' 'holder-001' 30000, where the number 3 represents the number of resource keys.

[0043] After receiving the EVAL command, the distributed storage service client 20 will atomically execute the locking script. The internal logic of the script can be summarized as follows:

[0044] (1) The script first receives the passed parameters, namely the resource key list, lock holder identifier and time to live.

[0045] (2) The script initializes a temporary list to record the resources that have been successfully locked in this call, such as locked_keys_this_call.

[0046] (3) The script begins to iterate through each resource key in the resource key list. Taking the first resource, resource_A, as an example:

[0047] a. The script uses server-side commands (such as Redis's HGET) to check if the key resource_A exists. If it exists, it further checks the holder identifier stored internally.

[0048] b. If resource_A does not exist, it indicates that the resource is not locked. The script will create a new lock record, which can be achieved in Redis using the HSET command. This creates a hash structure containing a holder field (set to holder-001) and a reentry_count field (reentry count, initialized to 1). The script then uses the PEXPIRE command to set an expiration time of 30,000 milliseconds for the key resource_A and finally adds resource_A to the locked_keys_this_call list.

[0049] c. If resource_A already exists and its holder field value is exactly holder-001, it indicates that the current client is attempting to re-enter a lock already held by itself. In this case, the script will use the HINCRBY command to increment the reentry_count field by 1 and also refresh the key's expiration time to 30000 milliseconds.

[0050] d. If resource_A already exists but its holder field value is another identifier (e.g., holder-002), it indicates that lock contention has occurred and the current lock request has failed.

[0051] (4) Failure and Rollback: If any resource locking fails during the traversal (as in case (d) above), the script will immediately stop attempting to lock subsequent resources and trigger an automatic rollback mechanism. This mechanism traverses the `locked_keys_this_call` list (which records the resources successfully locked in this failed request) and performs a deletion operation (such as Redis's DEL command) on each resource key in the list. This ensures the overall atomicity of the batch locking operations and avoids leaving some successful "orphan locks" in the system. After the rollback is complete, the script returns a specific value indicating failure to the client, such as 0 or nil.

[0052] (5) Success: If the script successfully traverses and locks all requested resources without encountering any conflicts, it will return a value indicating success to the client, such as 1.

[0053] Back at client 10, the distributed lock manager 12 processes the return value received from the server. If the return value is a success flag, it records the successfully acquired lock information ['resource_A', 'resource_B', 'resource_C'] in the local status table 14 and immediately starts and runs the asynchronous daemon thread 13. If the return value is a failure flag, it throws an exception to the business application 11 or returns false.

[0054] 2. Batch Atom Renewal Stage (corresponding to) Figure 2 (S210 and S220 in the middle)

[0055] After successful batch locking, asynchronous daemon thread 13 starts, operating independently of the main thread executing business logic. In this embodiment, the renewal period is set to be less than the lock's lifetime; this is a crucial setting to ensure renewal is completed before the lock naturally expires. As a preferred practice, the renewal period can be set to 80% of the lock's lifetime, i.e., 30000 * 0.8 = 24000 milliseconds.

[0056] Asynchronous daemon thread 13 performs a renewal check every 24 seconds in its main loop. The specific steps are as follows:

[0057] (1) After the thread is awakened, it accesses and scans the local state table 14 to obtain the list of resource keys of all locks held by the current client, namely ['resource_A', 'resource_B', 'resource_C'].

[0058] (2) The thread constructs a renewal instruction containing a list of resource keys to be renewed and the lock holder identifier holder-001.

[0059] (3) The thread initiates a single atomic request to the distributed storage service client 20, calling the pre-defined second atomic script (renewal script). For example, the command could be: EVAL<sha_of_renew_script> 3 'resource_A' 'resource_B', 'resource_C' 'holder-001' 30000.

[0060] (4) The server-side renewal script is executed atomically. Its internal logic is as follows: it iterates through the list of input resource keys. For each resource key, the script first checks whether it exists and whether its holder field matches the input holder-001. Only if there is a complete match will the script use the PEXPIRE command to reset the key's time-to-live to 30,000 milliseconds. If the key does not exist or the holder does not match, the script ignores the resource and does not perform any operation. This verification mechanism ensures that the client will not mistakenly renew a lock that no longer belongs to it.

[0061] (5) After the script finishes execution, it can return a status, such as the number of locks that were successfully renewed. The client's asynchronous daemon thread 13 can log or process based on the return value, but under normal circumstances, as long as the renewal request is successfully sent, the renewal can be considered to be working normally.

[0062] 3. Batch atomic unlocking stage (corresponding to...) Figure 2 (S140 in the middle)

[0063] When the business logic of business application 11 is completed and the lock needs to be released, it will call the unlocking interface of distributed lock manager 12, such as releaseLocks().

[0064] (1) The distributed lock manager 12 obtains the list of resource keys ['resource_A', 'resource_B', 'resource_C'] and the lock holder identifier holder-001 from the local state table 14.

[0065] (2) Subsequently, it constructs an unlock command and initiates a single atomic request to the distributed storage service client 20, invoking a pre-defined third atomic script (unlock script). For example, the command could be: EVAL<sha_of_unlock_script> 3'resource_A', 'resource_B', 'resource_C' 'holder-001'.

[0066] (3) The server-side unlocking script is executed atomically, and its internal logic is as follows:

[0067] a. The script iterates through the list of passed-in resource keys. For each resource key (e.g., resource_A), the script first checks if it exists and if its holder field matches the passed-in holder-001.

[0068] b. If the holder does not match, it means the lock may have been released due to timeout or acquired by another client. The script will ignore operations on this resource, or it can log it as an exception and reflect it in the final return value.

[0069] c. If the holder matches, the script uses the HINCRBY command to decrement the value of the reentry_count field by 1.

[0070] d. The script then checks the new value of `reentry_count`. If the new value is greater than 0, it indicates that this is only an inner-level unlock of the reentrant lock, and the lock should not be truly released; the script only updates the count value. If the new value is equal to 0, it indicates that this is an outermost-level unlock, and the lock should be completely released. At this point, the script will use the `DEL` command to completely delete the resource key from the lock data storage area 22.

[0071] (4) After the script is executed, return the result to the client, such as the number of resources successfully unlocked.

[0072] (5) After receiving a successful response, the client's distributed lock manager 12 removes these unlocked resource records from the local state table 14 and sends a stop signal to the asynchronous daemon thread 13 to allow it to exit safely.

[0073] Through the close coordination of the above three stages, this embodiment realizes a distributed lock batch processing method that maintains high performance and strong atomicity throughout the entire lifecycle of locking, renewal, and unlocking.

[0074] Example 2

[0075] This embodiment is a variant implementation of the batch atomic renewal strategy, optimized based on Embodiment 1. In Embodiment 1, the asynchronous daemon thread 13 attempts to renew all locks held by the client each time. It is understandable that when the number of locks held by the client is very large, and these locks were acquired at different times with different expiration times, initiating renewal requests for all locks each time may cause unnecessary waste of network bandwidth and server processing resources. Therefore, this embodiment aims to achieve a more refined and intelligent renewal management mechanism.

[0076] The system architecture and Figure 1 The process is basically the same as shown, and the overall flow is also followed. Figure 2 The logic shown is as follows. The main difference lies in the implementation details of the distributed lock manager 12 and the asynchronous daemon thread 13 within client 10.

[0077] First, the data structure of the local state table 14 is expanded. In addition to recording resource keys, for each acquired lock, the local state table 14 will also record its "next theoretical renewal time point". This time point is calculated and updated when the lock is successfully acquired or successfully renewed. For example, if a lock has a lifetime of 30 seconds and a renewal period factor of 0.8, then at the time T0 when the lock is acquired, its "next theoretical renewal time point" is calculated as T0 + 30 * 0.8.

[0078] Secondly, the working logic of asynchronous daemon thread 13 was modified. It no longer wakes up every long fixed period (e.g., 24 seconds), but instead polls at a shorter frequency, such as once per second. This high-frequency, low-load polling method makes renewal checks more timely.

[0079] In each polling iteration of the asynchronous daemon thread 13, the following "intelligent renewal" steps are executed: 1. After being awakened, the thread iterates through the local state table 14. 2. For each lock record in the table, it checks whether its "next theoretical renewal time" is approaching. Specifically, the thread filters out all lock resources that satisfy the condition that the next theoretical renewal time is less than or equal to the current time plus a preset time threshold. The "preset time threshold" is a configurable parameter, such as 5 seconds. This means the thread will identify all locks that need to be renewed within the next 5 seconds. 3. The thread collects the resource keys of all the filtered locks, forming a set of resource keys to be renewed. 4. If the set is not empty, the thread constructs a renewal instruction and calls the second atomic script (renewal script) described in Example 1, performing batch renewal only on the locks within this set. 5. After the renewal request is sent, regardless of success or failure, the thread updates the "next theoretical renewal time" of all locks in the set in the local state table 14, updating it to the current time plus the lock's lifetime multiplied by the renewal period factor. This avoids repeatedly requesting renewals for the same batch of locks within a short period of time.

[0080] In this way, this embodiment achieves batching and peak smoothing of renewal requests. For example, if a client locks 1000 resources at time T0, and then locks another 500 resources at time T0+10, with all locks having a lifespan of 30 seconds, according to the strategy of Embodiment 1, the daemon thread will attempt to renew all 1500 locks at time T0+24. However, according to the strategy of this embodiment, around time T0+24, only the first 1000 locks will meet the renewal conditions and be renewed; and around time T0+34, the last 500 locks will meet the conditions and be renewed. Thus, the size of a single renewal request is effectively controlled, the instantaneous pressure on the server is alleviated, the renewal behavior is smoother, and the overall system stability is higher. It should be noted that the implementation details of this scheme support the technical features described in claims 4 and 8.

[0081] Example 3

[0082] This embodiment provides a distributed lock optimization method based on storage engine atomic scripts. It replaces the serial requests of traditional distributed locks with batch processing by atomic scripts, and combines an asynchronous lock renewal mechanism to solve the performance bottleneck in large-scale chain scenarios.

[0083] The distributed lock optimization method based on storage engine atomic scripts in this embodiment includes the following steps:

[0084] Step 1: Lock the original batch (atomic operation)

[0085] This step aims to complete batch locking operations on multiple resources through a single atomic script call, avoiding the network overhead and latency caused by traditional serial locking. The locking process has reentrancy support and a failure rollback mechanism to ensure the atomicity and consistency of the operation.

[0086] Step 1 specifically includes the following steps:

[0087] Step 1.1: The client constructs a lock request, which includes a list of resource keys (a set of resource identifiers to be locked), a lock holder identifier (a globally unique identifier of the current requester), and a lock time-to-live (TTL, in milliseconds).

[0088] Step 1.2: The client sends a single atomic request to the distributed storage service (such as Redis). This request carries a preloaded locking script, which is executed atomically on the server side.

[0089] Step 1.3: The locking script iterates through the resource key list and performs the following judgment for each resource:

[0090] If the resource is currently unlocked, a new lock is created. The lock content includes the lock holder identifier, a reentry count (initialized to 1), and an expiration time set to the current time + TTL.

[0091] If the resource already has a lock record and the holder is the same as the current requester, the reentry count will be incremented by 1 and the lock's expiration time will be refreshed.

[0092] If the resource has been locked by another holder, the script execution will be terminated immediately, and the location of the first conflicting resource will be recorded.

[0093] Step 1.4: After the script finishes execution, determine if there are any resources that failed to acquire a lock.

[0094] If there are any failed entries (i.e., the number of conflicting resources > 0), the script will automatically roll back - iterate through the resources that have been successfully locked, delete their lock records one by one, and ensure that there are no residual lock states;

[0095] If all resources are successfully locked, a success status is returned.

[0096] Step 1.5: The client updates its local state table based on the results returned by the script.

[0097] Upon success, the identifiers, holder identifiers, and last update times (i.e., locking completion times) of all locked resources are recorded in the local status table for subsequent renewal and unlocking.

[0098] In case of failure, return to the location of the first conflicting resource for upper-level business decision-making.

[0099] Step 2: Asynchronous lock renewal (background guardian)

[0100] This step is automatically activated after successful batch locking. A background daemon thread periodically renews the lock records in the local state table to prevent locks from being released unexpectedly due to TTL expiration. At the same time, it ensures that the renewal operation only applies to valid locks that belong to the same owner.

[0101] Step 2.1: After successful locking, the system automatically starts an asynchronous daemon thread, which runs independently of the main business process;

[0102] Step 2.2: The daemon thread polls all lock records in the local state table at a fixed period (default is 80% of TTL);

[0103] Step 2.3: For each lock record, determine whether the difference between the current time and the "last renewal time" exceeds the preset renewal threshold (usually slightly less than TTL):

[0104] If the threshold is not exceeded, the renewal will be skipped.

[0105] If the threshold has been exceeded, a renewal operation will be triggered;

[0106] Step 2.4: When renewal is triggered, a single atomic renewal script request is sent to the storage service. The script executes the following logic:

[0107] Verify whether the current lock holder of the target resource matches the local record;

[0108] If they match, refresh the lock's expiration time to the current time + TTL, and return "Renewal successful";

[0109] If there is no consistency (e.g., the lock has been taken by someone else), then "Renewal expired" will be returned;

[0110] Step 2.5: The client updates its local status based on the result returned by the renewal script.

[0111] For records of "successful renewal", update their "last renewal time";

[0112] Records that have "expired renewal" are marked as invalid and automatically removed in the next scan.

[0113] Step 2.6: The daemon thread continues to run until all related locks are explicitly unlocked or the local state table is cleared, at which point it will automatically terminate.

[0114] Step 3: Batch unlocking (atomic operation)

[0115] This step is used to release multiple held locks at once after the business logic is completed, ensuring the original...

[0116] To ensure security and prevent resource deadlocks or inconsistent states caused by partial unlocking failures.

[0117] Step 3.1: The client prepares an unlock request, which includes a list of resource identifiers to be unlocked and the identifier of the original lock holder (must be consistent with the one when the lock was acquired).

[0118] Step 3.2: Send a single atomic request to the storage service, carrying the preloaded unlock script, which is executed atomically on the server side;

[0119] Step 3.3: Unlock the script to traverse the resource identifier list and perform the following operations for each resource:

[0120] If a lock record exists for the resource and the holder is the same as the requester, the reentrancy count will be decremented by 1.

[0121] If the count is zeroed after decrementing by 1 and then re-entering, the lock record for that resource is deleted.

[0122] If the holder does not match, the resource index is recorded as "abnormal resource";

[0123] Step 3.4: After the script execution is complete, summarize the index of all abnormal resources (if any);

[0124] Step 3.5: The client performs local cleanup based on the results returned by the script:

[0125] If there are no abnormal resources, remove all corresponding records from the local status table;

[0126] If there are abnormal resources, the records of those that were not successfully unlocked in the local status table are retained (for logging or alarm purposes), and the abnormal resource index list is returned for processing by the upper layer.

[0127] Step 3.6: Regardless of success or failure, the unlocking operation ensures the correctness of the server-side lock state and avoids lock remnants due to client anomalies.

[0128] This embodiment addresses the performance bottleneck in large-scale chain scenarios. Traditional solutions suffer from the drawback of using serial lock operations, which causes network overhead to increase linearly with the number of locks (O(n) complexity), resulting in a sharp drop in system throughput due to high latency. This solution processes all lock operations in batches using a single atomic script, keeping the network overhead constant at O(1) (requiring only one request regardless of the number of locks). Actual performance improvements include reducing the time taken to operate on a thousand locks from >1s to <50ms.

[0129] This invention optimizes locking, renewal, and unlocking into a single network request, achieving high performance throughout the entire process. It also ensures strong atomicity and consistency through atomic scripts and rollback mechanisms, thereby enhancing the robustness of the system.

[0130] Specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various changes or modifications within the scope of the claims, which do not affect the essence of the present invention. Unless otherwise specified, the embodiments and features of the present invention can be arbitrarily combined with each other.

Claims

1. A distributed lock batch processing method based on atomic scripts, characterized in that, The method, executed by a client, is applied to a system comprising the client and a distributed storage service client, and includes: Batch atomic locking: A single atomic request is initiated to the distributed storage service to invoke a first atomic script pre-installed on the server, passing in a list of resource keys containing multiple resources, a lock holder identifier, and a lock lifetime instruction; so that the first atomic script is executed atomically on the server: iterating through the list of resource keys, for each resource, if the resource is not locked, a lock record containing the lock holder identifier and a reentrancy count initialized to 1 is created for it; if the resource is already locked by the lock holder of the current request, the reentrancy count of the corresponding lock record is incremented by 1; wherein, during the execution of the first atomic script, if locking any resource fails, an automatic rollback is triggered, and the lock records of the resources that were successfully locked in this request are deleted; Batch atomic renewal: After successful batch atomic locking, an asynchronous daemon thread is started. The asynchronous daemon thread periodically constructs renewal instructions containing a list of resource keys to be renewed and sends a single atomic request to the distributed storage service client to call a second atomic script pre-set on the server to realize batch renewal of locks in the list of resource keys to be renewed. Batch atomic unlock: Initiate a single atomic request to the distributed storage service client to call a third atomic script pre-installed on the server, and pass in the list of resource keys to be unlocked and the lock holder identifier; so that the third atomic script is executed atomically on the server: traverse the list of resource keys, and for each resource, if its lock holder is the same as the requester, decrement the reentrancy count of its lock record by 1, and delete the lock record of the resource when the reentrancy count reaches zero.

2. The distributed lock batch processing method based on atomic scripts according to claim 1, characterized in that, The client maintains a local state table to record lock information that has been successfully acquired, and the asynchronous daemon thread operates based on the local state table.

3. The distributed lock batch processing method based on atomic scripts according to claim 2, characterized in that, The renewal period of the asynchronous daemon thread is set to be shorter than the lifetime of the lock.

4. The distributed lock batch processing method based on atomic scripts according to claim 3, characterized in that, The asynchronous daemon thread scans the local state table to select a set of lock resources whose next renewal time is within a preset time threshold, and initiates the batch atomic renewal request only for lock resources in that set.

5. The distributed lock batch processing method based on atomic scripts according to claim 1, characterized in that, When the batch atomic locking fails, the first atomic script, after executing the rollback, also returns to the client the location or identifier of the first conflicting resource that caused the locking failure.

6. The distributed lock batch processing method based on atomic scripts according to claim 1, characterized in that, When processing an unlock request, if the third atomic script finds that the lock holder of the resource to be unlocked does not match the requester, it records the resource as an abnormal resource and returns it to the client.

7. A distributed lock batch processing system based on atomic scripts, comprising a client and a distributed storage service, characterized in that, The client includes: a lock request module, used to construct a lock instruction containing a list of resource keys for multiple resources, a lock holder identifier, and a lock lifetime, and initiate a single atomic request to the distributed storage service to call a first atomic script to implement batch atomic locking; a renewal management module, used to start and manage an asynchronous daemon thread after successful batch atomic locking, the asynchronous daemon thread periodically constructing a renewal instruction containing a list of resource keys to be renewed, and initiating a single atomic request to the distributed storage service to call a second atomic script to implement batch renewal of locks in the list of resource keys to be renewed; and an unlock request module, used to construct an unlock instruction containing a list of resource keys to be unlocked and a lock holder identifier, and initiate a single atomic request to the distributed storage service to call a third atomic script to implement batch atomic unlocking. The distributed storage service includes a script execution engine for atomically executing the first, second, and third atomic scripts; wherein, when executing the first atomic script, the script execution engine supports lock reentrancy counting and automatic rollback on failure; and when executing the third atomic script, it supports unlocking based on the reentrancy count.

8. The distributed lock batch processing system based on atomic scripts according to claim 7, characterized in that, The client's renewal management module is specifically used to: scan the local status table and filter out the set of lock resources whose next renewal time is within a preset time threshold.

9. The distributed lock batch processing system based on atomic scripts according to claim 8, characterized in that, The client's renewal management module only initiates batch atomic renewal requests for lock resources within the lock resource set.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the distributed lock batch processing method based on atomic scripts as described in any one of claims 1 to 6.

Images