A method for processing high-concurrency batch tasks and related products
By using Redis's distributed lock and thread buffer pool mechanisms in distributed task scheduling, the problems of task duplication and lock failure in high-concurrency batch tasks are solved, achieving efficient batch task processing and business stability.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SICHUAN XW BANK CO LTD
- Filing Date
- 2022-07-15
- Publication Date
- 2026-07-14
AI Technical Summary
Existing distributed batch task scheduling methods suffer from low single-task execution performance and easy failure of distributed locks when processing high-concurrency batch tasks. This is especially true when using distributed locks implemented with Zookeeper, databases, and Redis, where performance is insufficient or locks are prone to failure, affecting business execution.
We employ Redis's distributed task scheduling method, combined with a thread buffer pool mechanism. By creating Redis locks on servers in the same cluster, we ensure that locked threads are separated from unlocked threads. After the locked thread completes its business, it releases the lock. We use Redis's SETNX and expire methods to prevent lock invalidation and combine them with the sleep function to put threads to sleep under high concurrency, ensuring the uniqueness of the lock and the single execution of the business.
It improves the single execution of batch tasks, reduces the problems of task duplication and lock failure, ensures the uniqueness of tasks and the stable execution of business in high-concurrency environments, and reduces the risk of dirty data generated by multiple business servers executing batch tasks at the same time.
Smart Images
Figure CN115220891B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of computer technology, and more specifically to a method and related products for processing high-concurrency batch tasks. Background Technology
[0002] In the daily operations of financial systems, banking systems, e-commerce platforms, and other systems, there is often batch task scheduling. Among the existing distributed batch task scheduling methods, the more common solutions mainly include: using three major distributed locks (ZooKeeper, database table creation, and Redis) to implement task scheduling; or using third-party task scheduling platforms (such as Elastic-job, Xxl-job, and Saturn) to achieve unified configuration and monitoring of tasks across the entire domain. However, both of the above solutions face numerous problems in practical applications. For example, distributed locks implemented with Zookeeper are not as performant as caching services because they require dynamic creation and destruction of ephemeral nodes to implement the lock function each time a lock is created and released, resulting in low single-task execution. Distributed locks implemented with databases have no expiration time; once the unlocking operation fails, the lock record remains in the database, preventing other threads from acquiring the lock, which can easily lead to dirty data during batch task execution. Distributed locks implemented with Redis can easily cause lock expiration during high-concurrency batch tasks because thread A's completion may mistakenly kill thread B, affecting normal business execution. If a third-party scheduling platform is used, the resources and time required to connect to the third-party task scheduling platform are significant, and the reliance on the daily maintenance of the third-party platform results in limitations and poor scalability.
[0003] In view of the above, this application is hereby submitted. Summary of the Invention
[0004] The technical problem to be solved by the present invention is that existing common distributed batch task scheduling methods have low individual execution performance and the distributed lock is prone to failure when processing high-concurrency batch tasks. The purpose is to provide a method and related products for processing high-concurrency batch tasks to solve the above problems.
[0005] This invention is achieved through the following technical solution:
[0006] In a first aspect, embodiments of the present invention provide a method for processing high-concurrency batch tasks, comprising the following steps:
[0007] S1: Determine if there are batch scheduled tasks running simultaneously on servers in the same cluster, and obtain all threads;
[0008] S2: Create a Redis lock in all threads, and obtain the locked threads and the remaining blocked, unlocked threads;
[0009] S3: Place the locked thread and the unlocked thread into the thread buffer pool for buffering, while the locked thread executes the relevant business logic.
[0010] S4: After the locking thread finishes its business logic, release the Redis lock and return to step S2.
[0011] In an optional embodiment of this application, before determining whether there are concurrently executing batch scheduled tasks on servers in the same cluster, the method further includes:
[0012] A1: Get multi-threaded concurrent processing requests;
[0013] A2: Create concurrent threads to be processed based on the multi-threaded concurrent processing requests, and send the concurrent processing threads to the sub-servers.
[0014] In an optional embodiment of this application, creating a Redis lock across all threads includes:
[0015] S21: Generate a Redis lock, obtain the key value of the Redis lock, set the key value as the value, and call the SETNX function to get the return value of the value;
[0016] S22: If the return value is false, obtain the locking thread and the remaining blocked unlocked threads, and proceed to step S3;
[0017] If the return value is true, perform the polling operation until the return value is false.
[0018] In one optional embodiment of this application, during the polling operation,
[0019] If the Redis lock is a non-blocking call, then return to step S21;
[0020] If the Redis lock is a blocking call type, the Redis lock will be polled within a specified time until it is successfully acquired.
[0021] In one optional embodiment of this application, during the polling of the Redis lock within a specified time period, the expire method is used to set a specified time. When the polling time exceeds the set specified time, the Redis lock is automatically destroyed.
[0022] In an optional embodiment of this application, during the buffering process in step S3, based on the number of requests per second in Redis in the system and the lock holding time of the locking thread, the sleep function is called to put the unlocked thread to sleep.
[0023] In one optional embodiment of this application, the time precision in the sleep function is at the millisecond level.
[0024] Secondly, embodiments of the present invention provide a system for processing high-concurrency batch tasks, comprising:
[0025] The thread acquisition module is used to determine whether there are batch scheduled tasks running simultaneously on servers in the same cluster, and to obtain all threads.
[0026] The locking module is used to create Redis locks in all threads, and to obtain the locking threads and the remaining blocked, unlocked threads;
[0027] The execution module is used to put locked threads and unlocked threads into the thread buffer pool for buffering, while the locked threads execute related business logic.
[0028] The return module is used to release the Redis lock after the locking thread has completed its business operations, and then return to the locking module repeatedly.
[0029] Thirdly, embodiments of the present invention provide an electronic device, including a memory and a processor, wherein the memory stores a computer program, characterized in that the processor executes the computer program to implement the method for processing high-concurrency batch tasks as described above.
[0030] Fourthly, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a method for processing high-concurrency batch tasks as described above.
[0031] Compared with the prior art, the present invention has the following advantages and beneficial effects:
[0032] This invention utilizes Redis's distributed task scheduling method as a foundation to solve the problem of duplicate task registration. It also allocates tasks to several processors for specific business processing, ensuring that a task is processed by only one processor at a time, thereby improving the single-processability of batch tasks. Furthermore, a thread buffering mechanism is added to effectively address the issue of Redis distributed locks easily failing under high concurrency. In addition, the combination of Redis distributed locks and thread buffering creates a lightweight architecture design in the application, which helps strengthen the principle of single-processing of distributed tasks, reducing dirty data generated by multiple business servers simultaneously executing batch tasks, and ultimately ensuring unified deployment and management of the application. Attached Figure Description
[0033] To more clearly illustrate the technical solutions of the exemplary embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly described below. It should be understood that the following drawings only show some embodiments of the present invention and should not be considered as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort. In the drawings:
[0034] Figure 1 A flowchart illustrating a method for processing high-concurrency batch tasks, provided in an embodiment of the present invention;
[0035] Figure 2 A flowchart illustrating the actual high-concurrency batch task control using distributed locks and thread buffers provided in this embodiment of the invention;
[0036] Figure 3 This is a schematic diagram of a system architecture for processing high-concurrency batch tasks, provided as an embodiment of the present invention. Detailed Implementation
[0037] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.
[0038] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.
[0039] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.
[0040] The terms “first,” “second,” “third,” and “fourth,” etc. (if present), in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.
[0041] With the construction of new systems and the upgrading and expansion of old systems in the financial industry, the operations and maintenance department is taking over the maintenance of more and more application systems. Every day, dozens of systems need to execute daily task switching, and nearly a million batch jobs need to be processed, resulting in a very heavy workload. If a task fails, it is difficult to troubleshoot in a timely manner, leading to very low work efficiency. The emergence of distributed task scheduling aims to coordinate multiple nodes to process the same tasks, avoiding duplicate processing of business logic and improving the processing efficiency of batch tasks. However, in current mainstream distributed task scheduling methods, the single execution of batch tasks is too low. A task may be registered repeatedly, making it impossible to guarantee that a task will only be processed by one processor at the same time. Furthermore, under high concurrency, distributed locks are prone to failure. Using third-party scheduling systems often faces difficulties such as the lack of free technical support, high maintenance costs, and low security.
[0042] To address the above problems, embodiments of the present invention provide a method for handling high-concurrency batch tasks, such as... Figure 1 and Figure 2 As shown, it includes the following steps:
[0043] S1: Determine if there are batch scheduled tasks running simultaneously on servers in the same cluster, and obtain all threads;
[0044] The purpose of using a server cluster is to connect multiple systems together, enabling multiple servers to work like a single machine, thereby improving system stability and the data processing and service capabilities of the network center. Returning to this embodiment, it is determined that there are three task servers: Task Server A, Task Server B, and Task Server C. Each server is in the same cluster and has the same function of initiating batch scheduled tasks. When a task begins execution, it enters the Redis lock management of the next node. It is worth noting that the number of servers in the same cluster is not specifically limited, and the server cluster type can be either a manual cluster or an automatic cluster. The deployment methods used are conventional methods known to those skilled in the art and will not be elaborated here.
[0045] S2: Create a Redis lock in all the threads, and obtain the locking thread and the remaining blocked unlocked threads;
[0046] Redis (Remote Dictionary Server) is a cross-platform, non-relational database with superior performance, supporting hundreds of thousands of read / write operations per second. It supports configurations such as clustering, distributed systems, and master-slave synchronization. In financial industry scenarios, multiple processes often need to exclusively access shared resources in a mutually exclusive manner. In such cases, distributed locks are the most direct and effective solution. This embodiment proposes a distributed lock based on Redis, which is simple to implement. Compared to databases and distributed systems, this solution has a lighter architecture and better performance.
[0047] S3: Place the locked thread and the unlocked thread into the thread buffer pool for buffering, while the locked thread performs related business;
[0048] S4: After the business operation of the locking thread is completed, release the Redis lock and return to step S2.
[0049] This invention utilizes Redis's distributed task scheduling method as a foundation to solve the problem of duplicate task registration. It also allocates tasks to several processors for specific business processing, ensuring that a task is processed by only one processor at a time, thereby improving the single execution performance of batch tasks. Furthermore, a thread buffering and waiting mechanism is added to effectively solve the problem of Redis distributed locks easily failing under high concurrency.
[0050] In an optional embodiment of this application, before determining whether there are concurrently executing batch scheduled tasks on servers in the same cluster, the method further includes:
[0051] A1: Get multi-threaded concurrent processing requests;
[0052] A2: Create a concurrent thread to be processed according to the multi-threaded concurrent processing request, and send the concurrent processing thread to the sub-server;
[0053] The sub-servers can be servers providing various services, such as backend servers processing data displayed on terminal devices. The backend servers can analyze and process received concurrency and feed the processing results back to the terminal devices. It should be noted that servers can be hardware or software. When the server is hardware, it can be implemented as a distributed server cluster consisting of multiple servers, or as a single server. When the server is software, it can be implemented as multiple software programs or software modules (e.g., multiple software programs or software modules used to provide distributed services), or as a single software program or software module. No specific limitations are made here.
[0054] In an optional embodiment of this application, creating a Redis lock in all threads includes:
[0055] S21: Generate a Redis lock, obtain the key value of the Redis lock, set the key value as the value, and call the SETNX function to obtain the return value of the value;
[0056] S22: If the return value is false, obtain the locking thread and the remaining blocked unlocked threads, and proceed to step S3.
[0057] If the return value is true, perform the polling operation until the return value is false.
[0058] The method for locking is the setIfAbsent method.
[0059] In one optional embodiment of this application, during the polling operation,
[0060] If the Redis lock is a non-blocking call, then return to step S21;
[0061] If the Redis lock is a blocking call, it will be polled within a specified time until it is successfully acquired. However, if the client attempting to acquire the lock takes too long, is killed, or crashes due to other exceptions, preventing the lock from being released, a deadlock will occur. Deadlock has a fatal impact on business operations; since the program cannot correctly acquire the lock, business operations will remain suspended.
[0062] For the reasons mentioned above, in an optional embodiment of this application, the `expire` method is used to set a specified time during the polling of the Redis lock within a specified time. When the polling time exceeds the set specified time, the Redis lock is automatically destroyed. Specifically, in this embodiment, when locking, the `expire` method needs to be used to set the timeout time for the key. When the polling time exceeds the set timeout time, the key is automatically destroyed, which means the lock has expired, preventing the lock from being locked indefinitely. The methods used for polling and key destruction are conventional methods known to those skilled in the art and will not be described in detail here.
[0063] Furthermore, in high-concurrency scenarios, if lock failures are detected simultaneously, and deadlocks are simply and crudely deleted before re-locking using `setIfAbsent`, a race condition may occur, meaning multiple servers may acquire the lock concurrently. Acquiring multiple locks simultaneously is also a serious problem for business operations. Scheduled tasks are supposed to execute only once, but due to lock issues, they may execute multiple times, causing duplicate execution of business logic. Therefore, it's not advisable to directly delete the lock after a timeout.
[0064] To address the aforementioned issues, in an optional embodiment of this application, during the buffering process in step S3, based on the Redis requests per second in the system and the lock holding time of the locking thread, the `sleep` function is invoked to put the unlocked thread to sleep. The purpose is to ensure the uniqueness of the distributed lock in a multi-server environment. Specifically, in this embodiment, the specific parameters of the `sleep` function's sleep time need to be calculated reasonably based on the actual business scenario, combined with the Redis QPS (requests per second) of the system, and the lock holding time. The specific formula is as follows:
[0065] QPS = Concurrency / Average latency;
[0066] Sleep time = (QPS*K)*N / Business duration;
[0067] Where: K is a coefficient between 0 and 1, and N is a random integer between 1 and 1000.
[0068] In real-world scenarios, time differences exist between servers, which can lead to discrepancies in determining whether a lock has timed out, resulting in race conditions. Furthermore, lock timeout depends strictly on the timestamp; however, timestamps themselves have precision limitations. If the time precision is in seconds, meaning the entire process from locking to execution and unlocking must be completed within one second, to prevent thread race conditions, in an optional embodiment of this application, the sleep function uses millisecond-level precision, thus ensuring that millisecond-level locks are safe.
[0069] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily essential to this application.
[0070] Secondly, embodiments of the present invention provide a system for processing high-concurrency batch tasks, such as... Figure 3 As shown, it includes:
[0071] The thread acquisition module is used to determine whether there are batch scheduled tasks running simultaneously on servers in the same cluster, and to obtain all threads.
[0072] The locking module is used to create Redis locks in all the threads, and obtain the locking threads and the remaining blocked, unlocked threads;
[0073] The execution module is used to put the locked thread and the unlocked thread into the thread buffer pool for buffering, while the locked thread performs related business.
[0074] The return call module is used to release the Redis lock and return to the locking module repeatedly after the business execution of the locking thread is completed.
[0075] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the described module can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.
[0076] Thirdly, embodiments of the present invention provide an electronic device including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement a method for processing high-concurrency batch tasks as described above.
[0077] The device includes a central processing unit (CPU), which can perform various appropriate actions and processes based on computer program instructions stored in read-only memory (ROM) or loaded from storage units into random access memory (RAM). The RAM can also store various programs and data required for device operation. The CPU, ROM, and RAM 603 are interconnected via a bus. Input / output (I / O) interfaces are also connected to the bus.
[0078] Returning to the present invention, the method for processing high-concurrency batch tasks provided in the embodiments of the present invention can be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit. In some embodiments, part or all of the computer program can be loaded and / or installed on a device via ROM and / or a communication unit. When the computer program is loaded into RAM and executed by the CPU, one or more steps of the method for processing high-concurrency batch tasks described above can be performed. Alternatively, in other embodiments, the CPU can be configured to execute the method by any other suitable means (e.g., by means of firmware).
[0079] Fourthly, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a method for processing high-concurrency batch tasks as described above.
[0080] The computer-readable storage medium may include random access memory (RAM), memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, register, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.
[0081] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.
Claims
1. A method for handling high-concurrency batch tasks, characterized in that, Includes the following steps: S1: Determine if there are batch scheduled tasks running simultaneously on servers in the same cluster, and obtain all threads; S2: Create a Redis lock in all the threads, and obtain the locking thread and the remaining blocked unlocked threads; S3: Place the locked thread and the unlocked thread into the thread buffer pool for buffering, while the locked thread performs related business; S4: After the business logic of the locking thread is completed, release the Redis lock and return to step S2; In the buffering process of step S3, based on the number of requests per second in Redis in the system and the lock holding time of the locking thread, the sleep function is called to put the unlocked thread to sleep; the calculation formula for the sleep time of the sleep function is as follows: QPS = Concurrency / Average latency; ； Where K is a coefficient between 0 and 1, N is a random integer between 1 and 1000, QPS is the number of requests per second of the Redis instance, and the business time consumption is the lock holding time of the locking thread.
2. The method for processing high-concurrency batch tasks according to claim 1, characterized in that, Before determining whether there are concurrent batch scheduled tasks running on servers within the same cluster, the following steps are also included: A1: Get multi-threaded concurrent processing requests; A2: Create a concurrent thread to be processed according to the multi-threaded concurrent processing request, and send the concurrent processing thread to the sub-server.
3. The method for processing high-concurrency batch tasks according to claim 1, characterized in that, The creation of Redis locks in all the aforementioned threads includes: S21: Generate a Redis lock, obtain the key value of the Redis lock, set the key value as the value, and call the SETNX function to obtain the return value of the value; S22: If the return value is false, obtain the locking thread and the remaining blocked unlocked threads, and proceed to step S3; If the return value is true, perform the polling operation until the return value is false.
4. The method for processing high-concurrency batch tasks according to claim 3, characterized in that, In the polling operation, If the Redis lock is a non-blocking call, then return to step S21; If the Redis lock is a blocking call type, the Redis lock will be polled within a specified time until it is successfully acquired.
5. The method for processing high-concurrency batch tasks according to claim 4, characterized in that, When polling a Redis lock within a specified time, the expire method is used to set a time limit. If the polling time exceeds the set time limit, the Redis lock will be automatically destroyed.
6. The method for processing high-concurrency batch tasks according to claim 1, characterized in that, The time precision in the sleep function is at the millisecond level.
7. A system for processing high-concurrency batch tasks, characterized in that, include: The thread acquisition module is used to determine whether there are batch scheduled tasks running simultaneously on servers in the same cluster, and to obtain all threads. The locking module is used to create Redis locks in all the threads, and obtain the locking threads and the remaining blocked, unlocked threads; The execution module is used to place the locked thread and the unlocked thread into a thread buffer pool for buffering processing, while the locked thread executes related business logic. During the buffering process, based on the Redis requests per second in the system and the lock holding time of the locked thread, the sleep function is called to put the unlocked thread to sleep. The sleep time of the sleep function is calculated as follows: QPS = Concurrency / Average Elapsed Time. Where K is a coefficient between 0 and 1, N is a random integer between 1 and 1000, QPS is the number of requests per second of the Redis instance, and the business consumption time is the lock holding time of the locking thread. The return call module is used to release the Redis lock and return to the locking module repeatedly after the business execution of the locking thread is completed.
8. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements a method for processing high-concurrency batch tasks as described in any one of claims 1 to 6.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements a method for processing high-concurrency batch tasks as described in any one of claims 1 to 6.