A data processing method, device and storage medium

Abstract

The present application discloses a method, device and storage medium for data processing, specifically, obtaining a task set submitted by a thread pool and a task identifier corresponding to the task set, wherein the task set contains at least one pending task, and the mapping relationship between the task set and the task identifier corresponding to the task set is saved. In response to the thread pool creating a task thread, at least one pending task in the task set matching the task identifier of the thread pool is screened out based on the mapping relationship, and the thread pool obtains at least one corresponding pending task, and caches the obtained pending task to the blocking work queue corresponding to the thread pool, waiting to be executed by the corresponding task thread. The embodiment of the present application improves data processing efficiency by centrally submitting and saving the pending tasks of each thread pool, and then obtaining the corresponding pending tasks after the subsequent thread pool creates a thread, thereby avoiding the risk of service unavailability due to memory overflow caused by the memory upper limit of each server.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a data processing method, device and storage medium. Background Art

[0002] In daily development work, in order to improve system concurrency and throughput, the thread pool plays an important role in an application. According to the operating principle of the thread pool, it is mainly divided into two steps. The first is to add tasks and put them into the blocking queue (such as bounded queue ArrayBlockingQueue, unbounded queue LinkedBlockingQueue, delay queue DelayQueue, lock queue SynchronousQueue, etc.). In this stage, the storage of tasks mainly consumes server resources, namely memory; the second is to execute tasks. In this stage, the main consumption of server resources is the central processing unit (CPU). However, memory space is a limited resource, which increases memory consumption in the process of adding and executing tasks. When the memory limit is reached, memory overflow will cause the system to be unavailable. When the concurrency continues to increase, extreme cases will cause all services to crash. If you use a bounded queue ArrayBlockingQueue, when you submit more tasks than the queue capacity, the rejection strategy will be triggered and the task cannot be executed normally; or if you use an unbounded queue LinkedBlockingQueue, continuous submission of tasks will cause memory overflow. When using the existing blocking queue of the thread pool, when the query rate per second is high, memory or business logic abnormalities may occur. Summary of the invention

[0003] The embodiment of the present application provides a method for data processing, which overcomes the problem of service unavailability that may occur in each application server due to memory overflow.

[0004] The method includes:

[0005] Obtaining a task set submitted by at least one thread pool and a task identifier corresponding to the task set, wherein the task set includes at least one task to be processed;

[0006] Saving the mapping relationship between the task set and the task identifiers corresponding to the task set;

[0007] In response to the thread pool creating a task thread, at least one of the to-be-processed tasks in the task set that matches the task identifier of the thread pool is screened out based on the mapping relationship;

[0008] The thread pool obtains at least one corresponding task to be processed, and caches the obtained task to be processed in the blocking work queue corresponding to the thread pool, waiting to be executed by the corresponding task thread.

[0009] Optionally, the acquired task data contained in the task to be processed is serialized and converted into a byte sequence corresponding to the task to be processed.

[0010] Optionally, when saving the task to be identified, the storage time of the task to be identified is counted;

[0011] In response to the saving time length exceeding a preset time threshold corresponding to the task set to which the task to be identified belongs, the operation of saving the task to be identified is stopped.

[0012] Optionally, if the task to be processed is not obtained in the task set that conforms to the mapping relationship with the task identifier sent by the received thread pool, an operation of waiting to obtain the task to be processed is performed until the task thread of the thread pool corresponding to the task set is interrupted, and / or the task set is empty for longer than a preset time, then the operation of obtaining the task to be processed in the task set is stopped.

[0013] Optionally, the byte sequence contained in the acquired task to be processed is deserialized into a task data object.

[0014] Optionally, the number of the saved task sets and the number of the to-be-processed tasks contained in the task sets, as well as the number of the thread pools and the number of the task threads are queried, and the number of application servers where the thread pools are located is adjusted.

[0015] In another embodiment of the present invention, a data processing device is provided, the device comprising:

[0016] An acquisition module, used to acquire a task set submitted by at least one thread pool and a task identifier corresponding to the task set, wherein the task set includes at least one task to be processed;

[0017] A saving module, used for saving the mapping relationship between the task set and the task identifiers corresponding to the task set;

[0018] A screening module, configured to, in response to the thread pool creating a task thread, screen out at least one of the to-be-processed tasks in the task set that matches the task identifier of the thread pool based on the mapping relationship;

[0019] The cache module is used for the thread pool to obtain at least one corresponding task to be processed, and cache the obtained task to be processed to the blocked work queue corresponding to the thread pool, waiting to be executed by the corresponding task thread.

[0020] Optionally, the acquisition module is further used for:

[0021] The acquired task data contained in the task to be processed is serialized and converted into a byte sequence corresponding to the task to be processed.

[0022] In another embodiment of the present invention, a non-transitory computer-readable storage medium is provided, wherein the non-transitory computer-readable storage medium stores instructions, and when the instructions are executed by a processor, the processor executes each step in the above-mentioned data processing method.

[0023] In another embodiment of the present invention, a terminal device is provided, including a processor, wherein the processor is used to execute each step in the above-mentioned data processing method.

[0024] Based on the above embodiment, firstly, a task set submitted by at least one thread pool and a task identifier corresponding to the task set are obtained, wherein the task set contains at least one pending task. Secondly, a mapping relationship between the task set and the task identifier corresponding to the task set is saved. Furthermore, in response to the thread pool creating a task thread, at least one pending task in the task set matching the task identifier of the thread pool is screened out based on the mapping relationship. Finally, the thread pool obtains at least one corresponding pending task, and caches the obtained pending task to the blocking work queue corresponding to the thread pool, waiting to be executed by the corresponding task thread. The embodiment of the present application improves data processing efficiency by centrally submitting and saving the pending tasks of each thread pool, and then obtaining the corresponding pending tasks after the subsequent thread pool creates a thread, thereby avoiding the risk of service unavailability due to memory overflow caused by the memory upper limit of each server. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0026] Figure 1 A schematic diagram showing a data interaction scenario of a data processing method provided in embodiment 100 of the present application is shown;

[0027] Figure 2A schematic diagram showing a data processing method provided by embodiment 200 of the present application;

[0028] Figure 3 A schematic diagram showing a specific process of a data processing method provided by embodiment 300 of the present application;

[0029] Figure 4 A schematic diagram showing a data processing device further provided by the embodiment 400 of the present application;

[0030] Figure 5 A schematic diagram of a terminal device provided by embodiment 500 of the present application is shown. DETAILED DESCRIPTION

[0031] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0032] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the implementation of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0033] Based on the problems in the prior art, the present application provides a data processing method, which is mainly applicable to the field of computer technology. Figure 1 As shown, it is a schematic diagram of a data interaction scenario of a data processing method provided by the embodiment 100 of the present application. By centrally submitting the pending tasks to be processed by each server using the thread pool to the central task server, and then identifying and obtaining the corresponding pending tasks from the central task server after the thread pool creates the task thread and executing them, the processing efficiency of each server thread pool is improved, and the risk of service unavailability caused by memory overflow is reduced.

[0034] Furthermore, if Figure 2FIG. 2 is a flow chart of a data processing method provided by Embodiment 200 of the present application. The detailed steps are as follows:

[0035] Step S11, obtaining a task set submitted by at least one thread pool and a task identifier corresponding to the task set, wherein the task set includes at least one task to be processed.

[0036] In this step, after each application server creates a thread pool for processing its own corresponding tasks, it submits a task set consisting of at least one pending task to be processed to the central task server. The central task server is independent of each application server. When submitting the task set corresponding to each thread pool, the task identifier corresponding to the task set is submitted at the same time. Each task identifier corresponds to the corresponding task one by one.

[0037] Step S12: Save the mapping relationship between the task set and the task identifier corresponding to the task set.

[0038] In this step, after receiving the task sets and corresponding task identifiers submitted by each thread pool, the central task server establishes a mapping relationship between the task sets and the corresponding task identifiers, and saves the task sets and the corresponding mapping relationship.

[0039] Step S13, in response to the thread pool creating a task thread, at least one to-be-processed task in the task set that matches the task identifier of the thread pool is screened out based on the mapping relationship.

[0040] In this step, after submitting the task, the thread pool creates a corresponding task thread based on the pending tasks that need to be executed. The central task server filters out the task set corresponding to the mapping relationship that matches the task identifier of the thread pool from the saved task set, and obtains at least one pending task in the task set.

[0041] Step S14, the thread pool obtains at least one corresponding task to be processed, and caches the obtained task to be processed in a blocking work queue corresponding to the thread pool, waiting to be executed by the corresponding task thread.

[0042] In this step, each thread pool corresponds to a blocking work queue workQueue for storing pending tasks waiting to be executed, so that the current system can create task threads based on the number of central processors to process the pending tasks cached in the task work queue. Further, the pending tasks corresponding to the current thread pool obtained in the central task server are cached in the blocking work queue, waiting for the execution of subsequent task threads.

[0043] As described above, based on the above embodiments, firstly, a task set submitted by at least one thread pool and a task identifier corresponding to the task set are obtained, wherein the task set contains at least one pending task, and secondly, a mapping relationship between the task set and the task identifier corresponding to the task set is saved, and further, in response to the thread pool creating a task thread, at least one pending task in the task set that matches the task identifier of the thread pool is screened out based on the mapping relationship, and finally, the thread pool obtains at least one corresponding pending task, and caches the obtained pending task to the blocking work queue corresponding to the thread pool, waiting to be executed by the corresponding task thread. The embodiment of the present application improves data processing efficiency by centrally submitting and saving the pending tasks of each thread pool, and then obtaining the corresponding pending tasks after the subsequent thread pool creates a thread, thereby avoiding the risk of service unavailability due to memory overflow caused by the memory upper limit of each server.

[0044] like Figure 3 FIG. 3 is a schematic diagram of a specific process of a data processing method shown in Embodiment 300 of the present application. The specific process is as follows:

[0045] S201, creating a central task server.

[0046] Here, the central task server in the embodiment of the present application is used to obtain and store the pending tasks that each application server needs to execute. Among them, the central task server is preferably Redis middleware. Specifically, since Redis uses memory as the data storage medium, just like the traditional thread pool to save tasks, the efficiency of reading and writing data is extremely high, far exceeding the relational database and providing rich data types. At the same time, Redis also has the following characteristics:

[0047] (1) Fast response

[0048] Redis responds very quickly, and can perform about 110,000 read operations or 81,000 write operations per second, which is much faster than a database. If you store some commonly used data, you can effectively improve the performance of the system.

[0049] (2) Support multiple data types

[0050] Redis supports data types such as strings, hash structures, lists, sets, and sortable sets. Among them, some Java basic data types can be stored in strings, hashes can store objects, lists can store List objects, etc. This makes it easy to select the data type to be stored according to your needs in the application, which is convenient for development. In the embodiment of the present application, the data structure of the list collection is adopted. Since the types executed by each thread pool are different, the required task identification Key can be customized according to the business logic for easy reading.

[0051] (3) Operation Atomicity

[0052] The operations of application servers accessing Redis are all atomic, ensuring that when two application servers access the Redis server at the same time, they get the updated value (the latest value). When multiple task threads obtain pending tasks, since Redis is single-threaded, only one task thread can obtain pending tasks at the same time, ensuring data consistency.

[0053] (4) Persistent Data and Disaster Recovery

[0054] Currently, there are two ways of Redis persistence: RDB (Redis database) and AOF (append only file), which can save the data in the server memory to the disk to prevent data loss due to server restart. In addition, a Redis cluster can be built. When a server node fails, another server node can be switched through sentinel mode to continue providing services. This ensures data security and prevents data loss due to downtime in traditional systems.

[0055] S202, obtaining a task set submitted by at least one thread pool and a task identifier corresponding to the task set.

[0056] Here, the embodiment of the present application takes the creation of a thread pool ThreadPoolExecutor as an example. The input object for creating a thread pool is a task blocking queue workerQueue, which is a BlockingQueue <runnable>Type, so you need to customize the task blocking queue to inherit the object and rewrite the method of saving tasks in the object.

[0057] Since the central server can be expanded and there is no shortage of space, the two methods offer(Ee) and put(Ee) can be combined, and a method parameter String taskName can be added to submit the data directly to the central task server. TaskName can be used as the task identifier of the task collection. Different thread pools submit different task identifiers taskName.

[0058] S203, serializing the task data contained in the acquired task to be processed.

[0059] Here, the task data contained in the acquired pending tasks is serialized and converted into a byte sequence corresponding to the pending tasks. Specifically, since the central task server does not identify whether it is a pending task of the thread pool or other data when saving data, the data saved by the central task server is only binary data. Therefore, when the thread pool submits a pending task, it should serialize the pending task and convert the pending task represented as a task object into a byte sequence, such as serializing it into a json string and submitting it to the central task server.

[0060] S204: The central task server saves the mapping relationship between the task set and the task identifier corresponding to the task set.

[0061] In this step, when the central task server obtains the task set and the corresponding task identifier, it saves the mapping relationship between the task set and the corresponding task identifier. Furthermore, when saving the task to be identified, the saving time of the task to be identified is counted. In response to the saving time exceeding the preset time threshold corresponding to the task set to which the task to be identified belongs, the operation of saving the task to be identified is stopped. Specifically, a method input parameter String taskName is added through the offer(Ee,long timeout,TimeUnitunit) method to obtain the preset time threshold timeout, and null is returned if the timeout occurs.

[0062] S205, the thread pool creates a task thread.

[0063] In this step, after the application server creates the thread pool, by default, there is no task thread in the thread pool. Instead, it waits for pending tasks to arrive before creating task threads to execute pending tasks. In the addThread method, a Worker object is first created with the submitted pending tasks, then the thread factory threadFactory is called to create a new task thread, and then the reference of the task thread is assigned to the member variable thread of the Worker object, and then the Worker object is added to the working set through workers.add(w). After the thread is created, the thread enters the Ready (Runnable) state.

[0064] S206, filtering out a task set corresponding to the thread pool in the central task server.

[0065] Here, in response to the thread pool creating a task thread, at least one pending task in the task set that matches the task identifier of the thread pool is screened out based on the mapping relationship. Specifically, in the task set that matches the mapping relationship with the task identifier sent by the received thread pool, if no pending task is obtained, the operation of waiting to obtain the pending task is performed until the task thread of the thread pool corresponding to the task set is interrupted, and / or the task set is continuously empty for a period of time exceeding the preset period of time, and the operation of obtaining the pending tasks in the task set is stopped. Further, the pending tasks in the corresponding task set are obtained using methods such as poll, take, poll(longtimeout,TimeUnit unit). Specifically, poll(): If there is no pending task in the corresponding task set, null is returned directly; if there is an element, it is obtained. take(): If the task set is empty, wait until there is a pending task or the corresponding task thread is interrupted. poll(long timeout,TimeUnit unit): If the task set is not empty, it is obtained; if the queue is empty and has timed out, null is returned; if the queue is empty and the time TimeUnit has not timed out, it enters waiting. These three methods also need to add the method input parameter task identifier taskName to obtain the pending tasks from the task collection specified in the central task server.

[0066] S207: The thread pool obtains at least one corresponding task to be processed.

[0067] Here, the thread pool obtains the pending tasks stored in the corresponding task collection on the central task server.

[0068] S208, the thread pool deserializes the task data contained in the acquired task to be processed.

[0069] Here, the byte sequence contained in the acquired pending task is deserialized into a task data object. Specifically, since the data acquired by the thread pool of each application server from the central task server is not identifiable data but a string, it should be deserialized into a task object so that it can be recognized by the thread pool.

[0070] S209: Cache the acquired pending tasks into the blocking work queue corresponding to the thread pool, waiting to be executed by the corresponding task thread.

[0071] In this step, each thread pool corresponds to a task cache queue, namely workQueue, which is used to store pending tasks waiting to be executed. Among them, the type of workQueue is BlockingQueue <runnable>, usually three types can be selected: ArrayBlockingQueue (a first-in-first-out queue based on an array, the size of this queue must be specified when it is created); LinkedBlockingQueue (a first-in-first-out queue based on a linked list. If the queue size is not specified when it is created, the default is Integer.MAX_VALUE); synchronousQueue (this queue is special. It will not save the submitted tasks, but will directly create a new thread to execute the new tasks).

[0072] Furthermore, each task thread first executes the pending task firstTask passed in by the constructor. After calling runTask() to execute firstTask, it continuously gets new pending tasks from the blocking work queue through getTask() in the while loop for execution. getTask is a method in the ThreadPoolExecutor class. If the number of task threads in the current thread pool is greater than the core pool size corePoolSize or the idle survival time can be set for the task threads in the core pool, poll(time, timeUnit) is called to get the pending tasks. This method will wait for a certain period of time and return null if no pending tasks can be obtained.

[0073] S210, adjusting the number of servers where the thread pool is located based on the central task server.

[0074] Here, the number of saved task sets and the number of pending tasks contained in the task sets, as well as the number of thread pools and the number of task threads, are queried, and the number of application servers where the thread pools are located is adjusted. The number of application servers can be reduced or increased by querying the number of pending tasks. This step can be performed during the entire data processing process.

[0075] This application implements the above-mentioned data processing method based on the above steps. By extracting the pending tasks that need to be executed by the thread pool of each application server and saving them to an independent central task server, and obtaining the corresponding pending tasks after establishing the task thread, the risk of unavailability of the service due to memory overflow caused by the memory upper limit of each application server is avoided. At the same time, the status of the submitted pending tasks can be queried through the central task server to appropriately adjust the number of application servers to prevent waste of resources. In addition, the central task server supports capacity expansion and can be used as a shared resource to access different business systems.

[0076] Based on the same inventive concept, the embodiment 400 of the present application further provides a data processing device, wherein: Figure 4 As shown, the device comprises:

[0077] An acquisition module 31 is used to acquire a task set submitted by at least one thread pool and a task identifier corresponding to the task set, wherein the task set includes at least one task to be processed;

[0078] A storage module 32, used for storing a mapping relationship between a task set and a task identifier corresponding to the task set;

[0079] A screening module 33 is used for screening out at least one to-be-processed task in a task set matching a task identifier of the thread pool based on a mapping relationship in response to the thread pool creating a task thread;

[0080] The cache module 34 is used for the thread pool to obtain at least one corresponding task to be processed, and cache the obtained task to be processed into a blocking work queue corresponding to the thread pool, waiting to be executed by the corresponding task thread.

[0081] In this embodiment, the specific functions and interaction methods of the acquisition module 31, the storage module 32, the screening module 33 and the cache module 34 can be found in Figure 1 The description of the corresponding embodiments will not be repeated here.

[0082] Optionally, the acquisition module 31 is further used for:

[0083] The task data contained in the acquired task to be processed is serialized and converted into a byte sequence corresponding to the task to be processed.

[0084] like Figure 5 As shown, another embodiment 500 of the present application further provides a terminal device, including a processor 501, wherein the processor 501 is used to execute the steps of the above-mentioned data processing method. Figure 5 It can also be seen that the terminal device provided in the above embodiment further includes a non-transitory computer-readable storage medium 502, on which a computer program is stored, and the computer program executes the steps of the above data processing method when it is run by the processor 501. In practical applications, the terminal device can be one or more computers, as long as it includes the above computer-readable medium and processor.

[0085] Specifically, the storage medium can be a general storage medium, such as a mobile disk, a hard disk, and FLASH, etc. When the computer program on the storage medium is run, each step in the above-mentioned data processing method can be executed. In practical applications, the computer-readable medium can be included in the device / apparatus / system described in the above embodiments, or it can exist independently without being assembled into the device / apparatus / system. The above-mentioned computer-readable storage medium carries one or more programs, and when the above-mentioned one or more programs are executed, each step in the above-mentioned data processing method can be executed.

[0086] According to the embodiments disclosed in the present application, the computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, it may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof, but it is not used to limit the scope of protection of the present application. In the embodiments disclosed in the present application, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, apparatus, or device.

[0087] The flow chart and block diagram in the accompanying drawings of the present application show the possible architecture, function and operation of the system, method and computer program product according to the various embodiments disclosed in the present application. In this regard, each box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the above-mentioned module, program segment or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the box can also occur in the order marked in different drawings. For example, the boxes represented by two connections can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0088] Those skilled in the art will appreciate that the features described in the various embodiments and / or claims of the present disclosure may be combined and / or combined in a variety of ways, even if such combinations or combinations are not explicitly described in the present application. In particular, without departing from the spirit and teachings of the present application, the features described in the various embodiments and / or claims of the present application may be combined and / or combined in a variety of ways, and all of these combinations and / or combinations fall within the scope disclosed in the present application.

[0089] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present application, which are used to illustrate the technical solutions of the present application, rather than to limit them. The protection scope of the present application is not limited thereto. Although the present application is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still change the technical solutions recorded in the above-described embodiments within the technical scope disclosed in the present application, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these changes, modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.< / runnable> < / runnable>

Claims

1. A method for data processing, characterized in that: include: Obtaining a task set submitted by at least one thread pool and a task identifier corresponding to the task set, wherein the task set includes at least one task to be processed; Saving the mapping relationship between the task set and the task identifiers corresponding to the task set; In response to the thread pool creating a task thread, at least one of the to-be-processed tasks in the task set that matches the task identifier of the thread pool is screened out based on the mapping relationship; The thread pool obtains at least one corresponding task to be processed, and caches the obtained task to be processed in a blocking work queue corresponding to the thread pool, waiting to be executed by the corresponding task thread.

2. The method according to claim 1, characterized in that: The step of obtaining a task set submitted by at least one thread pool and a task identifier corresponding to the task set comprises: The acquired task data contained in the task to be processed is serialized and converted into a byte sequence corresponding to the task to be processed.

3. The method according to claim 1, characterized in that The step of saving the mapping relationship between the task set and the task identifiers corresponding to the task set comprises: When saving the pending tasks, counting the storage time of the pending tasks; In response to the saving time length exceeding the preset time threshold corresponding to the task set to which the to-be-processed task belongs, the operation of saving the to-be-processed task is stopped.

4. The method according to claim 1, characterized in that: The step of filtering out at least one of the to-be-processed tasks in the task set that matches the task identifier of the thread pool based on the mapping relationship comprises: In the task set that conforms to the mapping relationship with the task identifier sent by the received thread pool, if the task to be processed is not obtained, the operation of waiting to obtain the task to be processed is executed until the task thread of the thread pool corresponding to the task set is interrupted, and / or the task set is empty for more than a preset time, and the operation of obtaining the task to be processed in the task set is stopped.

5. The method according to claim 2, characterized in that: The step of the thread pool acquiring at least one corresponding task to be processed comprises: The byte sequence contained in the acquired task to be processed is deserialized into a task data object.

6. The method according to claim 1, characterized in that The method further comprises: The number of the saved task sets and the number of the to-be-processed tasks contained in the task sets, as well as the number of the thread pools and the number of the task threads are queried, and the number of application servers where the thread pools are located is adjusted.

7. A data processing device, characterized in that: include: An acquisition module, used to acquire a task set submitted by at least one thread pool and a task identifier corresponding to the task set, wherein the task set includes at least one task to be processed; A saving module, used for saving the mapping relationship between the task set and the task identifiers corresponding to the task set; A screening module, configured to, in response to the thread pool creating a task thread, screen out at least one of the to-be-processed tasks in the task set that matches the task identifier of the thread pool based on the mapping relationship; The cache module is used for the thread pool to obtain at least one corresponding task to be processed, and cache the obtained task to be processed in the blocking work queue corresponding to the thread pool, waiting to be executed by the corresponding task thread.

8. The device according to claim 7, characterized in that The acquisition module is further used for: The acquired task data contained in the task to be processed is serialized and converted into a byte sequence corresponding to the task to be processed.

9. A non-transitory computer-readable storage medium, characterized in that The non-transitory computer-readable storage medium stores instructions, which, when executed by a processor, cause the processor to perform each step in the data processing method according to any one of claims 1 to 6.

10. A terminal device, characterized in that: The method comprises a processor, wherein the processor is used to execute each step in the data processing method according to any one of claims 1 to 6.

Images