Thread pool dynamic creation method, device, equipment and storage medium

Abstract

The present invention discloses a method, device, equipment and storage medium for dynamically creating a thread pool. The method includes: receiving a task to be processed; allocating the task to be processed to an initial thread pool for processing and recording the total time consumed for processing the task to be processed. The initial thread pool includes an already created initial thread pool or a newly created initial thread pool; when a preset condition is satisfied, generating new thread pool parameters according to the current core thread number, total time consumed and preset expected time consumed of the initial thread pool; creating a new thread pool according to the new thread pool parameters and allocating subsequent tasks to the new thread pool for processing. By processing the task using the initial thread pool after receiving the task, recording the total time consumed for processing, and generating a new thread pool in combination with the preset expected time consumed and the core thread number of the initial thread pool, the present invention can process tasks more efficiently and optimize the utilization of system resources.

Description

Technical Field

[0001] This application relates to the field of computer technologies, and particularly to a method, apparatus, device, and storage medium for dynamically creating a thread pool. Background Art

[0002] Computer programs execute related tasks and process data through threads. When dealing with concurrent tasks or batch data, the processing efficiency of a single-threaded solution is very low, and generally a multi-threaded solution is chosen. Currently, when using multi-threads in Java development, a thread pool is usually used. ThreadPoolExecutor is a commonly used thread pool. In program development, the thread pool and its configurable parameters are set. After the program development is completed, relevant parameters can be set before the application starts.

[0003] When a task is submitted to the thread pool, first, it is judged whether the core thread pool is full. If it is not full, a working thread is created / assigned to process the task. If it is full, then it is judged whether the task blocking queue is full. If it is not full, the newly submitted task is stored in the queue according to the first-in-first-out principle and waits to be executed. If it is full, then it is judged whether the entire thread pool is full. If it is not full, a working thread is created / assigned to process the task. If it is full, the task is processed according to the rejection policy. Therefore, the most core part of using a thread pool is the parameter configuration of the thread pool. If the parameter configuration of the thread pool is unreasonable, it will cause failures and bring unpredictable problems to the system. Currently, the parameter settings of the thread pool are all one-time. After the application starts, the configuration parameters of the thread pool cannot be dynamically set. For different types of tasks, the most reasonable thread pool parameter configuration is different. For example, when the business changes from CPU-intensive to IO-intensive, too many threads will cause the consumption of thread switching to continue to increase, thereby affecting the response time of the business. On the one hand, it reduces the task processing efficiency of the system, and on the other hand, the utilization of system resources is not reasonable enough. Summary of the Invention

[0004] This application provides a method, apparatus, device, and storage medium for dynamically creating a thread pool to solve the problem that the existing thread pool parameter configuration is too rigid, resulting in unreasonable utilization of system resources and low task processing efficiency.

[0005] To solve the above technical problems, a technical solution adopted in this application is: to provide a method for dynamically creating a thread pool, including: receiving a task to be processed; allocating the task to be processed to an initial thread pool for processing, and recording the total time taken to process the task to be processed. The initial thread pool includes an already created initial thread pool or a newly created initial thread pool; when a preset condition is met, generate new thread pool parameters based on the current core thread count, total time taken, and preset expected time taken of the initial thread pool; create a new thread pool according to the new thread pool parameters, and allocate subsequent tasks to the new thread pool for processing.

[0006] As a further improvement of this application, when a preset condition is met, generating new thread pool parameters based on the current core thread count, total time taken, and preset expected time taken of the initial thread pool includes: determining whether the task to be processed is a CPU-intensive task or an IO-intensive task; when the task to be processed is an IO-intensive task, obtain the IO read time and CPU calculation time when the task to be processed is being processed according to the total time taken; when the total time taken is greater than the preset expected time taken, confirm the magnitude relationship between the IO read time and the CPU calculation time; when the IO read time is greater than or equal to the CPU calculation time, calculate the first core thread count according to the current core thread count and the first preset rule, and the first core thread count is more than the current core thread count; when the IO read time is less than the CPU calculation time, calculate the second core thread count according to the current core thread count and the second preset rule, and the second core thread count is less than the current core thread count; based on the first core thread count or the second core thread count, generate new thread pool parameters in combination with the previously obtained user configuration parameters. The new thread pool parameters include the first core thread count or the second core thread count, the maximum thread count, the thread idle time, and the task blocking queue length.

[0007] As a further improvement of this application, the first preset rule includes: the first core thread count = (current core thread count + 1) * 125 / 100; the second preset rule includes: the second core thread count = current core thread count * 75 / 100 + 1.

[0008] As a further improvement of this application, allocating the task to be processed to the initial thread pool for processing and recording the total time taken to process the task to be processed includes: confirming the task type corresponding to the task to be processed, and the task type includes CPU-intensive and IO-intensive; determining whether there is an initial thread pool corresponding to the task type; if so, allocate the task to be processed to the corresponding initial thread pool for processing and record the total time taken; if not, obtain the system performance parameters, create an initial thread pool according to the system performance parameters and the task type, and allocate the task to be processed to the initial thread pool for processing and record the total time taken.

[0009] As a further improvement of the present application, an initial thread pool is created according to system performance parameters and task types, including: obtaining the number of CPU cores of the system; setting the number of initial thread cores according to the task type and the number of CPU cores. When the task type is CPU-intensive, the number of initial thread cores is the same as the number of CPU cores. When the task type is I / O-intensive, the number of initial thread cores is twice the number of CPU cores; setting the maximum number of threads, thread idle time, and task blocking queue length according to the user configuration parameters obtained in advance; creating an initial thread pool according to the number of initial thread cores, the maximum number of threads, thread idle time, and task blocking queue length.

[0010] As a further improvement of the present application, before confirming the task type corresponding to the task to be processed, it further includes: obtaining the task type tags pre-assigned by the user to each application in the system, and the task type tags represent the task types corresponding to the tasks generated by each application.

[0011] As a further improvement of the present application, after allocating subsequent tasks to a new thread pool for processing, it further includes: when the initial thread pool and the new thread pool are both used for tasks of the same task type, after each thread in the initial thread pool finishes processing the task being executed, deleting the initial thread pool.

[0012] To solve the above technical problems, another technical solution adopted by the present application is: providing a thread pool dynamic creation device, including: a receiving module for receiving tasks to be processed; a recording module for allocating the tasks to be processed to an initial thread pool for processing and recording the total time taken to process the tasks to be processed, where the initial thread pool includes an already created initial thread pool or a newly created initial thread pool; a generating module for generating new thread pool parameters according to the current core thread number, total time taken, and preset expected time taken of the initial thread pool when a preset condition is met; a creating module for creating a new thread pool according to the new thread pool parameters and allocating subsequent tasks to the new thread pool for processing.

[0013] To solve the above technical problems, another technical solution adopted by the present application is: providing a computer device, the computer device includes a processor and a memory coupled to the processor, and program instructions are stored in the memory. When the program instructions are executed by the processor, the processor is caused to execute the steps of the thread pool dynamic creation method as described in any one of the above.

[0014] To solve the above technical problems, another technical solution adopted by the present application is: providing a storage medium storing program instructions capable of implementing the above thread pool dynamic creation method.

[0015] The beneficial effects of the present application are as follows: The method for dynamically creating a thread pool in the present application processes tasks by using an initial thread pool, records the total time taken to process the tasks, generates new thread pool parameters by combining the core thread number, total time taken, and preset expected time taken of the initial thread pool, and finally creates a new thread pool according to the new thread pool parameters and uses the new thread pool to process subsequent tasks. When a certain business generates a large number of tasks of the same type, the thread pool parameters are adjusted in a timely manner according to the situation of the first execution of tasks of this type, and a new thread pool is constructed, so that the new thread pool can execute all tasks generated by this business more reasonably. It reasonably sets the thread pool parameters in combination with the actual situation of the tasks, greatly improving the rationality of system resource utilization and the efficiency of task execution. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 is a schematic flowchart of the method for dynamically creating a thread pool according to an embodiment of the present invention;

[0017] Figure 2 is a schematic flowchart of step S102 according to an embodiment of the present invention;

[0018] Figure 3 is a schematic flowchart of step S103 according to an embodiment of the present invention;

[0019] Figure 4 is a schematic diagram of the functional modules of the device for dynamically creating a thread pool according to an embodiment of the present invention;

[0020] Figure 5 is a schematic diagram of the structure of a computer device according to an embodiment of the present invention;

[0021] Figure 6 is a schematic diagram of the structure of a storage medium according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0022] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0023] The terms "first", "second", and "third" in this application are only used for descriptive purposes and should not be construed as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, features defined with "first", "second", and "third" may explicitly or implicitly include at least one of such features. In the description of this application, the meaning of "a plurality" is at least two, such as two, three, etc., unless otherwise specifically and clearly defined. In the embodiments of this application, all directional indications (such as up, down, left, right, front, back...) are only used to explain the relative positional relationship and movement conditions between components in a specific posture (as shown in the attached drawings). If the specific posture changes, then the directional indication will also change accordingly. In addition, the terms "comprise" and "have" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the listed steps or units, but optionally further includes steps or units not listed, or optionally further includes other steps or units inherent to these processes, methods, products, or devices.

[0024] Referring to "embodiments" herein means that the specific features, structures, or characteristics described in connection with the embodiments can be included in at least one embodiment of this application. The appearance of this phrase at various positions in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment mutually exclusive with other embodiments. Those skilled in the art will explicitly and implicitly understand that the embodiments described herein can be combined with other embodiments.

[0025] Figure 1 is a schematic flowchart of the method for dynamically creating a thread pool in an embodiment of the present invention. It should be noted that if there are substantially the same results, the method of the present invention is not limited to Figure 1 the process sequence shown. As Figure 1 shown, the method includes the steps:

[0026] Step S101: Receive the task to be processed.

[0027] Specifically, the task to be processed in this embodiment refers to the task generated when the system runs a certain service. The system runs on a computer terminal, and the service can be an application installed on the computer terminal. When the application runs, a large number of tasks will be continuously generated and sent to the message queue of the computer terminal, waiting for the CPU to process these tasks.

[0028] It should be understood that the task to be processed in this embodiment refers to the task generated during the execution of the service. These tasks have obvious aggregation, that is, the tasks to be processed within a certain period of time are generated by the same service. Therefore, such tasks have strong aggregation in terms of task type and are very likely to be of the same type.

[0029] Step S102: Assign the task to be processed to the initial thread pool for processing, and record the total time consumed for processing the task to be processed. The initial thread pool includes the already created initial thread pool or the newly created initial thread pool.

[0030] Specifically, after receiving the task to be processed, assign the task to be processed to the initial thread pool for processing. It should be understood that in this embodiment, the initial number of threads can be understood as the initial thread pool created by default when the system starts, or the thread pool that executed the previous task. When there is an already created initial thread pool in the system, directly assign the task to be processed to this initial thread pool for processing. When there is no thread pool in the system, create a new initial thread pool and assign the task to be processed to this initial thread pool for processing. When using the initial thread pool to process the task to be processed, record the total time consumed for processing this task to be processed for subsequent adjustment of the thread pool parameters.

[0031] It should be understood that in this embodiment, in order to ensure the reliability of the data, when receiving the task to be processed, put the task to be processed into the message queue. When recording the total time consumed for the task to be processed, each time use the average time consumed of the first three tasks to be processed in this message queue as the total time consumed. For example, assume that there are currently five tasks A, B, C, D, and E arranged in sequence in the message queue. After obtaining the completion of tasks A, B, and C, use the average time consumed of tasks A, B, and C as the first total time consumed, then the average time consumed of tasks B, C, and D is the second total time consumed, and the average time consumed of tasks C, D, and E is the third total time consumed. It confirms the total time consumed by using the average method, thereby improving the credibility of the total time consumed and avoiding frequent adjustment of the thread pool parameters due to the existence of special tasks.

[0032] Furthermore, in order to improve the processing efficiency of tasks, in this embodiment, different types of tasks are processed using thread pools with different parameters. Therefore, as Figure 2 shown, this step S102 specifically includes:

[0033] Step S201: Confirm the task type corresponding to the task to be processed. The task types include CPU-intensive and IO-intensive.

[0034] It should be noted that in this embodiment, the tasks to be processed are divided into two task types: CPU-intensive and IO-intensive. Among them, CPU-intensive, also known as compute-intensive, means that the performance of the server's hard disk and memory hardware is much better than that of the CPU, or the utilization rate is much lower. When the system runs and the CPU reads and writes I / O (hard disk / memory), it can be completed in a very short time, with almost no blocking time (waiting time for I / O), while the CPU always has a large amount of operations to process. Therefore, the CPU load is too high for a long time. For example, in scenarios such as image processing and training algorithm models, a large number of CPU-intensive tasks will be generated. The IO-intensive type is exactly the opposite of the CPU-intensive type. Generally, it means that the performance of the server CPU is much better than that of the hard disk and memory hardware, or the utilization rate is much lower. The system mostly runs with the CPU waiting for the read and write operations of I / O (hard disk / memory). In such scenarios, the CPU load is not high. For example, in scenarios such as file reading and writing, DB reading and writing, and network requests, a large number of IO-intensive tasks will be generated.

[0035] Specifically, based on the different task types, when using a thread pool to process these tasks, for the purpose of making full use of system resources, thread pools with different parameters need to be used to process tasks of different task types. Therefore, first, it is necessary to confirm the task type of the task to be processed to facilitate the subsequent selection of an appropriate thread pool.

[0036] Furthermore, in order to ensure the accurate identification of the type of the task to be processed, before confirming the task type corresponding to the task to be processed, it further includes:

[0037] Obtain the task type labels pre-assigned by the user for each application in the system. The task type labels represent the task types corresponding to the tasks generated by each application.

[0038] Specifically, when the user runs a relevant application, according to the actual situation of the application, pre-assign the corresponding task type label to the application. For an application that will generate a large number of CPU-intensive tasks, assign a CPU-intensive label. Thus, when the application generates a task to be processed, the generated task to be processed will carry this CPU-intensive label. For an application that will generate IO-intensive tasks, assign an IO-intensive label.

[0039] Step S202: Determine whether there is an initial thread pool corresponding to the task type. If so, execute step S203; if not, execute step S204.

[0040] Specifically, after confirming the task type of the task to be processed, it is further confirmed whether there is a thread pool suitable for processing this task type. For example, CPU-intensive tasks are suitable to be processed using a thread pool with the same number of core threads as the number of CPU cores, while IO-intensive tasks are suitable to be processed using a thread pool with twice the number of core threads as the number of CPU cores, so as to achieve the purpose of making full use of system resources. Further, in order to achieve the correspondence between the task type and the thread pool, when creating the initial thread pool, the initial thread pool is tagged according to the number of core threads when creating the initial thread pool. For example, if the number of core threads of the initial thread pool is the same as the number of CPU cores, the initial thread pool is tagged as CPU-intensive; when the number of core threads of the initial thread pool is twice the number of CPU cores, the initial thread pool is tagged as IO-intensive. Thus, when confirming whether there is an initial thread pool corresponding to the task type, it only needs to match the task type with the tag of the initial thread pool to know.

[0041] Step S203: Allocate the task to be processed to the corresponding initial thread pool for processing, and record the total time consumption.

[0042] Specifically, when there is an initial thread pool corresponding to the task type of the task to be processed, the task to be processed is allocated to the corresponding initial thread pool for processing, and then the total time consumption when the task to be processed is processed is recorded.

[0043] Step S204: Obtain the system performance parameters, create an initial thread pool according to the system performance parameters and the task type, allocate the task to be processed to the initial thread pool for processing, and record the total time consumption.

[0044] Specifically, when there is no initial thread pool corresponding to the task type of the task to be processed, an initial thread pool corresponding to it is created according to the system performance parameters in combination with the task type of the task to be processed, and then the created initial thread pool is used to process the task to be processed, and the total time consumption is recorded.

[0045] Further, in this embodiment, creating an initial thread pool according to the system performance parameters and the task type includes:

[0046] 1. Obtain the number of CPU cores of the system.

[0047] Among them, the CPU, i.e., the central processing unit, is one of the most important and core hardware components in a computer. The main job of the CPU is to process and calculate data. Therefore, the computing speed, main frequency, cache, and number of cores of the CPU determine the quality of the CPU. The higher the main frequency, the larger the cache, and the more cores the CPU has, the faster the CPU operates. The number of CPU cores refers to the number of CPU cores, indicating how many cores a CPU is composed of, and it is the most important part of the CPU. Common numbers of CPU cores include dual-core, quad-core, six-core, eight-core, twelve-core, etc. Under the condition that the core frequency, cache size, etc. are the same, the more CPU cores there are, the stronger the overall performance of the CPU. For example, a 6-core CPU with a frequency of 3.8GHz has stronger performance than a dual-core CPU with a frequency of 3.8GHz.

[0048] 2. Set the initial thread core number according to the task type and the number of CPU cores. When the task type is CPU-intensive, the initial thread core number is the same as the number of CPU cores. When the task type is IO-intensive, the initial thread core number is twice the number of CPU cores.

[0049] Specifically, when the task type is CPU-intensive, it needs to use the CPU for a large amount of computing and processing. Therefore, the CPU occupancy is relatively high. To ensure that the CPU is fully utilized without obvious blocking of the thread pool, the initial core thread number of the initial thread pool can be set to be the same as the number of CPU cores, which can not only ensure that all CPU cores are utilized but also prevent the task queue of the thread pool from being blocked for too long. When the task type is IO-intensive, to improve the utilization rate of the CPU cores, the initial thread core number of the initial thread pool is set to twice the number of CPU cores to improve the utilization rate of the CPU cores and avoid the situation where the CPU cores need to wait for the completion of IO operations after finishing all task calculations.

[0050] 3. Set the maximum number of threads, thread idle time, and task blocking queue length according to the pre-obtained user configuration parameters.

[0051] It should be understood that a thread pool usually includes seven core parameters, namely: core thread number, maximum thread number, thread idle time, unit of idle thread retention time, task blocking queue length, thread factory, and rejection policy. Among them, the thread factory and rejection policy are set by default, and the core thread number, maximum thread number, thread idle time, unit of idle thread retention time, and task blocking queue length are set in advance. In this embodiment, the core thread number is set according to the task type and the number of CPU cores, the maximum thread number, thread idle time, and task blocking queue length are set according to the configuration parameters preset by the user, and the unit of idle thread retention time corresponding to the thread idle time also needs to be set according to the preset configuration parameters.

[0052] 4. Create an initial thread pool based on the initial number of thread cores, the maximum number of threads, the thread idle time, and the length of the task blocking queue.

[0053] Specifically, after obtaining the parameters for creating the thread pool, create the initial thread pool according to the parameters.

[0054] Step S103: When the preset conditions are met, generate new thread pool parameters based on the current core thread number, the total elapsed time, and the preset expected elapsed time of the initial thread pool.

[0055] Specifically, after obtaining the total elapsed time, based on the preset conditions, confirm whether the initial thread pool is suitable for processing the current type of task by combining the relationship between the total elapsed time and the preset expected elapsed time. If not, it means that the parameters of the thread pool need to be adjusted to create a thread pool that is more suitable for processing the current task. Thus, generate new thread pool parameters by combining the current core thread number, the total elapsed time, and the preset expected elapsed time of the initial thread pool.

[0056] Further, please refer to Figure 3 , and this step S103 specifically includes:

[0057] Step S301: Determine whether the task to be processed is a CPU-intensive task or an IO-intensive task.

[0058] Specifically, the task type of the task to be processed is determined based on the label pre-assigned to the application.

[0059] Step S302: When the task to be processed is an IO-intensive task, obtain the IO read time and the CPU calculation time when the task to be processed is being processed according to the total elapsed time.

[0060] It should be understood that when the task to be processed is being processed, it usually includes two stages, one is the calculation stage when the CPU is processing, and the other is the IO operation stage. Therefore, when the task to be processed is being processed, its total elapsed time includes the CPU calculation time and the IO read time.

[0061] Step S303: When the total elapsed time is greater than the preset expected elapsed time, confirm the magnitude relationship between the IO read time and the CPU calculation time.

[0062] Specifically, for an IO-intensive task, when the total elapsed time is greater than the preset expected elapsed time, it means that the processing time of this task is too long and the utilization of system resources is unreasonable, and the parameters of the thread pool need to be adjusted; when the total elapsed time is less than or equal to the preset expected elapsed time, it means that the parameter configuration of the thread pool is reasonable at this time and no adjustment is required. And when the parameters of the thread pool need to be adjusted, the corresponding adjustment method can be selected according to the magnitude relationship between the IO read time and the CPU calculation time.

[0063] Step S304: When the IO reading time is greater than or equal to the CPU computing time, calculate the first number of core threads according to the current number of core threads and the first preset rule, where the first number of core threads is more than the current number of core threads.

[0064] Specifically, when the IO reading time is greater than or equal to the CPU computing time, it indicates that the CPU has completed the computing operation stage of a task in advance and is waiting for the completion of the IO operation stage. At this time, in order to improve the utilization rate of the CPU, the number of core threads can be increased. Therefore, calculate the first number of core threads greater than the current number of core threads by combining the current number of core threads in the initial thread pool and the first preset rule.

[0065] Furthermore, the first preset rule includes:

[0066] The first number of core threads = (the current number of core threads + 1) * 125 / 100.

[0067] Step S305: When the IO reading time is less than the CPU computing time, calculate the second number of core threads according to the current number of core threads and the second preset rule, where the second number of core threads is less than the current number of core threads.

[0068] Specifically, when the IO reading time is less than the CPU computing time, it indicates that there are more context switches at this time and the performance loss is greater. Then, the number of core threads needs to be reduced to reduce the performance loss. Therefore, calculate the second number of core threads less than the current number of core threads by the current number of core threads in the initial thread pool and the second preset rule.

[0069] Furthermore, the second preset rule includes:

[0070] The second number of core threads = the current number of core threads * 75 / 100 + 1.

[0071] It should be noted that context switching (sometimes also called process switching or task switching) refers to the CPU switching from one process or thread to another. A process (sometimes also called a task) refers to an instance of a program running. In the Linux system, a thread is a lightweight process that can run in parallel and share the same address space (a memory area) and other resources with its parent process (the process that created it). Context refers to the content of the CPU registers and the program counter at a certain point in time. Registers are a small number of but fast memories inside the CPU (corresponding to the relatively slower main memory RAM outside the CPU). Registers improve the running speed of computer programs by quickly accessing frequently used values (usually intermediate values of operations). The program counter is a special register used to indicate the position in the instruction sequence that the CPU is executing, and the stored value is the position of the instruction being executed or the position of the next instruction to be executed, depending on the specific system. Context switching can be considered as the kernel (the core of the operating system) performing the following activities on the CPU for processes (including threads): 1. Suspend a process and store the state (context) of this process in the CPU in a certain place in memory. 2. Retrieve the context of the next process in memory and restore it in the CPU registers. 3. The position pointed to by the counter (i.e., jump to the code line where the process was interrupted) to resume the process. Context switching is usually computationally intensive. That is to say, it requires a considerable amount of processor time. In dozens or hundreds of switches per second, each switch requires nanosecond-level time. Therefore, context switching means consuming a large amount of CPU time for the system. In fact, it may be the operation that consumes the most time in the operating system. Therefore, when there are more computationally intensive operations, the number of core threads should be reduced to reduce performance loss.

[0072] Step S306: Based on the first core thread number or the second core thread number, combine with the user configuration parameters obtained in advance to generate new thread pool parameters. The new thread pool parameters include the first core thread number or the second core thread number, the maximum thread number, the thread idle time, and the task blocking queue length.

[0073] Specifically, when the IO read time is greater than or equal to the CPU calculation time, calculate the first core thread number, and then combine with the maximum thread number, the thread idle time, and the task blocking queue length to generate new thread pool parameters to create a new thread pool. When the IO read time is less than the CPU calculation time, calculate the second core thread number, and then combine with the maximum thread number, the thread idle time, and the task blocking queue length to generate new thread pool parameters to create a new thread pool.

[0074] It should be noted that in this embodiment, when the task to be processed is a CPU-intensive task, the number of core threads is the number of CPU cores of the server, and the performance loss caused by context switching of the task is low. Therefore, it is not necessary to adjust the parameters of the initial thread pool.

[0075] Step S104: Create a new thread pool according to the new thread pool parameters, and allocate subsequent tasks to the new thread pool for processing.

[0076] Specifically, after obtaining the new thread pool parameters, create a new thread pool, and allocate all subsequent tasks to the new thread pool for processing.

[0077] It should be understood that while allocating subsequent tasks to the new thread pool for processing, it is still necessary to continue comparing and judging the total duration of the task and the preset expected duration, and adjust the parameters of the thread pool when the preset conditions are met again to ensure that the performance of the thread pool is always optimal and improve the utilization rate of system resources.

[0078] Further, after allocating subsequent tasks to the new thread pool for processing, it further includes:

[0079] When both the initial thread pool and the new thread pool are used for tasks of the same task type, after each thread in the initial thread pool finishes processing the task being executed, delete the initial thread pool.

[0080] Specifically, after generating new thread pool parameters and creating a new thread pool according to the new thread pool parameters, use the new thread pool to process subsequent tasks, and then wait for the initial thread pool to finish processing the tasks that are already being executed, and then delete the initial thread pool, so as to avoid occupying too many resources due to creating too many thread pools.

[0081] The thread pool dynamic creation method of the embodiment of the present invention processes tasks by using the initial thread pool, records the total duration of processing the tasks, then generates new thread pool parameters in combination with the number of core threads, total duration and preset expected duration of the initial thread pool, and finally creates a new thread pool according to the new thread pool parameters and uses the new thread pool to process subsequent tasks. When a certain business generates a large number of tasks of the same type, adjust the thread pool parameters in a timely manner according to the situation of initially executing tasks of this type and construct a new thread pool, so that the new thread pool can execute all tasks generated by this business more reasonably. It reasonably sets the thread pool parameters in combination with the actual situation of the tasks, greatly improving the rationality of system resource utilization and the efficiency of task execution.

[0082] Figure 4 It is a schematic diagram of the functional modules of the thread pool dynamic creation device of the embodiment of the present invention. As Figure 4As shown in the figure, the thread pool dynamic creation device 40 includes a receiving module 41, a recording module 42, a generating module 43, and a creating module 44.

[0083] The receiving module 41 is configured to receive tasks to be processed;

[0084] The recording module 42 is configured to allocate the tasks to be processed to an initial thread pool for processing and record the total time consumed for processing the tasks to be processed. The initial thread pool includes an already created initial thread pool or a newly created initial thread pool;

[0085] The generating module 43 is configured to generate new thread pool parameters according to the current core thread number, the total time consumed, and a preset expected time consumed of the initial thread pool when a preset condition is satisfied;

[0086] The creating module 44 is configured to create a new thread pool according to the new thread pool parameters and allocate subsequent tasks to the new thread pool for processing.

[0087] Optionally, when the generating module 43 executes the operation of generating new thread pool parameters according to the current core thread number, the total time consumed, and the preset expected time consumed of the initial thread pool when the preset condition is satisfied, it specifically includes: determining whether the task to be processed is a CPU-intensive task or an IO-intensive task; when the task to be processed is an IO-intensive task, obtaining the IO reading time and the CPU calculation time when the task to be processed is being processed according to the total time consumed; when the total time consumed is greater than the preset expected time consumed, confirming the magnitude relationship between the IO reading time and the CPU calculation time; when the IO reading time is greater than or equal to the CPU calculation time, calculating a first core thread number according to the current core thread number and a first preset rule, where the first core thread number is more than the current core thread number; when the IO reading time is less than the CPU calculation time, calculating a second core thread number according to the current core thread number and a second preset rule, where the second core thread number is less than the current core thread number; based on the first core thread number or the second core thread number, generating new thread pool parameters in combination with the previously obtained user configuration parameters. The new thread pool parameters include the first core thread number or the second core thread number, the maximum thread number, the thread idle time, and the task blocking queue length.

[0088] Optionally, the first preset rule includes: the first core thread number = (the current core thread number + 1) * 125 / 100; the second preset rule includes: the second core thread number = the current core thread number * 75 / 100 + 1.

[0089] Optionally, the recording module 42 performs operations of allocating tasks to be processed to the initial thread pool for processing and recording the total time consumed for processing the tasks to be processed, specifically including: confirming the task type corresponding to the task to be processed, where the task type includes CPU-intensive and IO-intensive; determining whether there is an initial thread pool corresponding to the task type; if there is, allocating the task to be processed to the corresponding initial thread pool for processing and recording the total time consumed; if not, obtaining system performance parameters, creating an initial thread pool according to the system performance parameters and the task type, and allocating the task to be processed to the initial thread pool for processing and recording the total time consumed.

[0090] Optionally, the recording module 42 performs operations of creating an initial thread pool according to the system performance parameters and the task type, specifically including: obtaining the number of CPU cores of the system; setting the number of initial thread cores according to the task type and the number of CPU cores, where when the task type is CPU-intensive, the number of initial thread cores is the same as the number of CPU cores, and when the task type is IO-intensive, the number of initial thread cores is twice the number of CPU cores; setting the maximum number of threads, the thread idle time, and the length of the task blocking queue according to the pre-obtained user configuration parameters; creating an initial thread pool according to the number of initial thread cores, the maximum number of threads, the thread idle time, and the length of the task blocking queue.

[0091] Optionally, before the recording module 42 performs operations of confirming the task type corresponding to the task to be processed, it is also used for: obtaining the task type tags pre-assigned by the user to each application in the system, where the task type tags represent the task types corresponding to the tasks generated by each application.

[0092] Optionally, after the creating module 44 performs operations of allocating subsequent tasks to a new thread pool for processing, it is also used for: when both the initial thread pool and the new thread pool are used for tasks of the same task type, deleting the initial thread pool after each thread in the initial thread pool finishes processing the task being executed.

[0093] Regarding other details of the technical solutions implemented by each module in the thread pool dynamic creation device in the above embodiments, reference can be made to the description in the thread pool dynamic creation method in the above embodiments, which will not be elaborated here.

[0094] It should be noted that each embodiment in this specification is described in a progressive manner. The key point of each embodiment is to illustrate the differences from other embodiments. The same or similar parts among the embodiments can be referred to each other. For the device embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can refer to the partial description of the method embodiments.

[0095] Please refer to Figure 5 , Figure 5 which is the structural schematic diagram of the computer device according to the embodiment of the present invention. AsFigure 5 As shown, the computer device 50 includes a processor 51 and a memory 52 coupled to the processor 51. Program instructions are stored in the memory 52. When the program instructions are executed by the processor 51, the processor 51 is caused to execute the steps of the thread pool dynamic creation method described in any of the above embodiments.

[0096] Among them, the processor 51 can also be referred to as a CPU (Central Processing Unit). The processor 51 may be an integrated circuit chip with signal processing capabilities. The processor 51 can also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc.

[0097] Refer to Figure 6 , Figure 6 is a schematic structural diagram of the storage medium of the embodiment of the present invention. The storage medium of the embodiment of the present invention stores program instructions 61 capable of implementing all of the above methods. Among them, the program instructions 61 can be stored in the above storage medium in the form of a software product, including several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to execute all or part of the steps of the methods described in various embodiments of the present application. The foregoing storage medium includes: various media that can store program codes such as a USB flash drive, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk, or an optical disc, or a computer device such as a computer, a server, a mobile phone, or a tablet.

[0098] In several embodiments provided in the present application, it should be understood that the disclosed computer devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some interfaces, and the indirect couplings or communication connections of the apparatuses or units can be in electrical, mechanical, or other forms.

[0099] In addition, each functional unit in various embodiments of the present invention may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit. The above-mentioned integrated units may be implemented in the form of hardware or in the form of software functional units. The above are only the implementation manners of the present application, and do not limit the patent scope of the present application. Any equivalent structural or equivalent process transformation made by using the content of the specification and drawings of the present application, or directly or indirectly applied in other related technical fields, shall be equally included in the patent protection scope of the present application.

Claims

1. A method for dynamically creating a thread pool, characterized in that, Including: Receiving a task to be processed; Allocating the task to be processed to an initial thread pool for processing, and recording the total time taken to process the task to be processed, where the initial thread pool includes an already created initial thread pool or a newly created initial thread pool; When a preset condition is met, generating new thread pool parameters based on the current core thread count of the initial thread pool, the total time taken, and a preset expected time taken, including: Determining whether the task to be processed is a CPU-intensive task or an I / O-intensive task; When the task to be processed is an I / O-intensive task, obtaining the I / O read time and CPU calculation time when the task to be processed is being processed based on the total time taken; When the total time taken is greater than the preset expected time taken, confirming the magnitude relationship between the I / O read time and the CPU calculation time; When the I / O read time is greater than or equal to the CPU calculation time, calculating a first core thread count according to the current core thread count and a first preset rule, where the first core thread count is more than the current core thread count; When the I / O read time is less than the CPU calculation time, calculating a second core thread count according to the current core thread count and a second preset rule, where the second core thread count is less than the current core thread count; Based on the first core thread count or the second core thread count, generating new thread pool parameters in combination with pre-obtained user configuration parameters, where the new thread pool parameters include the first core thread count or the second core thread count, the maximum thread count, the thread idle time, and the task blocking queue length; Creating a new thread pool according to the new thread pool parameters, and allocating subsequent tasks to the new thread pool for processing.

2. The method for dynamically creating a thread pool according to claim 1, wherein The first preset rule includes: The first core thread count = (the current core thread count + 1) * 125 / 100; The second preset rule includes: The second core thread count = the current core thread count * 75 / 100 + 1.

3. The method for dynamically creating a thread pool according to claim 1, wherein The allocating the task to be processed to an initial thread pool for processing and recording the total time taken to process the task to be processed includes: Confirming the task type corresponding to the task to be processed, where the task type includes CPU-intensive and I / O-intensive; Determining whether there is an initial thread pool corresponding to the task type; If so, allocating the task to be processed to the corresponding initial thread pool for processing, and recording the total time taken; If not, obtaining system performance parameters, creating the initial thread pool according to the system performance parameters and the task type, and allocating the task to be processed to the initial thread pool for processing and recording the total time taken.

4. The method for dynamically creating a thread pool according to claim 3, wherein The creating the initial thread pool according to the system performance parameters and the task type includes: Obtaining the number of CPU cores of the system; Setting the initial thread core count according to the task type and the number of CPU cores. When the task type is CPU-intensive, the initial thread core count is the same as the number of CPU cores. When the task type is I / O-intensive, the initial thread core count is twice the number of CPU cores; Set the maximum number of threads, thread idle time, and task blocking queue length according to the pre-obtained user configuration parameters; Create the initial thread pool according to the initial thread core number, the maximum number of threads, the thread idle time, and the task blocking queue length.

5. The method for dynamically creating a thread pool according to claim 3, wherein Before confirming the task type corresponding to the to-be-processed task, it further includes: Obtain the task type tags pre-assigned by the user to each application in the system, and the task type tags represent the task types corresponding to the tasks generated by each application.

6. The method for dynamically creating a thread pool according to claim 3, wherein After allocating the subsequent tasks to the new thread pool for processing, it further includes: When both the initial thread pool and the new thread pool are used for tasks of the same task type, after each thread in the initial thread pool finishes processing the task being executed, delete the initial thread pool.

7. A thread pool dynamic creation device, characterized in that It includes: A receiving module, configured to receive the to-be-processed task; A recording module, configured to allocate the to-be-processed task to the initial thread pool for processing and record the total time taken to process the to-be-processed task, and the initial thread pool includes the already created initial thread pool or the newly created initial thread pool; A generating module, configured to generate new thread pool parameters according to the current core thread number of the initial thread pool, the total time taken, and the preset expected time taken when a preset condition is satisfied, including: Judge whether the to-be-processed task is a CPU-intensive task or an IO-intensive task; When the to-be-processed task is an IO-intensive task, obtain the IO reading time and CPU calculation time when the to-be-processed task is being processed according to the total time taken; When the total time taken is greater than the preset expected time taken, confirm the magnitude relationship between the IO reading time and the CPU calculation time; When the IO reading time is greater than or equal to the CPU calculation time, calculate the first core thread number according to the current core thread number and the first preset rule, and the first core thread number is more than the current core thread number; When the IO reading time is less than the CPU calculation time, calculate the second core thread number according to the current core thread number and the second preset rule, and the second core thread number is less than the current core thread number; Based on the first core thread number or the second core thread number, generate new thread pool parameters in combination with the pre-obtained user configuration parameters, and the new thread pool parameters include the first core thread number or the second core thread number, the maximum number of threads, the thread idle time, and the task blocking queue length; A creating module, configured to create a new thread pool according to the new thread pool parameters and allocate subsequent tasks to the new thread pool for processing.

8. A computer device, characterized in that, The computer device includes a processor and a memory coupled to the processor, and program instructions are stored in the memory. When the program instructions are executed by the processor, the processor executes the steps of the thread pool dynamic creation method according to any one of claims 1-6.

9. A storage medium, characterized in that, Store program instructions capable of implementing the thread pool dynamic creation method according to any one of claims 1-6.

Images