Database thread pool dynamic capacity expansion and contraction method and device and electronic equipment

Abstract

The invention relates to a dynamic capacity expansion and contraction method and device for a database thread pool. The method comprises the following steps: periodically collecting indexes such as session response and processing delay, request queue length and active thread number of a database system; smoothing processing is carried out on the collection indexes to eliminate the influence of instantaneous data fluctuation; and judging the current load state (overload, normal or idle) of the system according to the indexes, if the current load state is the overload state or the idle state, executing the step-by-step thread pool capacity expansion and contraction operation according to the system resource occupation condition, and returning to form closed-loop feedback regulation after the execution is finished. The method has an adaptive dynamic adjustment capability, and accurately senses the system state through an exponential weighted average algorithm; intelligent decision-making and stepping capacity expansion and contraction strategies are adopted, and efficient adjustment of'changing from quantity to demand 'is realized in combination with resource occupancy detection; and finally, self-adaptive optimization of the database thread pool is realized, the resource utilization rate is improved while stable performance is ensured, the manual intervention cost is reduced, and an efficient technical solution is provided for a high-concurrency scene.

Description

Technical Field

[0001] The present application relates to the technical field of database operation, and in particular to a method, device, computer-readable storage medium, and electronic device for dynamically scaling a database thread pool. Background Art

[0002] A thread, the basic unit of operating system scheduling in a computer system, is the actual execution unit within a process. The operating system executes the control flow within a thread according to a specific scheduling policy. A thread pool is a technical solution for managing and reusing threads, primarily designed to improve the performance and resource utilization of multi-threaded applications in high-concurrency, high-load scenarios.

[0003] Database systems typically need to handle a large number of concurrent queries and updates. In high-concurrency scenarios, thread pool technology significantly improves database performance and throughput. In traditional multi-threaded programming, frequent thread creation and destruction leads to excessive consumption of system resources and increased thread switching overhead. When a database system needs to handle a large number of concurrent tasks, creating a separate thread for each session will increase the operating system's scheduling burden and waste a large amount of memory and CPU resources. Thread pool technology, on the other hand, avoids frequent thread creation and destruction by pre-creating a certain number of threads and assigning database session tasks to these threads for execution. Figure 1 and Figure 2 shown.

[0004] The core advantage of the thread pool lies in "thread reuse." It pre-creates and maintains a certain number of threads, which can be put back into use after completing their tasks, reducing the cost of thread creation and destruction. At the same time, the thread pool can dynamically adjust the number of threads based on the load, allowing the database system to operate efficiently under different loads. In high-concurrency situations, the thread pool prevents excessive exhaustion of system resources by controlling the maximum number of threads; in low-load situations, sufficient resources can be returned to the operating system to prevent preemption of resources from other processes. It should also be pointed out that an excellent scaling solution is crucial for thread pool technology. Inappropriate scaling may result in wasted system resources or an inability to flexibly adapt to business changes.

[0005] In addition, although some new thread pool scaling technologies have been developed, for example, the patent application "A Thread Pool Scaling Method and Apparatus" (CN118467374A) discloses a thread pool scaling method and apparatus that can learn from historical task data and dynamically adjust the thread pool size based on task characteristics to adapt to complex business scenarios. However, this method still has certain drawbacks in its application. For example, it is not suitable for scenarios with high real-time requirements (such as burst traffic scenarios that require millisecond-level response), the prediction model based on historical data may not be able to adapt to sudden load changes, the uncertainty risk of the prediction results is high, the introduction of machine learning components significantly increases the complexity of the system, the model training cost is high, and the model inference process requires the use of computing resources of the thread pool service.

[0006] In summary, further optimizing the scaling technology of the database thread pool and developing a simpler, easier-to-use, and more real-time method for dynamic scaling of the database thread pool are of great significance for improving the resource utilization, task processing efficiency, and stability of the database system in high-concurrency scenarios. Summary of the Invention

[0007] In order to address the above problems, this application proposes a new method for dynamically expanding and shrinking the database thread pool.

[0008] This paper aims to provide a stable and efficient dynamic scaling strategy for thread pool design in common database systems. Its core goal is to prevent frequent scaling operations during occasional peak hours, while avoiding problems such as resource exhaustion and performance loss caused by excessive scaling.

[0009] The present invention primarily meets the following technical requirements: 1) handling burst traffic, automatically expanding the thread pool size in the face of a surge in session requests, and achieving rapid expansion to cope with load peaks; 2) dynamically deciding whether to increase or decrease threads by comprehensively considering multiple performance indicators such as session request queue length, response latency, and CPU utilization; 3) using moving average and exponentially weighted moving average (EWMA) methods to smooth indicators and avoid misjudgments caused by sampling jitter; and 4) ensuring that scheduling decision logic has millisecond-level execution capabilities, thereby enabling sub-second evaluation and adjustment of thread pool size.

[0010] In order to achieve the above objectives, the present invention adopts the following technical strategies: 1. Comprehensive and accurate indicator monitoring By periodically collecting multiple key indicators including session response and processing delays, request queue length and number of active threads, as well as CPU, memory resources and disk I / O resources, we can fully and meticulously grasp the operating status of the database thread pool, providing a rich and comprehensive data foundation for subsequent accurate decision-making, and avoiding decision-making errors caused by missing or inaccurate data.

[0011] 2. Effective smoothing mechanism The collected indicator data is processed using smoothing algorithms such as the exponentially weighted moving average (EWMA), effectively reducing the interference of instantaneous data fluctuations, enabling the indicators to more steadily and accurately reflect the actual operating status of the thread pool, enhancing the stability of the system in the face of data jitter, and avoiding frequent or erroneous scaling operations due to short-term fluctuations.

[0012] 3. Detailed state division and decision logic The thread pool load is meticulously divided into three states: overloaded, normal, and idle. Upper and lower thresholds are set for each metric, and logical conditions are used for judgment. Anti-shake strategies are also introduced to ensure the accuracy of state transitions. Based on this, expansion decisions comprehensively consider multiple resource factors such as CPU utilization and disk I / O throughput, using a step-by-step adjustment with uneven step sizes. This allows for rapid response to load changes when resources are abundant, while avoiding over-scaling when resources are nearing saturation. Scaling decisions, while recycling idle threads, check CPU utilization to prevent over-recycling, ensuring that the thread pool size always matches the actual load, improving resource utilization and system performance.

[0013] 4. Efficient execution strategy When performing scaling operations, the POSIX thread (pthread) interface is used to create and destroy threads. Combined with a thread-safe task queue design and a reasonable thread notification mechanism, this ensures that tasks can be allocated and executed in a timely and efficient manner. At the same time, the resource release mechanism ensures the complete recovery of resources and stable operation of the system when the thread exits, thereby improving the execution efficiency and reliability of the thread pool during dynamic adjustment.

[0014] 5. Complete stability guarantee mechanism Through various measures, including anti-jitter and smoothing, cooldown settings, thread keepalive policies, scaling limits, and resource protection, the thread pool's stability during dynamic scaling has been comprehensively enhanced. For example, the cooldown setting prevents the impact of frequent scaling operations in a short period of time on the system; the thread keepalive policy ensures that extra threads can exit promptly when idle to free up resources, while also enabling rapid response when needed; and resource protection measures prevent system crashes or performance degradation caused by blind scaling when system resources are nearing their limits, providing a strong guarantee for the stable operation of the thread pool.

[0015] Specifically, this application provides the following technical solutions: The first aspect of the present application provides a method for dynamically expanding and contracting a database thread pool, such as Figure 4 As shown, the method includes the following steps: S1. Periodically collect operational indicators of the database system; S2. Smoothing the collected metrics to eliminate the impact of instantaneous data fluctuations and obtain effective metrics that represent the system status. S3 according to the smoothed indicators to determine the current load state of the system, the load state includes an overload state, a normal state and an idle state; S4. If the system is in an overloaded or idle state, the system performs incremental thread pool expansion and contraction based on system resource usage, including: In an overloaded state, the number of worker threads is gradually increased according to the preset step size until the system load is relieved or the maximum size of the thread pool is reached; In idle state, the number of worker threads is gradually reduced according to the preset step size until the system load is balanced or the minimum size of the thread pool is reached; S5. After the scaling operation is complete, return to the step of periodically collecting operating indicators to form a closed-loop feedback adjustment.

[0016] Furthermore, the operating indicators of the database system described in step S1 of the method of the present application include but are not limited to: session response time, processing delay time, request queue length, number of active threads, CPU resource occupancy, memory resource occupancy and disk I / O resource occupancy.

[0017] Furthermore, the smoothing process described in step S2 of the method of the present application adopts an exponentially weighted average (EWMA) algorithm, which performs weighted calculation on the current indicator and the historical indicator through a smoothing coefficient to reduce the interference of instantaneous jumps on decision-making.

[0018] Furthermore, the step S3 of the present method of determining the current load state of the system includes: (1) When judging the system load status, set independent overload thresholds and idle thresholds for each indicator in advance; (2) When the session response delay exceeds the overload threshold or the request queue length exceeds the overload threshold, it is determined to be in an overload state; (3) When the session response delay is lower than the idle threshold and the request queue length is lower than the idle threshold, it is determined to be in an idle state; (4) State transitions must meet the matching conditions for at least three consecutive sampling periods to avoid misjudgments caused by instantaneous fluctuations.

[0019] Furthermore, the step-by-step thread pool expansion and contraction operation in step S4 of the present method further includes: Check the system CPU and disk I / O resource usage. If the resources are close to saturation, appropriately reduce the number of threads added at a time when expanding capacity. Maintain the thread pool's thread count boundaries to ensure that the number of threads after scaling is no less than the minimum size and no more than the maximum size.

[0020] Furthermore, the method of the present application also includes: when performing expansion and contraction operations, the number of threads is adjusted by creating new working threads or recycling idle working threads, and after expansion or before contraction, the task queue is load balanced to ensure the rationality of task allocation.

[0021] Furthermore, the present application method also includes the following stability assurance mechanism: Anti-jitter processing: Filter out short-term abnormal fluctuations through sliding windows or continuous counting, and trigger expansion and contraction operations only when the load trend continues to change; Cooling-off period: After each scaling operation is completed, a cooling-off period of preset duration is set, during which new scaling decisions are prohibited. Scaling limit: Specifies the maximum number of threads that can be added or reduced in a single scaling operation to avoid drastic changes in the number of threads in a short period of time.

[0022] A second aspect of the present application provides a database thread pool dynamic expansion and contraction device, the device comprising: Index collection module, used to periodically collect operating indicators of the database system; The indicator processing module is used to smooth the collected indicators to obtain effective indicators that represent the system status; The state decision module is used to judge the current load state of the system based on the effective indicators and output the decision results of overload, normal or idle; The scaling execution module is used to perform step-by-step thread number adjustment based on resource usage in overloaded or idle states, including thread creation, destruction, and task queue rebalancing; The stability control module is used to implement anti-jitter filtering, cool-down period timing, scaling range limits, and resource usage pre-detection to ensure the stability of scaling operations.

[0023] The device implements the steps of the aforementioned database thread pool dynamic expansion and contraction method during operation.

[0024] A third aspect of the present application provides an electronic device, comprising: a memory and a processor; Memory: used to store computer programs; Processor: used to execute the computer program to implement the steps of the aforementioned database thread pool dynamic expansion and contraction method.

[0025] A fourth aspect of the present application provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the aforementioned method for dynamically expanding and contracting a database thread pool are implemented.

[0026] In summary, compared with existing methods, the database thread pool dynamic expansion and contraction method proposed in this application has the following advantages: (1) Improve system performance and throughput: Through dynamic scaling strategies, it is possible to quickly scale up when session requests surge, fully utilize system resources to meet high concurrency requirements, reduce task queuing time, and improve the response speed and throughput of the database system; when the load decreases, it can scale down in time to avoid wasting thread resources, enabling the system to run more efficiently and improving the overall performance of the database system.

[0027] (2) Optimize resource utilization: Accurate decision-making logic and reasonable expansion and contraction strategies ensure that the thread pool size is adapted to the actual load, avoiding resource occupation caused by excessive expansion and resource shortage caused by excessive contraction, achieving refined management and efficient utilization of database system resources, reducing system operating costs, and improving the return on investment of resources.

[0028] (3) Enhance system stability and reliability: The perfect stability guarantee mechanism effectively reduces the risk of system failure caused by data jitter, frequent expansion and contraction, or insufficient resources, enabling the database system to maintain stable and reliable operation under various complex load conditions, reducing system maintenance workload and downtime, improving system availability and reliability, and providing strong support for the business continuity of the enterprise.

[0029] (4) Reduce the difficulty and cost of operation and maintenance: The method of the present invention realizes the automatic dynamic expansion and contraction management of the thread pool, reduces the manual configuration and adjustment work of the operation and maintenance personnel on the thread pool, and reduces the risk of problems caused by human operation errors. At the same time, by providing detailed indicator monitoring data and decision-making basis, it helps the operation and maintenance personnel to have a deeper understanding of the system operation status, quickly locate and solve problems, and further reduce the difficulty and cost of operation and maintenance of the database system.

[0030] (5) Adaptability to complex and changing business scenarios: Whether facing sudden business traffic peaks or daily load fluctuations, the method of the present invention can respond quickly and accurately, dynamically adjust the thread pool size to meet business needs, improve the database system's adaptability to complex and changing business scenarios, and provide flexible and reliable support for the company's business development.

[0031] Other features and advantages of this application will be described in detail in the following description, or will be understood through the implementation of the relevant technical solutions of this application. The objectives and other advantages of this application can be achieved through the technical features and technical means clearly indicated in the description, claims, and drawings, and obtained through the implementation of these technical contents. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] To more clearly illustrate the technical solution of this application, the following briefly introduces the prior art and the drawings involved in the description of the embodiments of this application. It should be noted that the drawings only illustrate some embodiments of this application. Those skilled in the art can deduce other relevant drawings based on these drawings without engaging in creative work.

[0033] Figure 1 This is a diagram of the macro session access model of the thread pool in a conventional database system.

[0034] Figure 2 This is a diagram of the internal component structure of a thread pool in a conventional database system.

[0035] Figure 3 This is the basic process framework diagram of this application method.

[0036] Figure 4 This is the overall implementation flow chart of the database thread pool dynamic expansion and contraction method for this application.

[0037] Figure 5 This is a diagram of database configuration parameters in an embodiment of the present application.

[0038] Figure 6 This is a distribution diagram of multiple groups of thread pool threads in the database process in an embodiment of the present application.

[0039] Figure 7 This is a structural diagram of the database thread pool dynamic expansion and contraction device of this application.

[0040] Figure 8 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0041] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It should be understood that the described embodiments are only some embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0042] In this document, the term "including" and any variations thereof (such as "including," "comprising," etc.) are open-ended expressions and should be understood as meaning "including but not limited to," meaning that the listed contents are not exhaustive and may include other contents not explicitly mentioned. The term "based on" should be understood as meaning "based at least in part on," meaning that the basis or condition referred to may not be the only factor and may also involve other relevant factors. The term "one embodiment" should be understood as meaning "at least one embodiment," meaning that the described embodiment is not the only possible implementation method and that other similar embodiments may exist.

[0043] In this application, the terms "a" and "a plurality" are used to modify related elements or features in an illustrative, non-restrictive manner. Unless the context clearly indicates otherwise, "a" should be understood as meaning "at least one," and "a plurality" should be understood as meaning "at least two." Those skilled in the art should interpret these terms appropriately based on the semantics and logical relationships of the context to ensure that they encompass the possibility of "one or more."

[0044] Figure 3 The basic process of this application method is shown, including the following steps: Step 1: Metrics collection: The metrics collection thread periodically collects relevant metrics, including session response and processing delays, request queue length, number of active threads, as well as CPU, memory resources, and disk I / O resources.

[0045] Step 2: Indicator smoothing: Smoothing the collected indicators to prevent drastic changes, thereby reducing the impact of instantaneous data fluctuations on decision-making and making the indicators more accurately reflect the system status.

[0046] Step 3: Decision-making: Make a decision based on the smoothed indicators to determine whether expansion or contraction is necessary.

[0047] Step 4: Scaling (if necessary): If scaling is required, schedule the worker thread group to perform a step-by-step scaling operation. Step-by-step adjustments involve gradually increasing or decreasing the number of threads based on a specific step size to achieve a smooth scaling process and avoid system impact caused by drastic changes in the number of threads.

[0048] Step 5: After completing the current expansion or reduction operation, the process returns to the indicator collection thread to periodically collect relevant indicators and continue the indicator collection and subsequent processes for the next cycle.

[0049] If it is determined in the decision-making phase that no expansion or reduction operations are required, the process directly returns to the indicator collection thread to periodically collect relevant indicator steps, and continues the indicator collection and subsequent processes of the next cycle to maintain continuous monitoring and evaluation of the system status.

[0050] In order to more clearly illustrate the technical solution of the present application, the following will further illustrate it through embodiments of specific scenarios.

[0051] This embodiment provides an efficient and stable thread pool dynamic expansion and contraction solution. The following defines this method from five aspects: indicator collection, status judgment, decision logic, execution strategy, and stability assurance.

[0052] (1) Indicator collection The metrics collection module periodically collects data related to system load. Common metrics include session response time, request queue length, CPU utilization, and the number of active threads in the thread pool. Sampling is triggered by a background scheduled task, with one sample taken per cycle. Exponentially weighted average (EWMA) is used to smooth metrics. In this solution, various metrics are incorporated into the EWMA calculation. For example: 1) Response time: The average or percentile response time of recent requests reflects whether the database system throughput is reasonable or meets expectations. The latency can be recorded and accumulated when each session is completed.

[0053] 2) Request queue length: The number of sessions currently waiting to be executed can be obtained by maintaining a waiting queue count.

[0054] 3) CPU utilization: The CPU utilization of the process where the thread pool is located indicates whether there is a CPU bottleneck. It can be read regularly through the system interface / proc / stat.

[0055] 4) I / O throughput: The disk I / O utilization of the thread pool process indicates whether there is an I / O bottleneck. This can be read regularly through / proc / diskstats.

[0056] Pseudocode algorithm: 1. function CollectMetrics(): 2. cur_queue = GetCurrentQueueLength() 3. cur_latency = GetRecentResponseLatency() 4. cur_cpu = GetCurrentCPUUsage() 5. cur_disk_io = GetCurrentDiskIOUsage() 6. avg_queue = α * cur_queue + (1 – α) * avg_queue_prev 7. avg_latency = β * cur_latency + (1– β) * cur_latency_prev 8. avg_cpu = γ * cur_cpu + (1 – γ) * avg_cpu_prev 9. avg_disk_io = θ * cur_avg_disk_io + (1 – θ) * cur_avg_disk_io_prev Where α, β, γ, and θ∈(0, 1) are smoothing coefficients. EWMA filtering can make the indicator less sensitive to transient changes. The sampling interval can be set to tens of milliseconds and is user-configurable based on system overhead and decision rate.

[0057] (2) Status judgment Based on collected metrics, thread pool load is categorized into overloaded (OVERLOAD), normal (NORMAL), and idle (UNDERLOAD) states. Upper and lower thresholds are set for each metric, and logical conditions are used to determine the current state. For example, when the response time consistently exceeds RESP_TIME_HIGH and the queue length exceeds QUEUE_HIGH, it is considered overloaded. If various metrics fall below lower thresholds such as RESP_TIME_LOW, QUEUE_LOW, and CPU_LOW, it is considered excessively idle. The state machine also incorporates an anti-shake strategy: a state transition occurs only when a certain condition is met multiple times consecutively (recommended for three sampling periods), thus preventing false positives caused by momentary jitter.

[0058] Pseudocode algorithm: 1. if avg_queue>queue_high or avg_latency>resp_time_high: 2. overload_count += 1 3. idle_count = 0 4. if overload_count>= N_high: 5. state = OVERLOAD 6. elif avg_queue <queue_low and avg_latency<resp_time_low: 7. idle_count += 1 8. overload_count = 0 9. if idle_count>= N_low: 10. state = UNDERLOAD 11. else: 12. overload_count = 0 13. idle_count = 0 14. state = NORMAL The thresholds queue_high, resp_time_high, queue_low, and resp_time_low can be set based on experience or online tuning; N_high / N_low controls the duration requirement.

[0059] (3) Decision-making logic Decide whether to increase or decrease the number of threads or thread groups based on the system status: 1) Expansion Decision: If overload is detected, threads need to be added. CPU utilization and disk I / O throughput are considered. If both are below the high-load threshold, capacity can be expanded rapidly. If the CPU or disk I / O is nearing saturation, capacity expansion will be limited, and a conservative increase is recommended. The number of threads added is not a fixed value, but rather a step-by-step approach with uneven increments. This means that when resources are abundant, the number of threads is expanded in multiple steps, and the number of threads added each time is gradually reduced to dynamically adapt.

[0060] 2) Scaling decision: If idle threads are detected, it indicates an excess of threads. Idle threads can be gradually reclaimed to restore the system to a reasonable scale. CPU utilization should also be checked during scaling to avoid excessive reclaiming that could lead to idle CPUs.

[0061] 3) Boundary control: Maintain the maximum and minimum thread pool sizes MIN_THREADS / MAX_THREADS, and ensure that they do not exceed this range after expansion or contraction.

[0062] (IV) Implementation Strategy The execution policy module is responsible for actually adjusting the thread pool according to the decision logic: creating or destroying threads (groups), and managing the task distribution and notification mechanism. The implementation details include: 1) Thread creation / destruction: Use the POSIX thread (pthread) interface to create worker threads. To expand the pool, call pthread_create at the end of the original thread array to create a new thread. To shrink the pool, send an exit signal to the thread pool, allowing idle threads to terminate their loops and call pthread_join to release resources.

[0063] 2) Task Queue Design: The thread pool maintains a thread-safe task queue using mutexes and condition variables. After the main thread pushes a new task into the queue, it wakes up waiting worker threads using pthread_cond_signal or pthread_cond_broadcast. The worker threads then lock, wait, retrieve (or acquire) tasks, unlock, and execute tasks until they receive an exit signal or exit voluntarily.

[0064] 3) Thread notification mechanism: When a new task arrives, pthread_cond_signal is called to wake up at least one thread after it is queued. When there are no tasks and a request to scale down is made, a broadcast can be used to notify all idle threads to exit.

[0065] 4) Resource Release: When a thread exits, it must perform cleanup, release thread-specific resources, and terminate with a pthread_exit call. The main thread then reclaims the thread handle via pthread_join. Throughout this process, the task queue and thread counter must be modified synchronously in a multithreaded environment to ensure thread safety.

[0066] (V) Stability assurance To avoid scaling jitter and system uncertainty, this solution incorporates the following stability protection mechanisms into the strategy: 1) Anti-jitter and smoothing: Smoothing out instantaneous fluctuations through sliding windows, EWMA, and continuous counting. Scaling and reduction are triggered only when load trends actually change, thus avoiding frequent adjustments caused by instantaneous peaks.

[0067] 2) Cooling-off period: After a capacity expansion or contraction, a cooling-off period (usually a few seconds) is set. No new adjustments are made during the cooling-off period until the system returns to a steady state.

[0068] 3) Thread keepalive: For additional threads exceeding the number of core threads, set an idle keepalive time, and automatically exit after the timeout to release resources.

[0069] 4) Scaling Limits: Limit the maximum step size for each expansion or contraction, and use non-uniform step sizes to adjust multiple times within a unit time to prevent large-scale thread changes in a short period of time.

[0070] 5) Resource protection: Before scaling up, check whether system resources (such as CPU and memory) are close to saturation to avoid blindly adding threads when resources are at their limit. When scaling down, ensure that at least the necessary number of threads are retained to handle new sessions.

[0071] The above is the optimization strategy for dynamic expansion and contraction of thread pool in database system proposed by the present invention. As a specific implementation example, for users with high concurrency business model, this function can be enabled by configuring database parameters. Figure 5 and Figure 6 As shown, Figure 5 The value of enable_thread_pool is on, which means that the thread pool mode is turned on. Figure 6 The distribution of multiple thread pool threads in the database process is shown in FIG. , and the threads under these thread pools will be dynamically expanded or reduced according to the strategy described in this embodiment.

[0072] Figure 7 The present invention provides a method for dynamically expanding and contracting a database thread pool, which includes: Index collection module, used to periodically collect operating indicators of the database system; The indicator processing module is used to smooth the collected indicators to obtain effective indicators that represent the system status; The state decision module is used to judge the current load state of the system based on the effective indicators and output the decision results of overload, normal or idle; The scaling execution module is used to perform step-by-step thread number adjustment based on resource usage in overloaded or idle states, including thread creation, destruction, and task queue rebalancing; The stability control module is used to implement anti-jitter filtering, cool-down period timing, scaling range limits, and resource usage pre-detection to ensure the stability of scaling operations.

[0073] When the above device is running, the steps of the database thread pool dynamic expansion and contraction method disclosed in this application are implemented.

[0074] The flowcharts and block diagrams in the accompanying drawings illustrate possible implementations of the apparatus, methods, and computer program products according to various embodiments of the present application, including architecture, functions, and operations. In these figures, each box may represent a module, a program segment, or a portion of a code, which contains one or more executable instructions for implementing a specified logical function. It should be noted that each box in the block diagram and / or flowchart, and the combination of these boxes, can be implemented using a dedicated hardware-based system to implement the specified function or operation, or can be implemented by a combination of dedicated hardware and computer instructions.

[0075] like Figure 8As shown, an embodiment of the present application further discloses an electronic device, comprising: a processor 310, a communication interface 320, a memory 330 for storing a computer program executable by the processor, and a communication bus 340. The processor 310, the communication interface 320, and the memory 330 communicate with each other via the communication bus 340. The processor 310 executes the executable computer program to implement the steps of the above-mentioned method for dynamically scaling a database thread pool.

[0076] It is understood that, in addition to the memory and processor, the electronic device may also include an input device (e.g., a keyboard), an output device (e.g., a display), and other communication modules. These input devices, output devices, and other communication modules all communicate with the processor via an I / O interface (i.e., an input / output interface).

[0077] The operation of the present application can be implemented by writing computer program code using one or more programming languages ​​or a combination thereof. The programming languages ​​include but are not limited to the following types: Object-oriented programming languages, such as Java, Smalltalk, C++, etc.; A conventional procedural programming language, such as "C" or a similar programming language.

[0078] The execution methods of the program code include but are not limited to: Executes entirely on the user's computer; Partially executed on the user's computer and partially on a remote computer; Executed as a standalone software package; Executes entirely on the remote computer or server.

[0079] In scenarios involving a remote computer, the remote computer can be connected to the user's computer via any type of network, including but not limited to a local area network (LAN) or a wide area network (WAN). Additionally, the remote computer can be connected to an external computer via an Internet service provider, such as the Internet.

[0080] Furthermore, the present application also discloses a computer-readable storage medium. When the instructions in the computer-readable storage medium are executed by the processor of an electronic device, the electronic device can execute the various steps of the database thread pool dynamic expansion and contraction method disclosed in the present application.

[0081] In the context of this application, computer-readable storage media refers to tangible media that can store computer program code and related data. Specific examples include, but are not limited to, the following: (1) Portable computer disk: A removable magnetic storage medium such as a floppy disk.

[0082] (2) Hard disk: includes fixed storage devices such as mechanical hard disks and solid-state hard disks.

[0083] (3) Random Access Memory (RAM): Volatile storage medium used for temporary storage of data and program code.

[0084] (4) Read-only memory (ROM): A non-volatile storage medium used to store fixed programs and data.

[0085] (5) Erasable Programmable Read-Only Memory (EPROM) or Flash Memory: A non-volatile storage medium that supports multiple erasing and programming.

[0086] (6) Fiber optic storage device: storage medium based on fiber optic technology.

[0087] (7) Compact Disc Read-Only Memory (CD-ROM): A read-only medium that stores data in the form of an optical disc.

[0088] (8) Optical storage devices: storage media based on optical principles, such as DVDs and Blu-ray discs.

[0089] (9) Magnetic storage devices: storage media based on magnetic principles, such as magnetic tapes and disks.

[0090] (10) Any suitable combination of the above: for example, combining multiple storage media to meet different storage requirements.

[0091] These computer-readable storage media can be used to store the program code and related data described in this application to support the operation of the program and the persistent storage of data.

[0092] In particular, according to an embodiment of the present application, the process described in the flowchart can be implemented as a computer software program. For example, an embodiment of the present application relates to a computer program product, which includes a computer program carried on a non-transitory computer-readable medium. The computer program includes program code for executing the database thread pool dynamic expansion and contraction method disclosed in the present application. When the computer program is executed by a processing device, it can implement the above-mentioned functions defined in the embodiments of the present application.

[0093] Although the above discussion contains several specific implementation details, these details should not be interpreted as limiting the scope of this application. The above description is only a preferred embodiment of the present application and an illustration of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in this application is not limited to the technical solutions formed by the specific combination of the above technical features. At the same time, this application should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concepts.

[0094] Those skilled in the art should also understand that they may modify the technical solutions described in the aforementioned embodiments, or replace some of the technical features therein with equivalents, without departing from the spirit and scope of the technical solutions of the embodiments of the present application. Such modifications or replacements will not cause the essence of the corresponding technical solutions to deviate from the core spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for dynamically expanding and contracting a database thread pool, characterized in that: The method comprises: S1. Periodically collect operational indicators of the database system; S2. Smoothing the collected metrics to eliminate the impact of instantaneous data fluctuations and obtain effective metrics that represent the system status. S3 according to the smoothed indicators to determine the current load state of the system, the load state includes an overload state, a normal state and an idle state; S4. If the system is in an overloaded or idle state, the system performs incremental thread pool expansion and contraction based on system resource usage, including: In an overloaded state, the number of worker threads is gradually increased according to the preset step size until the system load is relieved or the maximum size of the thread pool is reached; In idle state, the number of worker threads is gradually reduced according to the preset step size until the system load is balanced or the minimum size of the thread pool is reached; S5. After the scaling operation is complete, return to the step of periodically collecting operating indicators to form a closed-loop feedback adjustment.

2. The method according to claim 1, characterized in that The operation indicators of the database system in step S1 include: session response time, processing delay time, request queue length, number of active threads, CPU resource occupancy rate, memory resource occupancy rate and disk I / O resource occupancy rate.

3. The method according to claim 1, characterized in that The smoothing process in step S2 adopts the exponential weighted average (EWMA) algorithm, which performs weighted calculation on the current indicator and the historical indicator through the smoothing coefficient to reduce the interference of instantaneous jumps on decision-making.

4. The method according to claim 1, wherein Determining the current load state of the system in step S3 includes: (1) When judging the system load status, set independent overload thresholds and idle thresholds for each indicator in advance; (2) When the session response delay exceeds the overload threshold or the request queue length exceeds the overload threshold, it is determined to be in an overload state; (3) When the session response delay is lower than the idle threshold and the request queue length is lower than the idle threshold, it is determined to be in an idle state; (4) State transitions must meet the matching conditions for at least three consecutive sampling periods to avoid misjudgments caused by instantaneous fluctuations.

5. The method according to claim 1, wherein The step S4 of performing the step-by-step thread pool expansion and contraction operation further includes: Check the system CPU and disk I / O resource usage. If the resources are close to saturation, appropriately reduce the number of threads added at a time when expanding capacity. Maintain the thread pool's thread count boundaries to ensure that the number of threads after scaling is no less than the minimum size and no more than the maximum size.

6. The method according to claim 1, characterized in that The method further includes: when executing the expansion and contraction operations, adjusting the number of threads by creating new working threads or recycling idle working threads, and performing load balancing on the task queue after expansion or before contraction to ensure the rationality of task allocation.

7. The method according to claim 1, characterized in that The method also includes the following stability assurance mechanism: Anti-jitter processing: Filter out short-term abnormal fluctuations through sliding windows or continuous counting, and trigger expansion and contraction operations only when the load trend continues to change; Cooling-off period: After each scaling operation is completed, a cooling-off period of preset duration is set, during which new scaling decisions are prohibited. Scaling limit: Specifies the maximum number of threads that can be added or reduced in a single scaling operation to avoid drastic changes in the number of threads in a short period of time.

8. A database thread pool dynamic expansion and contraction device, characterized in that: The device comprises: Index collection module, used to periodically collect operating indicators of the database system; The indicator processing module is used to smooth the collected indicators to obtain effective indicators that represent the system status; The state decision module is used to judge the current load state of the system based on the effective indicators and output the decision results of overload, normal or idle; The scaling execution module is used to perform step-by-step thread number adjustment based on resource usage in overloaded or idle states, including thread creation, destruction, and task queue rebalancing; The stability control module is used to implement anti-jitter filtering, cool-down period timing, scaling range limits, and resource usage pre-detection to ensure the stability of scaling operations.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method for dynamic expansion and contraction of a database thread pool according to any one of claims 1 to 7 are implemented.

10. An electronic device, characterized in that: include: memory and processor; Memory: used to store computer programs; Processor: used to execute the computer program to implement the steps of the database thread pool dynamic expansion and contraction method according to any one of claims 1 to 7.

Images