Methods, media, and products for dynamic management of database resource quotas
By dynamically adjusting database resource limits through closed-loop feedback control of kernel-level resource collection and decision-making strategies, the problem of rigid static configuration is solved, resource utilization and operational efficiency are improved, and resource isolation and service quality are guaranteed in a multi-tenant environment.
Patent Information
- Application Number
- CN202611256752.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-08-18
- Publication Date
- 2026-11-17
AI Technical Summary
Existing database resource management mechanisms are static or semi-static, resulting in rigid resource configuration that cannot adapt to dynamic load changes, leading to low resource utilization, high operation and maintenance costs, and a lack of automated resource elastic allocation mechanisms, which affects system performance and stability.
By collecting multi-dimensional load metrics in real time through a resource collector embedded in the database kernel, and combining it with preset configurable resource decision-making strategies, dynamic adjustment instructions for resource limits are generated and executed without loss, forming a closed-loop feedback control to achieve automated dynamic adjustment of resource limits.
It enables fully automatic and lossless dynamic control of database resources, adapts to load fluctuations, improves resource utilization, reduces operation and maintenance costs, ensures resource isolation and service quality in multi-tenant environments, and improves overall system performance.
Smart Images

Figure CN123115084A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of database technology, and in particular to a method, medium, and product for dynamic management of database resource limits. Background Technology
[0002] With the widespread adoption of cloud computing and multi-tenancy technologies, modern database systems (especially cloud-native databases) increasingly need to support a large number of diverse user workloads simultaneously within one or more database instances. In this context, database resource management has become a core technology for ensuring system stability, resource isolation, and service quality.
[0003] Existing database resource management mechanisms typically achieve this by setting resource quotas, and mainly include the following levels: Instance-level resource limits: Set the total amount of resources that can be used for a database instance, including but not limited to CPU usage, memory usage, I / O bandwidth, and other resource limits; User-level (tenant-level) resource quotas: Fixed resource usage quotas are allocated to specific user (or tenant) groups through resource groups (or resource pools); Session-level resource limits: Set the total amount of resources that the corresponding process or thread can use at the database session connection level, including but not limited to CPU usage, memory usage, I / O bandwidth, and other resource limits. Query-level (statement-level) resource limits: Limit the resource limits that a specified query statement can use to prevent abnormal queries from exhausting system resources.
[0004] However, existing resource quota management methods are mostly static or semi-static, and their main shortcomings are as follows: Inflexible resource allocation leads to low resource utilization. Database administrators typically set fixed resource limits based on historical workload experience or peak predictions, which are difficult to adjust flexibly during operation. During off-peak periods, a large amount of resources remain idle, resulting in resource waste; during peak periods, preset limits are insufficient to cope with sudden performance demands, causing query queuing, timeouts, or even execution failures, interfering with the normal operation of core business processes.
[0005] The system lacks adaptive processing capabilities for dynamic workloads. Modern database workloads exhibit significant time-varying, bursty, and mixed characteristics. For example, during the day, the workload is primarily comprised of high-concurrency online transaction processing (OLTP) tasks, while at night, a large number of analytical batch processing tasks are executed. Static resource quotas cannot detect these load changes, making it difficult for the system to allocate resources among different business operations based on the actual load status, thus hindering the overall operational efficiency of the database system.
[0006] Adjustments are often delayed and incur high maintenance costs. When static limits no longer suit business scenarios, manual intervention from database administrators is often required to analyze system monitoring data, locate performance bottlenecks, and execute corresponding management commands to adjust limit parameters. This process is slow and significantly increases operational complexity and labor costs. In large-scale distributed database cluster scenarios, manually optimizing resource limits for hundreds or thousands of tenants is practically impractical. Summary of the Invention
[0007] One objective of this invention is to provide a dynamic management method for database resource limits that can solve any of the above-mentioned problems.
[0008] A further objective of this invention is to address the problem that traditional static configuration of database resource quotas is rigid and cannot adapt to dynamic loads.
[0009] Another further objective of this invention is to avoid the drawbacks of slow manual parameter adjustment and high maintenance costs.
[0010] Specifically, the present invention provides a dynamic management method for database resource limits, comprising: collecting multi-dimensional load indicators of the database as a whole and each resource group; performing matching calculations on the multi-dimensional load indicators based on a preset configurable resource decision strategy to generate dynamic resource limit adjustment instructions; and executing the dynamic resource limit adjustment instructions without loss of quality while the database kernel is running continuously and the business is uninterrupted, thereby updating the resource limits corresponding to each resource group.
[0011] Optionally, the dynamic management method for database resource limits also includes: cyclically executing the above-mentioned process of collecting indicators, generating instructions, and updating limits to form a closed-loop feedback control, and automatically and elastically adjusting the resource limits of each resource group according to the real-time changes of multi-dimensional load indicators.
[0012] Optionally, the steps for collecting multi-dimensional load metrics of the entire database and each resource group include: collecting multi-dimensional load metrics through a resource collector embedded in the database kernel; and the resource collector has the ability to adaptively adjust the sampling period, dynamically adjusting the metric sampling period according to the load of the database kernel, shortening the sampling interval during peak load periods and extending the sampling interval during idle load periods.
[0013] Optionally, the resource collector includes multiple types of resource detectors, which collect load data in the dimensions of CPU, memory, and I / O respectively. The collected raw data is aligned and denoised before being stored in a sliding window statistical buffer for matching operations to read.
[0014] Optionally, the preset configurable resource decision-making strategy consists of multiple adjustment rules, and each rule contains at least four types of configuration elements: triggering conditions, adjustment objects, adjustment direction, and adjustment range. When the multi-dimensional load indicators meet the triggering conditions of any rule, a corresponding dynamic adjustment instruction for resource limits is generated.
[0015] Optionally, the preset configurable resource decision-making strategy is compatible with multiple decision-making models, including threshold triggering strategy, time period strategy, and business priority guarantee strategy, and selects PID control model and time series machine learning prediction model to assist in generating dynamic adjustment instructions for resource limits.
[0016] Optionally, the steps for executing dynamic resource limit adjustment instructions without loss of data include: directly modifying the limit of a single type of resource numerically; or switching the resource group to which the target session belongs to achieve batch limit changes for multiple resources.
[0017] Optionally, after the step of executing the dynamic adjustment command for resource limits without loss, the method may also include: automatically verifying the effect of the limit change and recording the adjustment log, and sending the verification result back to the collected indicators for participation in the next round of closed-loop matching calculation.
[0018] According to another aspect of the present invention, a computer-readable storage medium is also provided, on which a computer program is stored, wherein when the computer program is executed by a processor, the steps of the dynamic management method for database resource limits described above are implemented.
[0019] According to another aspect of the present invention, a computer program product is also provided, comprising a computer program, characterized in that, when the computer program is executed by a processor, it implements the steps of the dynamic management method for database resource limits described above.
[0020] The dynamic management method for database resource limits of this invention achieves fully automatic closed-loop dynamic adjustment of database resource limits through kernel-level real-time multi-dimensional load acquisition and adaptive sampling mechanism, combined with configurable intelligent control strategy. It can accurately sense the real-time load fluctuations of different resource groups and flexibly adjust resource limits for different operating scenarios such as peak business periods and idle periods. It effectively solves the technical problems of rigid static resource configuration and poor load adaptability, greatly improves the current situation of uneven database resource allocation, idle resources during off-peak periods and insufficient resources during peak periods, and significantly improves the overall resource utilization efficiency of the database.
[0021] Furthermore, the dynamic management method for database resource limits of the present invention adopts a lossless resource adjustment mechanism, distinguishes the resource adaptation logic between existing tasks and new services, and does not require interruption of database kernel operation and online services throughout the process, avoiding the shortcomings of traditional manual parameter tuning methods such as slow response, cumbersome operation, and high operation and maintenance costs. Through load data verification and feedback, log recording, and multi-type intelligent decision-making models to assist in regulation, the accuracy and stability of resource adjustment are further improved. Under the premise of ensuring strict resource isolation and stable service quality in multi-tenant and multi-service scenarios, the method greatly reduces the difficulty of database operation and maintenance and the cost of manual intervention, and adapts to the operational needs of various dynamic, sudden, and hybrid database workloads.
[0022] The above and other objects, advantages and features of the present invention will become more apparent to those skilled in the art from the following detailed description of specific embodiments of the invention in conjunction with the accompanying drawings. Attached Figure Description
[0023] The following sections will describe some specific embodiments of the invention in detail by way of example and not limitation, with reference to the accompanying drawings. The same reference numerals in the drawings denote the same or similar parts or portions. Those skilled in the art should understand that these drawings are not necessarily drawn to scale. In the drawings: Figure 1 This is a schematic flowchart of a dynamic management method for database resource quotas according to an embodiment of the present invention; Figure 2 This is a logical architecture diagram of a resource limit dynamic adjustment system for a database resource limit dynamic management method according to an embodiment of the present invention; Figure 3 This is a logical structure diagram of a resource collector for a dynamic management method of database resource limits according to an embodiment of the present invention; Figure 4 This is a flowchart illustrating the dynamic adjustment of a database resource quota dynamic management method according to an embodiment of the present invention. Figure 5 This is a schematic diagram of a computer program product according to an embodiment of the present invention; Figure 6 This is a schematic diagram of a computer-readable storage medium according to an embodiment of the present invention; and Figure 7 This is a schematic diagram of a computer device according to an embodiment of the present invention. Detailed Implementation
[0024] Those skilled in the art should understand that the embodiments described below are merely a part of the embodiments of the present invention, and not all of the embodiments of the present invention. These partial embodiments are intended to explain the technical principles of the present invention and are not intended to limit the scope of protection of the present invention. Based on the embodiments provided by the present invention, all other embodiments obtained by those skilled in the art without creative effort should still fall within the scope of protection of the present invention.
[0025] It should be noted that the logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be specifically implemented in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus or device (such as a computer-based system, a processor-included system or other system that can fetch and execute instructions from, an instruction execution system, apparatus or device).
[0026] This embodiment provides a dynamic management method for database resource limits, which can realize fully automatic, lossless, and dynamic adjustment of database resource limits. It can adapt to load fluctuations in real time for elastic resource allocation, effectively solving the problems of rigidity and low resource utilization of traditional static configuration. Figure 1 This is a schematic flowchart of a dynamic management method for database resource quotas according to an embodiment of the present invention, as shown below. Figure 1 As shown, the dynamic management method for database resource quotas generally includes the following steps: Step S102: Collect multi-dimensional load metrics for the entire database and each resource group.
[0027] Step S104: Based on the preset configurable resource decision strategy, perform matching calculations on multi-dimensional load indicators to generate dynamic adjustment instructions for resource limits.
[0028] Step S106: Execute the resource limit dynamic adjustment command without loss while the database kernel is running continuously and the business is uninterrupted, and update the resource limits corresponding to each resource group.
[0029] Specifically, the dynamic management method for database resource limits in this embodiment may further include: cyclically executing the above-mentioned process of collecting indicators, generating instructions, and updating limits to form a closed-loop feedback control, and automatically and elastically adjusting the resource limits of each resource group according to the real-time changes of multi-dimensional load indicators. That is, steps S102, S104, and S106 can be executed cyclically to form a closed-loop feedback control.
[0030] Specifically, step S102, which involves collecting multi-dimensional load metrics for the entire database and each resource group, may include: collecting multi-dimensional load metrics through a resource collector embedded in the database kernel. Furthermore, the resource collector has adaptive sampling period adjustment capabilities, dynamically adjusting the metric sampling period according to the load level of the database kernel, shortening the sampling interval during peak load periods and extending the sampling interval during idle periods.
[0031] For example, if the CPU / memory utilization of any resource group exceeds 80%, it is considered a peak load period, and the sampling period is shortened from the default 5 seconds to 1 second; if the CPU and memory utilization of all resource groups is below 30%, it is considered an idle system, and the sampling period is extended to 10 seconds.
[0032] Preferably, the resource collector may include multiple types of resource detectors to collect load data in the dimensions of CPU, memory, and I / O. The collected raw data is aligned and denoised before being stored in a sliding window statistical buffer for matching operations to read.
[0033] Specifically, the preset configurable resource decision strategy in step S104 can consist of multiple adjustment rules, with each rule containing at least four configuration elements: triggering condition, adjustment object, adjustment direction, and adjustment magnitude. When the multi-dimensional load index meets the triggering condition of any rule, a corresponding dynamic adjustment instruction for resource limits is generated.
[0034] Preferably, the preset configurable resource decision-making strategy is compatible with multiple decision-making models, including threshold triggering strategy, time period strategy, and business priority guarantee strategy, and uses PID control model and time series machine learning prediction model to assist in generating dynamic adjustment instructions for resource limits.
[0035] Specifically, step S106, which involves executing the dynamic adjustment instruction for resource limits without loss, may include: directly modifying the limit of a single type of resource by value; or switching the resource group to which the target session belongs to achieve batch limit changes for multiple resources.
[0036] Specifically, after step S106, which involves executing the resource limit dynamic adjustment instruction without loss, the process may further include: automatically verifying the effect of the limit change and recording the adjustment log, and sending the verification result back to the collected indicators for participation in the next round of closed-loop matching calculation.
[0037] It should be noted that existing database resource quota management mechanisms mainly employ static or semi-static configuration methods. This embodiment's dynamic database resource quota management method aims to address the following specific technical problems arising from this: This addresses the contradiction between low system resource utilization and compromised business performance caused by static resource limits failing to adapt to dynamically changing workloads. Specifically, during periods of low load, fixed resource limits result in a significant amount of idle computing, memory, and I / O resources, leading to resource waste. Conversely, during periods of high load, pre-set resource limits become performance bottlenecks, causing increased query response latency, task queue backlogs, and even execution failures, thus failing to meet the real-time performance requirements of the business.
[0038] This addresses the issue of performance isolation failure between tenants in multi-tenant database environments due to the lack of an automated resource elastic allocation mechanism. When one tenant's resource usage falls below its static limit while another tenant experiences resource bottlenecks, the system cannot allocate idle resources to the tenant in urgent need. This "resource silo" phenomenon prevents the system from achieving optimal global resource utilization while ensuring fairness.
[0039] This addresses the problems of response delays, high operational risks, and enormous maintenance costs associated with manual resource quota adjustments by database administrators in existing technologies. Manual adjustments cannot keep pace with changes in business load, resulting in significant lag. Furthermore, frequent manual intervention not only increases the operational burden but also risks impacting the stability and reliability of database services due to the high risk of human error.
[0040] In summary, the core objective of the dynamic management method for database resource limits in this embodiment is to provide a database kernel-level technical solution that can sense system load and resource pressure in real time and automatically and dynamically adjust resource limits accordingly, so as to achieve the goals of high resource utilization, strong performance isolation guarantee and low operation and maintenance cost without manual intervention.
[0041] Figure 2 This is a logical architecture diagram of a resource limit dynamic adjustment system for a database resource limit dynamic management method according to an embodiment of the present invention. Figure 2 This demonstrates the closed-loop structure of a dynamic resource quota adjustment system. The system consists of three core modules: a real-time monitoring module, an intelligent decision-making module, and a lossless hot update module. At the bottom is the database engine, which contains multiple resource groups and resource collectors. The real-time monitoring module collects monitoring data through the resource collectors and transmits it to the intelligent decision-making module. Based on the decision-making strategy configured by the system administrator and combined with the real-time monitoring data, the intelligent decision-making module generates an adjustment plan for the system's resource quotas. This adjustment plan is then passed to the lossless hot update module, which executes the plan by pointing to the resource group configuration in the database engine. The entire system forms a closed-loop feedback loop of "real-time monitoring - intelligent decision-making - lossless hot update - real-time monitoring," enabling resource allocation to continuously adapt to workload changes.
[0042] Resource groups, as logical objects within a database, are used to divide and isolate system resources (CPU, memory, etc.). They are typically bound to specific users, tenants, or business applications and are the basic unit for achieving multi-tenant resource isolation. The underlying implementation of resource groups can rely on operating system-level resource isolation mechanisms (including but not limited to cgroups).
[0043] Resource quotas (Quota / Limit) refer to the upper limit of resource usage allocated to a resource group, including but not limited to CPU utilization limits, memory usage limits, I / O bandwidth limits, etc. They are used to prevent the group from excessively consuming system resources. Quotas can be divided into hard limits (unbreakable) and soft limits (allowing short-term exceedances but triggering subsequent adjustments).
[0044] Specifically, the real-time monitoring module is embedded in the database engine and includes a resource collector used to continuously collect real-time performance metrics of the entire system and each resource group at a configurable sampling period. In one specific embodiment, resource groups in the database engine can achieve the partitioning and management of underlying resources through operating system-level resource isolation mechanisms (including but not limited to cgroups). The data collected by the resource collector includes, but is not limited to: CPU utilization and usage time, memory usage and utilization rate, I / O throughput and operation latency, session execution time, number of concurrent connections, and request queuing status in each resource group.
[0045] In addition, the resource collector can also have the ability to adjust the sampling frequency adaptively: when the system is under peak load, it automatically shortens the sampling period to capture rapid changes in load; when the system is idle, it appropriately extends the sampling period to reduce monitoring overhead, thereby achieving a dynamic balance between monitoring accuracy and system performance overhead.
[0046] The intelligent decision-making module receives monitoring data from the real-time monitoring module and, in conjunction with the decision-making strategies preset by the system administrator, generates a resource limit adjustment plan. The decision-making strategy is the core driving element of this embodiment. The administrator formulates one or more resource adjustment rules based on the system's business characteristics. Each rule includes (but is not limited to) the following elements: Triggering conditions: Define the system state under which the adjustment is activated, such as when the utilization rate of a certain resource in a certain resource group is higher or lower than a preset threshold, or when the request queuing depth exceeds a preset value. Adjustment Target: Specify which resource groups or resource categories will be adjusted; Adjustment direction and magnitude: Clarify whether to increase or decrease resource limits, and the step size or target value of the adjustment.
[0047] Through the above methods, administrators formulate a series of resource adjustment rules, and the collection of these rules constitutes the decision-making strategy. The intelligent decision-making module matches real-time monitoring data with the strategy rules, and when the system status meets the triggering conditions of a certain rule, it generates the corresponding adjustment plan.
[0048] In further implementations, decision-making strategies can incorporate richer strategy models, including but not limited to: time-window-based periodic strategies (such as using different quota benchmarks for daytime / nighttime), priority-based guarantee strategies (such as core business resource groups enjoying non-preemptible guaranteed quotas), and predictive strategies based on historical load trends.
[0049] Furthermore, in another equivalent implementation of this embodiment, the intelligent decision-making module can also integrate a mathematical model based on the Proportional-Integral-Derivative (PID) control algorithm to assist in decision-making. In this approach, the system introduces the concept of a "Pressure Index" (PrI), which is obtained through weighted fusion calculation of multi-dimensional indicators such as CPU utilization, memory utilization, I / O latency, and queue depth. PrI=w1×U_cpu+w2×U_mem+w3×L_io+w4×D_queue, The PID controller calculates the adjustment amount of the resource limit based on the deviation between the pressure index and the target value. Δ(t)=Kp×e(t)+Ki×∫e(τ)dτ+Kd×de(t) / dt.
[0050] Among them, the proportional gain (Kp) determines the response speed, the integral gain (Ki) eliminates steady-state error, and the derivative gain (Kd) suppresses overshoot and oscillation. e(t) is the deviation between the pressure index PrI at time t and the preset target threshold; Kp is the proportional gain, Ki is the integral gain, and Kd is the derivative gain; w1 to w4 are the weighting coefficients of each dimension index, with values ranging from 0 to 1.
[0051] Furthermore, machine learning prediction models can be introduced, such as Long Short-Term Memory (LSTM) networks trained based on historical load time series and reinforcement learning models, to predict future load trends, achieving an upgrade from "reactive adjustment" to "proactive pre-adjustment." The aforementioned PID algorithms and ML prediction models can serve as extension options beyond rule-driven strategies, providing more accurate control stability and more intelligent forward-looking capabilities, respectively.
[0052] The decision-making strategy is flexible and scalable, whether it is a simple threshold rule, a complex PID mathematical model, or a data-driven ML prediction, ensuring that this embodiment can adapt to various business scenarios from simple to complex.
[0053] The decision-making strategy is the core concept proposed in this embodiment, referring to a set of one or more resource adjustment rules preset by the system administrator based on business characteristics. Each rule includes at least a trigger condition (defining when the adjustment is activated) and an adjustment action (specifying the adjustment object, direction, and magnitude). The system automatically performs dynamic management of resource limits based on the decision-making strategy.
[0054] The lossless hot update module is responsible for safely and losslessly applying the adjustment plan generated by the intelligent decision-making module to the running database instance. This embodiment supports at least two hot update paths: The first approach is direct adjustment: This involves directly modifying the values of specific resource limits (such as CPU limits, memory limits, etc.) within a resource group. This method offers fine-grained adjustment and is suitable for scenarios requiring only minor tweaks to a single type of resource.
[0055] The second approach is resource group switching: by switching the target session or query from one resource group to another, the various resource quota constraints it is subject to can be changed in batches. This method is suitable for scenarios that require significant adjustments to multiple resource quotas simultaneously.
[0056] Regardless of the path used, this embodiment performs the update operation at the database kernel level, utilizing the concurrency control mechanism provided by the kernel to ensure that updating the quota parameters does not create race conditions with ongoing resource statistics and allocation operations. Simultaneously, this embodiment introduces "effective point" control technology: currently executing queries continue to run at the previous quota without being interrupted or rescheduled due to the quota update; newly arriving query requests read the latest quota value upon their first resource request, achieving a completely seamless transition for the user.
[0057] Hot update refers to dynamically modifying internal system parameters or configurations while the database system is running normally and providing services to the outside world, without causing service interruption, connection loss, or failure of ongoing tasks. Hot update is a key capability of modern high-availability database systems.
[0058] In this embodiment, "lossless" specifically means that when resource limits are adjusted, ongoing query tasks are not affected in any way, and their resource usage is still determined based on the limit value before the adjustment. They will not be interrupted, downgraded, or rescheduled due to the adjustment.
[0059] The entire system operates in a continuous feedback control loop: the real-time monitoring module collects performance indicators, the intelligent decision-making module generates adjustment plans based on the decision strategy, the lossless hot update module applies the plans to the database engine, and the real-time monitoring module continues to collect the adjusted system status. This process repeats, enabling resource allocation to closely follow changes in workload.
[0060] The resource limit dynamic adjustment system in this embodiment is a closed-loop feedback control system. As a control architecture, the system output (resource allocation status) is fed back to the input (decision module) through the monitoring module. It is compared with the desired status (the target defined by the decision strategy), and an adjustment action is generated, which is then applied to the system itself. This process is repeated to achieve continuous optimization and adjustment of the system status.
[0061] Figure 3 This is a logical structure diagram of a resource collector for a dynamic management method of database resource quotas according to an embodiment of the present invention. Figure 3 The internal module structure of the resource collector is shown. The top "sampling trigger" drives the entire collection process according to a configurable sampling period. The middle section contains "resource detectors," including but not limited to: CPU detectors, memory detectors, and I / O detectors, which obtain corresponding performance metrics from the database kernel and the underlying operating system's resource isolation mechanisms, respectively. The collected raw data undergoes multi-dimensional time alignment, outlier filtering, and unification processing by the "data aggregator" before being stored in the "statistical buffer." The buffer uses a sliding window mechanism to store data from the most recent sampling periods, allowing the intelligent decision-making module to consume it through an efficient memory read interface.
[0062] Figure 4 This is a flowchart illustrating the dynamic adjustment of a database resource quota dynamic management method according to an embodiment of the present invention. Figure 4 This demonstrates the complete closed-loop adjustment process of a dynamic management method for database resource limits: Start; S1: The resource collector collects real-time performance indicators; S2: The intelligent decision module matches the monitoring data with the rules in the decision strategy; S3: Determine if an adjustment rule is triggered (Yes / No); No: Return to S1 to continue to the next monitoring cycle; Yes: S4: Generate an adjustment plan (including but not limited to "adjustment object", "adjustment direction", "adjustment range", etc.); S5: The lossless hot update module executes the adjustment plan (direct adjustment or resource group switching); S6: Verify the update results and record logs; S7: Determine if the process has ended (Yes / No); No: Return to S1 to continue to the next monitoring cycle; Yes: End.
[0063] Compared with existing technologies, the dynamic management method for database resource quotas in this embodiment has the following significant advantages: Elastic scaling capability: It breaks the rigid limitations of traditional static resource quotas and can automatically adjust resource quotas according to real-time fluctuations in business load. It can automatically expand capacity to ensure performance during peak business periods and automatically shrink capacity to release resources during off-peak periods, significantly improving resource utilization.
[0064] Intelligent autonomy level: The system replaces the traditional manual tuning mode with administrator-preset decision-making strategies. It can automatically respond to load changes based on policy rules, greatly reducing the operational burden and risk of errors for maintenance personnel, and automating database resource management.
[0065] High-performance isolation guarantee: Through resource group management and dynamic quota adjustment, the "resource silo" problem in multi-tenant environments is effectively solved. While ensuring resource isolation between tenants, the cross-tenant transfer and sharing of idle resources is realized, ensuring the quality of service (QoS) of core services.
[0066] Business continuity is seamless: It adopts kernel-level lossless hot update technology, supports two update paths: direct adjustment and resource group switching. When adjusting resource limits, there is no need to restart the instance or interrupt the session, which ensures business continuity and system stability and is completely transparent to upper-layer applications.
[0067] It should be emphasized that the dynamic management method for database resource quotas in this embodiment has the following essential key technical points: A rule-based closed-loop decision-making and execution framework has been constructed, establishing a complete "monitoring-decision-execution-re-monitoring" closed-loop framework. Unlike simple single-threshold triggering mechanisms, this framework organically integrates real-time perception of system load, rule-based expression of administrator intent (decision strategies), and online execution of resource limits. Decision strategies are configured in the form of "condition-action" rules, making system behavior predictable, explainable, and auditable. Furthermore, the framework supports the integration of higher-order decision models on top of the rule engine, such as precision adjustment models based on PID control algorithms or predictive scheduling models based on machine learning, thereby maintaining framework stability while ensuring the accuracy and foresight of decisions. The rule engine supports strategy models (including but not limited to threshold triggering, periodic scheduling, and priority guarantees), which have good scalability and can adapt to various resource management scenarios ranging from simple to complex.
[0068] A lossless hot update mechanism for kernel-level resource limits: Achieving "business-unobtrusive" updates is crucial for the successful implementation of dynamic resource management in real-world production environments. This embodiment implements runtime modification capabilities for resource limits within the database kernel and provides at least two update paths: one is direct numerical modification of specific resource limit items, offering fine-grained precision; the other is batch modification of multiple limits through resource group switching, ensuring high operational efficiency. Both paths adhere to the "effective point" control principle: queries that have already obtained resource authorization continue to run with the original limits until completion, while newly arriving requests automatically apply the new limit values, thus achieving a smooth and dynamic adjustment that is completely imperceptible to the user.
[0069] Furthermore, the dynamic management method for database resource limits in this embodiment can bring the following significant technical effects: It significantly improved the overall resource utilization of the database system. Through a dynamic adjustment mechanism, it solved the problem of idle resources during off-peak business periods, enabling more balanced resource utilization in mixed load scenarios and effectively reducing the enterprise's IT infrastructure costs.
[0070] It safeguards Service Level Agreements (SLAs) under complex mixed loads. When faced with sudden traffic surges or mixed load scenarios, the system automatically identifies core business processes and ensures their resource supply according to preset strategies, avoiding blockages or failures of critical transactions due to resource contention, and significantly improving the system's throughput and response speed.
[0071] This significantly reduces the manpower costs and technical barriers to database operations and maintenance. The automated resource tuning process eliminates the lag and high risk of manual monitoring and configuration modification, freeing database administrators from tedious daily tuning tasks and allowing them to focus on higher-value architecture optimization.
[0072] It enhances fairness and isolation in a multi-tenant environment. The system breaks down resource barriers between tenants through a resource transfer mechanism, maximizing resource liquidity while ensuring isolation security, avoiding the unfair phenomenon of "one side idle while the other is congested," and improving the overall service quality.
[0073] It should be noted that regarding the collection of multidimensional load metrics, in some other embodiments, these metrics can be indirectly obtained through external monitoring tools (including but not limited to operating system-level tools such as top and iostat, or third-party performance collection agents), rather than being directly collected within the database engine. External indirect acquisition is simpler to implement and decoupled from the database engine, but it typically suffers from inherent drawbacks such as coarse granularity, accuracy limited by the external sampling period, and high latency. Kernel-embedded collectors can directly acquire fine-grained resource consumption data at the session and query levels within the database, resulting in higher data quality and making it a superior solution for accurate decision-making. However, external indirect acquisition is theoretically a functionally equivalent alternative.
[0074] Regarding the decision-making mechanism, this embodiment preferably adopts a decision-driven approach based on administrator-preset rules, which is transparent, interpretable, and auditable. In other embodiments, a mathematical model based on the PID control algorithm can be used. This involves constructing a multi-dimensional weighted stress index and using the PID controller to calculate the incremental adjustment of resource limits, offering advantages such as high mathematical accuracy and provable convergence, but with relatively high parameter tuning complexity. Alternatively, a model based on machine learning (including but not limited to reinforcement learning and time series prediction) can be used. This involves training the model to predict load trends or directly output adjustment decisions, enabling the discovery of complex load patterns that are difficult for humans to capture. However, this approach may face problems such as high model training costs, long convergence periods, and a "black box" decision-making process. All of the above solutions can achieve the goal of dynamic adjustment and are functionally equivalent technical means. However, the rule-driven approach preferred in this embodiment has a more comprehensive advantage in terms of interpretability, determinism, and deployment cost.
[0075] Regarding hot updates of quotas, in some other embodiments, the user space can modify the database configuration file through an external management program, and then send a signal to trigger the database to reload the configuration. However, this solution usually has an effect delay or requires connection migration to achieve, making it difficult to achieve the "instant effect, business-unaware" effect of kernel-level updates in this embodiment. Therefore, kernel-level hot updates are a better solution for achieving high-performance dynamic resource management, but the external management program solution is theoretically a lower-order implementation with equivalent functionality.
[0076] The flowcharts provided in the above embodiments are not intended to indicate that the operations of the method will be performed in any particular order, or that all operations of the method are included in all every case. Furthermore, the method may include additional operations. Within the scope of the technical concept provided by the methods in the above embodiments, additional variations can be made to the above methods.
[0077] This embodiment also provides a computer program product, a computer-readable storage medium, and a computer device. Figure 5This is a schematic diagram of a computer program product 500 according to an embodiment of the present invention. Figure 6 This is a schematic diagram of a computer-readable storage medium 300 according to an embodiment of the present invention. Figure 7 This is a schematic diagram of a computer device 400 according to an embodiment of the present invention.
[0078] Computer program product 500 includes computer program 310, which, when executed by processor 410, implements the steps of the dynamic management method for database resource limits described above. Computer-readable storage medium 300 stores the aforementioned computer program 310, which, when executed by processor 410, implements the steps of the dynamic management method for database resource limits described above. Computer device 400 may include memory 420, processor 410, and computer program 310 stored in memory 420 and running on processor 410.
[0079] The computer program 310 used to perform the operations of the present invention may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, status setting data, integrated circuit configuration data, or source code or object code written in any combination of one or more programming languages and procedural programming languages.
[0080] Computer program 310 may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter case, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).
[0081] In some embodiments, in order to perform aspects of the present invention, electronic circuits including, for example, programmable logic circuits, field-programmable gate arrays (FPGAs) or programmable logic arrays (PLAs) can execute computer-readable program instructions to personalize the electronic circuits by utilizing state information of computer-readable program instructions.
[0082] For the purposes of this embodiment, computer program product 500 is a related product containing computer program 310. For the purposes of this embodiment, computer-readable storage medium 300 is a tangible device capable of holding and storing computer program 310, and can be any device capable of containing, storing, communicating, propagating or transmitting computer program 310 for use by or in conjunction with an instruction execution system, apparatus or device.
[0083] More specific examples (a non-exhaustive list) of computer-readable storage media 300 include the following: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable optical disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, and any suitable combination of the foregoing.
[0084] Computer device 400 may include memory 420, processor 410, and computer program 310 stored on memory 420 and running on processor 410. When processor 410 executes computer program 310, it implements the steps of the dynamic management method for database resource quotas of any of the above embodiments.
[0085] It should be noted that the logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be specifically implemented in any machine-readable storage medium for use by, or in conjunction with, instruction execution systems, apparatuses or devices (such as computer-based systems, processor-based systems or other systems that can fetch and execute instructions from, or instruction execution systems, apparatuses or devices).
[0086] It should be understood that various parts of the present invention can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system.
[0087] Computer device 400 can be, for example, a server, desktop computer, laptop computer, tablet computer, or smartphone. In some examples, computer device 400 can be a cloud computing node. Computer device 400 can be described in the general context of computer system executable instructions (such as program modules) executed by a computer system. Typically, program modules can include routines, programs, object programs, components, logic, data structures, etc., that perform specific tasks or implement specific abstract data types. Computer device 400 can be implemented in a distributed cloud computing environment where tasks are performed by remote processing devices linked through a communication network. In a distributed cloud computing environment, program modules can reside on local or remote computing system storage media, including storage devices.
[0088] Computer device 400 may include a processor 410 adapted to execute stored instructions and a memory 420 that provides temporary storage space for the operation of instructions during operation. The processor 410 may be a single-core processor, a multi-core processor, a computing cluster, or any other configuration. The memory 420 may include random access memory (RAM), read-only memory, flash memory, or any other suitable storage system.
[0089] The processor 410 can be connected via a system interconnect (e.g., PCI, PCI-Express, etc.) to an I / O interface (input / output interface) suitable for connecting the computer device 400 to one or more I / O devices (input / output devices). I / O devices may include, for example, a keyboard and indicating devices, where indicating devices may include a touchpad or touchscreen, etc. I / O devices may be built into the computer device 400 or may be external devices connected to the computing device.
[0090] The processor 410 may also be linked via a system interconnect to a display interface suitable for connecting the computer device 400 to a display device. The display device may include a display screen as a built-in component of the computer device 400. The display device may also include an external computer monitor, television, or projector connected to the computer device 400. Furthermore, a network interface controller (NIC) may be adapted to connect the computer device 400 to a network via a system interconnect. In some embodiments, the NIC may use any suitable interface or protocol (such as an Internet Minicomputer System Interface) to transmit data. The network may be a cellular network, a radio network, a wide area network (WAN), a local area network (LAN), or the Internet, etc. Remote devices may connect to the computing device via the network.
[0091] Therefore, those skilled in the art should recognize that although numerous exemplary embodiments of the present invention have been shown and described in detail herein, many other variations or modifications conforming to the principles of the present invention can be directly determined or derived from the disclosure of the present invention without departing from the spirit and scope of the invention. Thus, the scope of the present invention should be understood and construed as covering all such other variations or modifications.
Claims
1. A dynamic management method for database resource quotas, comprising: Collect multi-dimensional load metrics for the entire database and each resource group; Based on a preset configurable resource decision-making strategy, the multi-dimensional load indicators are matched and calculated to generate a dynamic adjustment instruction for resource limits. as well as The resource limit dynamic adjustment command is executed seamlessly while the database kernel is running continuously and the business is uninterrupted, updating the resource limits corresponding to each resource group.
2. The method according to claim 1, further comprising: The process of collecting indicators, generating instructions, and updating limits is executed cyclically to form a closed-loop feedback control, which automatically and elastically adjusts the resource limits of each resource group according to the real-time changes of the multi-dimensional load indicators.
3. The method according to claim 1, wherein the step of collecting multi-dimensional load indicators of the database as a whole and each resource group includes: The multidimensional load metrics are collected through a resource collector embedded in the database kernel; and The resource collector has an adaptive sampling period adjustment capability, which dynamically adjusts the index sampling period according to the load of the database kernel, shortening the sampling interval during peak load periods and extending the sampling interval during idle load periods.
4. The method according to claim 3, wherein, The resource collector includes multiple types of resource detectors, which collect load data in the dimensions of CPU, memory, and I / O respectively. The collected raw data is aligned and denoised before being stored in a sliding window statistical buffer for the matching operation to read.
5. The method according to claim 1, wherein, The preset configurable resource decision-making strategy consists of multiple adjustment rules, and each rule contains at least four types of configuration elements: triggering conditions, adjustment objects, adjustment directions, and adjustment magnitudes. When the multidimensional load metric meets the triggering condition of any rule, a corresponding dynamic adjustment instruction for the resource limit is generated.
6. The method according to claim 5, wherein, The preset configurable resource decision-making strategy is compatible with multiple decision-making models, including threshold triggering strategy, time period strategy, and business priority guarantee strategy, and uses PID control model and time series machine learning prediction model to assist in generating the dynamic adjustment instruction of the resource limit.
7. The method according to claim 1, wherein the step of executing the resource limit dynamic adjustment instruction without loss includes: Directly modify the limit of a single type of resource; or Switch the resource group to which the target session belongs to achieve batch quota changes for multiple resources.
8. The method of claim 1, further comprising, after the step of executing the resource limit dynamic adjustment instruction without loss: The automatic verification of the limit change effect and the recording of the adjustment log are then sent back to the step of collecting indicators for participation in the next round of closed-loop matching calculation.
9. A computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the dynamic management method for database resource limits according to any one of claims 1 to 8.
10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the dynamic management method for database resource limits as described in any one of claims 1 to 8.
