Database archiving self-adaptive regulation and control method and system based on load awareness

Abstract

The invention belongs to the technical field of database management, and particularly relates to a load awareness-based database archiving adaptive regulation and control method and system, and the method comprises the steps: submitting a standardized archiving application, carrying out the forced verification of archiving parameters and structured query language statements, supporting the reuse of historical templates, and generating a verified archiving task; performing pre-execution evaluation on the verified archiving task, and generating an evaluation report containing an estimated influence line number and execution time consumption; dynamically identifying a low-peak window period based on a system load state of a database, and scheduling and executing the verified archiving task in the identified low-peak window period; during a task execution period, monitoring performance indexes of the database in real time, and adaptively adjusting a task execution rate according to a monitoring result; the full life cycle state of the archiving task is tracked, fed back and recorded to form closed-loop management, so that the problems that a traditional archiving mode depends on manpower, the efficiency is low, and the risk is uncontrollable are solved.

Description

Technical Field

[0001] This invention belongs to the field of database management technology, specifically relating to a load-aware adaptive control method and system for database archiving. Background Technology

[0002] In current database operation and maintenance management, data archiving operations typically rely on database administrators (DBAs) manually writing scripts and selecting static time periods (such as late at night) for execution, or using simple scheduled task tools. These technical solutions have the following drawbacks:

[0003] 1. The execution of archiving tasks lacks the ability to dynamically perceive the real-time load of the database, and cannot automatically avoid peak business periods. This can easily cause database performance fluctuations during execution, affecting the stability of online business.

[0004] 2. The entire process is highly dependent on human experience. From script correctness review and execution timing judgment to abnormal intervention during operation, DBAs are required to participate in the entire process, which is inefficient and poses a risk of human error.

[0005] 3. Lack of an effective pre-execution assessment mechanism makes it difficult for DBAs to quantify and assess the number of data rows and execution time that archiving operations will affect before execution, making it impossible to identify potential risks in advance. During task execution, there is a lack of an automatic adjustment mechanism based on resource indicators. When there is intense lock waiting or CPU resource exhaustion, it is impossible to automatically take protective measures such as slowing down or terminating. Summary of the Invention

[0006] The technical problem to be solved by this invention is to overcome the problems of low efficiency, uncontrollable risks, and easy impact on business caused by the reliance on manual operation and static strategies in traditional database archiving technology.

[0007] To address the aforementioned technical problems, a first aspect of this invention discloses a load-aware adaptive control method for database archiving, the method comprising:

[0008] Submit a standardized archiving application, enforce validation of archiving parameters and structured query language statements, support the reuse of historical templates, and generate validated archiving tasks;

[0009] Perform a pre-execution evaluation on the verified archiving task to generate an evaluation report that includes the estimated number of rows affected and the execution time;

[0010] Based on the database, the system load status dynamically identifies low-peak windows and schedules the execution of the verified archiving tasks within the identified low-peak windows.

[0011] During task execution, the database performance metrics are monitored in real time, and the task execution rate is adaptively adjusted or the task is terminated based on the monitoring results.

[0012] The entire lifecycle status of archived tasks is tracked, fed back, and recorded to form a closed-loop management system.

[0013] As an optional implementation, in the first aspect of the present invention, the submission of a standardized archiving application, the mandatory validation of archiving parameters and structured query language statements, and the support for historical template reuse, to generate a validated archiving task include:

[0014] Receive an archiving request submitted by a user, the request including the target database, target table, and structured query language statements for archiving conditions;

[0015] The archived conditional structured query language statement is parsed using a parser to generate an abstract syntax tree;

[0016] Traverse the abstract syntax tree to verify whether the statement type is an allowed operation type, whether the target table is in the pre-configured whitelist, whether the condition field is an allowed archive field, and identify dangerous condition patterns;

[0017] After verification, or after the user selects a historical template and fills in the parameters and verification is passed, the verified archive task is generated.

[0018] As an optional implementation, in the first aspect of the present invention, the step of performing a pre-execution evaluation on the verified archiving task to generate an evaluation report including the estimated number of affected rows and execution time includes:

[0019] Perform interpretive operations of the verified archiving task in a test environment or on a read-only slave;

[0020] The estimated number of rows affected by the archive task can be obtained using the execution plan analysis command provided by the database.

[0021] Estimate the execution time of the archiving task by executing analysis commands or conducting small-scale trial executions;

[0022] Summarize the analysis results and generate the aforementioned assessment report, which includes the number of rows affected, estimated time taken, and recommendations for index usage.

[0023] As an optional implementation, in the first aspect of the present invention, the dynamic identification of low-peak windows based on system load status using a database includes:

[0024] Continuously collect the number of database queries executed per second and the CPU utilization as load indicators;

[0025] Compare the current load metrics with the historical load baseline for the same period;

[0026] When the current load index is lower than the historical load baseline for the same period by a certain percentage and the absolute value is lower than the preset safety threshold, the current period is determined to be the off-peak window period.

[0027] As an optional implementation, in the first aspect of the present invention, the real-time monitoring of the database's performance indicators and the adaptive adjustment of the task execution rate or termination of the task based on the monitoring results include:

[0028] Monitor database CPU utilization and lock wait rate during task execution;

[0029] When the CPU utilization or lock wait rate exceeds a first threshold, the task execution rate is reduced by increasing the sleep interval between batch processing.

[0030] When the CPU utilization or lock wait rate exceeds a second threshold higher than the first threshold, the database connection is forcibly terminated to stop the archiving task.

[0031] As an optional implementation, in the first aspect of the present invention, the tracking, feedback, and recording of the entire lifecycle status of the archiving task includes:

[0032] The status of archived tasks is updated and displayed in real time, including statuses such as pending review, pending execution, in progress, successful execution, failed execution, and terminated.

[0033] When an exception occurs during task execution, a retry or rollback operation is automatically performed;

[0034] The entire process of archiving tasks, from application, review, pre-execution to execution and monitoring, is stored in a structured manner.

[0035] As an optional implementation, in the first aspect of the present invention, the historical template is stored in a structured data format, including a structured query language statement skeleton and configurable parameter definitions;

[0036] After the user selects a template and fills in the parameters, a new structured query language statement is generated by the parameterized template engine. This newly generated statement must go through the mandatory validation process again.

[0037] As an optional implementation, in the first aspect of the present invention, the method enables the submission of the standardized archiving application, the display of the evaluation report, the real-time feedback of the task status, and the query of the full-process log through a graphical user interface.

[0038] A second aspect of this invention discloses a load-aware adaptive control system for database archiving, used to implement the load-aware adaptive control method for database archiving described in any of the above embodiments, the system comprising:

[0039] The verification module is used to receive and process standardized archiving requests, perform mandatory verification of archiving parameters and structured query language statements, support the reuse of historical templates, and generate verified archiving tasks.

[0040] The evaluation module is used to perform a pre-execution evaluation on the verified archiving task and generate an evaluation report that includes the estimated number of rows affected and the execution time.

[0041] The scheduling module is used to dynamically identify low-peak windows in the database's system load status and schedule the execution of the verified archiving tasks within the identified low-peak windows.

[0042] The control module is used to monitor the database's performance metrics in real time during task execution and adaptively adjust the task execution rate or terminate the task based on the monitoring results.

[0043] The processing module is used to track, provide feedback on, and record the entire lifecycle status of archived tasks, forming a closed-loop management system.

[0044] A third aspect of this invention discloses another load-aware adaptive control system for database archiving, the system comprising:

[0045] Memory containing executable program code;

[0046] A processor coupled to the memory;

[0047] The processor calls the executable program code stored in the memory to execute a load-aware adaptive control method for database archiving disclosed in the first aspect of the present invention.

[0048] The fourth aspect of the present invention discloses a computer-readable storage medium storing computer instructions, which, when invoked by a processor, are used to execute a load-aware adaptive control method for database archiving disclosed in the first aspect of the present invention.

[0049] Compared with the prior art, the beneficial effects of the present invention are:

[0050] By constructing a standardized closed-loop management process, this invention automates and intelligentizes database archiving operations. Through mandatory validation and template reuse mechanisms, it standardizes the submission of archiving requests, reducing human error. The introduced pre-execution evaluation function can analyze the execution plan and resource consumption of archiving scripts in advance in a test environment, generating quantified reports on the number of rows affected and the estimated time consumption, providing data support for review and replacing manual judgment. The system continuously monitors database performance indicators, dynamically identifying time windows with low system load as execution periods, avoiding the impact of archiving tasks on peak business periods. During execution, by monitoring key indicators such as CPU utilization and lock wait rate in real time, it dynamically adjusts the task execution rate or automatically terminates tasks when resource pressure is too high, ensuring the stability of the main database service. This invention integrates previously scattered, manual processes into a complete, fully automated collaborative system, reducing operation and maintenance costs and improving the efficiency, security, and auditability of archiving operations. Attached Figure Description

[0051] The specific embodiments of the present invention will be further described in detail below with reference to the accompanying drawings, wherein:

[0052] Figure 1 This is a flowchart illustrating a load-aware adaptive control method for database archiving disclosed in an embodiment of the present invention.

[0053] Figure 2 This is a schematic diagram of the structure of a load-aware adaptive control system for database archiving disclosed in an embodiment of the present invention;

[0054] Figure 3 This is a schematic diagram of another load-aware adaptive control system for database archiving disclosed in an embodiment of the present invention.

[0055] Figure 4 This is a logical schematic diagram of a load-aware adaptive control method for database archiving disclosed in an embodiment of the present invention. Detailed Implementation

[0056] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0057] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this invention are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, apparatus, product, or end that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or ends.

[0058] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0059] This invention discloses a load-aware adaptive control method and system for database archiving. By constructing a standardized closed-loop management process, it achieves automation and intelligence in database archiving operations. Through mandatory verification and template reuse mechanisms, this invention standardizes the submission of archiving applications, reduces human error, and introduces a pre-execution evaluation function that can analyze the execution plan and resource consumption of archiving scripts in advance in a test environment, generating a quantitative report on the number of affected rows and the estimated time consumption, providing data support for review and replacing manual experience judgment. By continuously monitoring database performance indicators, the system can dynamically identify time windows with low system load as execution periods, avoiding the impact of archiving tasks on peak business periods.

[0060] Example 1

[0061] Please see Figure 1 , Figure 1 This is a flowchart illustrating a load-aware adaptive control method for database archiving disclosed in an embodiment of the present invention. Figure 1 The described load-aware adaptive control method for database archiving is applied to a data processing chip, processing terminal, or processing server, which can be a local server or a cloud server; this embodiment of the invention does not limit the application. Figure 1 As shown, this load-aware adaptive tuning method for database archiving may include the following operations:

[0062] 101. Submit a standardized archiving application, enforce validation of archiving parameters and structured query language statements, support the reuse of historical templates, and generate validated archiving tasks.

[0063] Specifically, this process standardizes the script submission process, which originally relied on manual experience review, through mandatory parameters and syntax validation rules. It can automatically identify and intercept non-compliant SQL statements and potentially dangerous operations, discovering and avoiding errors in advance during the application submission stage, thus reducing the risk of data misoperation or system failure due to script problems from the source.

[0064] As can be seen, by combining the historical template reuse mechanism, this process improves operational efficiency while ensuring security. Developers do not need to repeatedly write similar scripts; they can quickly generate applications by parameterizing them, reducing repetitive work and ensuring the consistency of the operation mode. This not only speeds up the application process but also provides standardized input for subsequent review and execution.

[0065] 102. Perform a pre-execution assessment on the verified archiving task and generate an assessment report that includes the estimated number of rows affected and the execution time.

[0066] Specifically, this pre-execution assessment mechanism transforms the impact of archiving operations from intangible, empirically predicted values ​​into quantifiable data metrics. By simulating execution in an isolated environment, it can obtain estimates of the number of rows affected in the database and the execution time in advance. This allows database administrators to assess task risks based on concrete data rather than personal experience, providing an objective basis for audit decisions.

[0067] As can be seen, this quantitative evaluation process helps to identify potential performance bottlenecks or script defects before execution, such as full table scans or unreasonable index usage. This allows database administrators to make optimization suggestions or request modifications before the task is officially executed, avoiding the risk of excessive resource consumption and execution interruption caused by running inefficient scripts directly in the production environment.

[0068] 103. Based on the database, the system load status dynamically identifies low-peak windows and schedules the execution of the verified archiving tasks within the identified low-peak windows.

[0069] Specifically, this step dynamically identifies off-peak windows based on real-time system load data, replacing manual judgment that relies on fixed schedules. This decision-making method based on actual operating status improves the accuracy of task execution timing selection and reduces the risk of impacting online business due to misjudgment of system busyness.

[0070] It is evident that by combining the task scheduling mechanism with the window period awareness capability, tasks are automatically triggered when system resources are relatively abundant. The collaborative work method reduces the reliance on manual supervision and ensures that archiving operations can be executed during the period with the least impact on database performance.

[0071] 104. During task execution, monitor database performance metrics in real time and adaptively adjust task execution rate or terminate task based on monitoring results.

[0072] Specifically, this mechanism provides real-time situational awareness for the execution of archiving tasks by continuously tracking key performance indicators of the database. This enables the system to detect resource contention or performance fluctuations caused by task execution in a timely manner, and to maintain the overall load of the database within a controllable range by dynamically adjusting the execution rate.

[0073] It is evident that the automatic control strategy based on preset thresholds enables the system to proactively respond to potential risks. When performance indicators deteriorate to the point that they may affect core business operations, the system can automatically terminate tasks, thereby avoiding a severe decline in database service performance or unavailability due to archiving operations, and ensuring system stability.

[0074] 105. Track, provide feedback on, and record the entire lifecycle status of archiving tasks to form a closed-loop management system.

[0075] Specifically, this closed-loop management mechanism enables transparent tracking and real-time feedback of the status of archived tasks at each stage from application to completion. It unifies and centralizes information that was originally scattered across different personnel or systems, enabling all relevant parties to clearly understand the current progress of the task and improving the efficiency of operation and maintenance collaboration.

[0076] It is evident that automated anomaly handling and structured recording of the entire process log provide a complete data foundation for operation and maintenance auditing and problem tracing. This not only enables rapid identification of the cause and implementation of remedial measures when task deviations occur, but also forms a traceable operation and maintenance archive, enhancing the standardization and auditability of the process.

[0077] As an optional embodiment, the steps described above, including submitting a standardized archiving application, performing mandatory validation of archiving parameters and structured query language statements, supporting the reuse of historical templates, and generating a validated archiving task, include:

[0078] Receive an archiving request submitted by a user, the request including the target database, target table, and structured query language statements for archiving conditions;

[0079] The archived conditional structured query language statement is parsed using a parser to generate an abstract syntax tree;

[0080] Traverse the abstract syntax tree to verify whether the statement type is an allowed operation type, whether the target table is in the pre-configured whitelist, whether the condition field is an allowed archive field, and identify dangerous condition patterns;

[0081] After verification, or after the user selects a historical template and fills in the parameters and verification is passed, the verified archive task is generated.

[0082] In this embodiment of the invention, the process improves script validation from shallow judgment based on text matching to precise analysis of SQL statement structure by introducing a deep parsing method using a syntax analyzer and an abstract syntax tree. This structured analysis method can reliably identify potentially dangerous conditions such as WHERE1=1, thereby intercepting risky scripts at the application submission stage.

[0083] As can be seen, by combining whitelist verification and historical template reuse mechanism, this process not only ensures security but also standardizes the operation mode. It ensures that archiving operations are only performed on allowed data objects and fields, and reduces the randomness of rewriting scripts through parameterized templates, thereby reducing the risk of operation failure due to syntax errors or improper logic from the source.

[0084] As an optional embodiment, the step described above, namely, performing a pre-execution evaluation on the verified archiving task to generate an evaluation report containing the estimated number of affected rows and execution time, includes:

[0085] Perform interpretive operations of the verified archiving task in a test environment or on a read-only slave;

[0086] The estimated number of rows affected by the archive task can be obtained using the execution plan analysis command provided by the database.

[0087] Estimate the execution time of the archiving task by executing analysis commands or conducting small-scale trial executions;

[0088] Summarize the analysis results and generate the aforementioned assessment report, which includes the number of rows affected, estimated time taken, and recommendations for index usage.

[0089] In this embodiment of the invention, this step isolates the script verification and resource evaluation process from the production environment by performing interpretive operations on the test environment or read-only slave database. This avoids performance risks or data inconsistency issues that may be caused by trial and error on the production database directly, thus creating conditions for security assessment.

[0090] As can be seen, by utilizing the database's own execution plan analysis function, this process can obtain internal execution details such as index utilization efficiency and data scanning methods. These details, together with quantitative indicators such as the number of rows affected and the estimated time, constitute an evaluation report, providing specific technical basis for database administrators to optimize scripts and execution strategies.

[0091] As an optional embodiment, the step above, namely, dynamically identifying the low-peak window period based on the system load status of the database, includes:

[0092] Continuously collect the number of database queries executed per second and the CPU utilization as load indicators;

[0093] Compare the current load metrics with the historical load baseline for the same period;

[0094] When the current load index is lower than the historical load baseline for the same period by a certain percentage and the absolute value is lower than the preset safety threshold, the current period is determined to be the off-peak window period.

[0095] In this embodiment of the invention, this step, by continuously collecting real-time load indicators and dynamically comparing them with historical baselines, changes the static scheduling mode that relies on a fixed schedule. The judgment method based on the actual load status can more accurately reflect the true idle level of the system and improve the accuracy of window period identification.

[0096] It is evident that by setting dual criteria of "below a certain percentage of the historical baseline" and "below the absolute safety threshold," the reliability of window period decisions is enhanced. This approach considers both the periodicity of system load and ensures that the absolute value of the load is within a safe range, thereby avoiding the execution of tasks during periods of relatively low load but still high absolute load.

[0097] As an optional embodiment, the step described above, namely, real-time monitoring of the database's performance metrics and adaptively adjusting the task execution rate or terminating the task based on the monitoring results, includes:

[0098] Monitor database CPU utilization and lock wait rate during task execution;

[0099] When the CPU utilization or lock wait rate exceeds a first threshold, the task execution rate is reduced by increasing the sleep interval between batch processing.

[0100] When the CPU utilization or lock wait rate exceeds a second threshold higher than the first threshold, the database connection is forcibly terminated to stop the archiving task.

[0101] In this embodiment of the invention, the mechanism provides a dynamic view of resource consumption for task execution by tracking two key performance indicators, CPU utilization and lock wait rate, in real time. This enables the system to proactively alleviate the competitive pressure on database resources by automatically increasing the batch processing interval when performance bottlenecks first appear.

[0102] It is evident that by adopting a tiered threshold control strategy, differentiated responses to abnormal system states are achieved. The system prioritizes non-intrusive rate reduction for adjustment. If the load continues to deteriorate to a higher threshold level, forced termination measures are taken. This tiered response mechanism aims to avoid systemic risks caused by the failure of a single control strategy.

[0103] As an optional embodiment, the step of tracking, providing feedback on, and recording the entire lifecycle status of the archiving task in the above steps includes:

[0104] The status of archived tasks is updated and displayed in real time, including statuses such as pending review, pending execution, in progress, successful execution, failed execution, and terminated.

[0105] When an exception occurs during task execution, a retry or rollback operation is automatically performed;

[0106] The entire process of archiving tasks, from application, review, pre-execution to execution and monitoring, is stored in a structured manner.

[0107] In this embodiment of the invention, by clearly defining the task status and updating feedback in real time, the mechanism provides a unified progress view for all participants. Status transparency reduces communication costs between different roles (such as developers and database administrators) and makes task progress predictable.

[0108] It is evident that by combining automatic exception handling with structured storage of full-process logs, this mechanism not only enhances the system's ability to autonomously respond to common operational problems, but also provides complete data records for analyzing operational behavior, tracing the root causes of problems, and conducting audit analysis.

[0109] As an optional embodiment, in the above steps, the historical template is stored in a structured data format, including a structured query language statement skeleton and configurable parameter definitions;

[0110] After the user selects a template and fills in the parameters, a new structured query language statement is generated by the parameterized template engine. This newly generated statement must go through the mandatory validation process again.

[0111] In this embodiment of the invention, the mechanism improves the standardization and reusability of script writing by saving the core logic of the archived statements in a structured template form and defining variable elements as parameters, thereby reducing the writing of repetitive code and lowering the risk of syntax errors and logical inconsistencies caused by manual writing.

[0112] It is evident that the combination of parameterized template engine and mandatory validation process improves efficiency while ensuring security. Even SQL statements generated by trusted templates must undergo complete syntax and security rule validation, which adds a layer of protection against new risks that may be introduced by filling in template parameters, thus ensuring the security of the operation.

[0113] As an optional embodiment, in the above steps, the method enables the submission of the standardized archiving application, the display of the evaluation report, the real-time feedback of the task status, and the query of the full-process log through a graphical user interface.

[0114] In this embodiment of the invention, the graphical user interface provides a unified entry point and a visual information display center for all archiving-related operations, integrating the operation process that originally relied on command lines or scattered tools into a standardized interface, reducing the technical threshold of operation, and improving the convenience of task submission, report viewing and progress tracking.

[0115] As can be seen, by providing real-time feedback on task status and visual query of the entire process log, this interface enhances the transparency and traceability of the operation and maintenance process, enabling administrators to quickly grasp the overall operation status of the system and providing intuitive data support for problem diagnosis and operation and maintenance audit.

[0116] Example 2

[0117] Please see Figure 2 , Figure 2 This is a schematic diagram of the structure of a load-aware adaptive control system for database archiving, as disclosed in an embodiment of the present invention. Figure 2 The described load-aware adaptive control system for database archiving can be applied to data processing chips, processing terminals, or processing servers, and the processing server can be a local server or a cloud server; this embodiment of the invention does not limit the application. Figure 2 As shown, the load-aware adaptive control system for database archiving can include the following operations:

[0118] The verification module 201 is used to receive and process standardized archiving applications, perform mandatory verification of archiving parameters and structured query language statements, support the reuse of historical templates, and generate verified archiving tasks.

[0119] Specifically, this process standardizes the script submission process, which originally relied on manual experience review, through mandatory parameters and syntax validation rules. It can automatically identify and intercept non-compliant SQL statements and potentially dangerous operations, discovering and avoiding errors in advance during the application submission stage, thus reducing the risk of data misoperation or system failure due to script problems from the source.

[0120] As can be seen, by combining the historical template reuse mechanism, this process improves operational efficiency while ensuring security. Developers do not need to repeatedly write similar scripts; they can quickly generate applications by parameterizing them, reducing repetitive work and ensuring the consistency of the operation mode. This not only speeds up the application process but also provides standardized input for subsequent review and execution.

[0121] The evaluation module 202 is used to perform a pre-execution evaluation on the verified archiving task and generate an evaluation report that includes the estimated number of rows affected and the execution time.

[0122] Specifically, this pre-execution assessment mechanism transforms the impact of archiving operations from intangible, empirically predicted values ​​into quantifiable data metrics. By simulating execution in an isolated environment, it can obtain estimates of the number of rows affected in the database and the execution time in advance. This allows database administrators to assess task risks based on concrete data rather than personal experience, providing an objective basis for audit decisions.

[0123] As can be seen, this quantitative evaluation process helps to identify potential performance bottlenecks or script defects before execution, such as full table scans or unreasonable index usage. This allows database administrators to make optimization suggestions or request modifications before the task is officially executed, avoiding the risk of excessive resource consumption and execution interruption caused by running inefficient scripts directly in the production environment.

[0124] The scheduling module 203 is used to dynamically identify low-peak windows in the system load status of the database, and schedule the execution of the verified archiving tasks within the identified low-peak windows.

[0125] Specifically, this step dynamically identifies off-peak windows based on real-time system load data, replacing manual judgment that relies on fixed schedules. This decision-making method based on actual operating status improves the accuracy of task execution timing selection and reduces the risk of impacting online business due to misjudgment of system busyness.

[0126] It is evident that by combining the task scheduling mechanism with the window period awareness capability, tasks are automatically triggered when system resources are relatively abundant. The collaborative work method reduces the reliance on manual supervision and ensures that archiving operations can be executed during the period with the least impact on database performance.

[0127] The control module 204 is used to monitor the performance indicators of the database in real time during task execution, and adaptively adjust the task execution rate or terminate the task based on the monitoring results.

[0128] Specifically, this mechanism provides real-time situational awareness for the execution of archiving tasks by continuously tracking key performance indicators of the database. This enables the system to detect resource contention or performance fluctuations caused by task execution in a timely manner, and to maintain the overall load of the database within a controllable range by dynamically adjusting the execution rate.

[0129] It is evident that the automatic control strategy based on preset thresholds enables the system to proactively respond to potential risks. When performance indicators deteriorate to the point that they may affect core business operations, the system can automatically terminate tasks, thereby avoiding a severe decline in database service performance or unavailability due to archiving operations, and ensuring system stability.

[0130] The processing module 205 is used to track, provide feedback on, and record the status of the entire lifecycle of the archiving task, forming a closed-loop management system.

[0131] Specifically, this closed-loop management mechanism enables transparent tracking and real-time feedback of the status of archived tasks at each stage from application to completion. It unifies and centralizes information that was originally scattered across different personnel or systems, enabling all relevant parties to clearly understand the current progress of the task and improving the efficiency of operation and maintenance collaboration.

[0132] It is evident that automated anomaly handling and structured recording of the entire process log provide a complete data foundation for operation and maintenance auditing and problem tracing. This not only enables rapid identification of the cause and implementation of remedial measures when task deviations occur, but also forms a traceable operation and maintenance archive, enhancing the standardization and auditability of the process.

[0133] Example 3

[0134] Please see Figure 3 , Figure 3 This is a schematic diagram of another load-aware adaptive control system for database archiving disclosed in an embodiment of the present invention. Figure 3 As shown, the device may include:

[0135] Memory 301 storing executable program code;

[0136] Processor 302 coupled to memory 301;

[0137] The processor 302 calls the executable program code stored in the memory 301 to execute some or all of the steps in the load-aware adaptive control method for database archiving disclosed in Embodiment 1 of the present invention.

[0138] Example 4

[0139] This invention discloses a computer storage medium storing computer instructions. When these computer instructions are invoked, they are used to execute some or all of the steps in a load-aware adaptive control method for database archiving disclosed in Embodiment 1 of this invention.

[0140] Example 5

[0141] This invention discloses a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to perform the steps of a load-aware database archiving adaptive control method described in Embodiment 1.

[0142] The system embodiments described above are merely illustrative. The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0143] Through the detailed description of the above embodiments, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, including read-only memory (ROM), random access memory (RAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), one-time programmable read-only memory (OTPROM), electrically-Erasable Programmable Read-Only Memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, disk storage, magnetic tape storage, or any other computer-readable medium that can be used to carry or store data.

[0144] Finally, it should be noted that the adaptive control method and system for database archiving based on load awareness disclosed in the embodiments of the present invention are merely preferred embodiments of the present invention and are only used to illustrate the technical solutions of the present invention, not to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A load-aware adaptive control method for database archiving, characterized in that, The method includes: Submit a standardized archiving application, enforce validation of archiving parameters and structured query language statements, support the reuse of historical templates, and generate validated archiving tasks; Perform a pre-execution evaluation on the verified archiving task to generate an evaluation report that includes the estimated number of rows affected and the execution time; Based on the database, the system load status dynamically identifies low-peak windows and schedules the execution of the verified archiving tasks within the identified low-peak windows. During task execution, the database performance metrics are monitored in real time, and the task execution rate is adaptively adjusted or the task is terminated based on the monitoring results. The entire lifecycle status of archived tasks is tracked, fed back, and recorded to form a closed-loop management system.

2. The load-aware adaptive control method for database archiving according to claim 1, characterized in that, The submission of standardized archiving requests, mandatory validation of archiving parameters and structured query language statements, support for the reuse of historical templates, and generation of validated archiving tasks include: Receive an archiving request submitted by a user, the request including the target database, target table, and structured query language statements for archiving conditions; The archived conditional structured query language statement is parsed using a parser to generate an abstract syntax tree; Traverse the abstract syntax tree to verify whether the statement type is an allowed operation type, whether the target table is in the pre-configured whitelist, whether the condition field is an allowed archive field, and identify dangerous condition patterns; After verification, or after the user selects a historical template and fills in the parameters and verification is passed, the verified archive task is generated.

3. The load-aware adaptive control method for database archiving according to claim 1, characterized in that, The pre-execution evaluation of the verified archiving task, generating an evaluation report including the estimated number of rows affected and execution time, includes: Perform interpretive operations of the verified archiving task in a test environment or on a read-only slave; The estimated number of rows affected by the archive task can be obtained using the execution plan analysis command provided by the database. Estimate the execution time of the archiving task by executing analysis commands or conducting small-scale trial executions; Summarize the analysis results and generate the aforementioned assessment report, which includes the number of rows affected, estimated time taken, and recommendations for index usage.

4. The load-aware adaptive control method for database archiving according to claim 1, characterized in that, The database-based dynamic identification of off-peak periods for system load status includes: Continuously collect the number of database queries executed per second and the CPU utilization as load indicators; Compare the current load metrics with the historical load baseline for the same period; When the current load index is lower than the historical load baseline for the same period by a certain percentage and the absolute value is lower than the preset safety threshold, the current period is determined to be the off-peak window period.

5. The load-aware adaptive control method for database archiving according to claim 1, characterized in that, The real-time monitoring of database performance metrics, and the adaptive adjustment of task execution rate or termination of tasks based on monitoring results, includes: Monitor database CPU utilization and lock wait rate during task execution; When the CPU utilization or lock wait rate exceeds a first threshold, the task execution rate is reduced by increasing the sleep interval between batch processing. When the CPU utilization or lock wait rate exceeds a second threshold higher than the first threshold, the database connection is forcibly terminated to stop the archiving task.

6. The load-aware adaptive control method for database archiving according to claim 1, characterized in that, The tracking, feedback, and recording of the entire lifecycle status of archiving tasks includes: The status of archived tasks is updated and displayed in real time, including statuses such as pending review, pending execution, in progress, successful execution, failed execution, and terminated. When an exception occurs during task execution, a retry or rollback operation is automatically performed; The entire process of archiving tasks, from application, review, pre-execution to execution and monitoring, is stored in a structured manner.

7. The load-aware adaptive control method for database archiving according to claim 2, characterized in that, The historical templates are stored in a structured data format, including a structured query language statement skeleton and configurable parameter definitions; After the user selects a template and fills in the parameters, a new structured query language statement is generated by the parameterized template engine. This newly generated statement must go through the mandatory validation process again.

8. The load-aware adaptive control method for database archiving according to claim 1, characterized in that, The method enables the submission of standardized archiving applications, the display of evaluation reports, real-time feedback on task status, and the query of the entire process log through a graphical user interface.

9. A load-aware adaptive control system for database archiving, used to implement the load-aware adaptive control method for database archiving as described in any one of claims 1-8, characterized in that, The system includes: The verification module is used to receive and process standardized archiving requests, perform mandatory verification of archiving parameters and structured query language statements, support the reuse of historical templates, and generate verified archiving tasks. The evaluation module is used to perform a pre-execution evaluation on the verified archiving task and generate an evaluation report that includes the estimated number of rows affected and the execution time. The scheduling module is used to dynamically identify low-peak windows in the database's system load status and schedule the execution of the verified archiving tasks within the identified low-peak windows. The control module is used to monitor the database's performance metrics in real time during task execution and adaptively adjust the task execution rate or terminate the task based on the monitoring results. The processing module is used to track, provide feedback on, and record the entire lifecycle status of archived tasks, forming a closed-loop management system.

10. A load-aware adaptive control system for database archiving, characterized in that, The system includes: Memory containing executable program code; A processor coupled to the memory; The processor calls the executable program code stored in the memory to execute a load-aware database archiving adaptive control method as described in any one of claims 1-8.

Images