An artificial intelligence-based database dynamic monitoring method and system

Abstract

The application discloses a kind of based on artificial intelligence's database dynamic monitoring method and system, the method specifically includes: through data smoothing algorithm, the performance time series data and its associated log, slow query and lock competition information are handled, obtain clean data;Based on clean data, through the threshold prediction model based on federal learning framework, generate dynamic alarm threshold for database instance;Real-time monitoring clean data, when clean data exceeds dynamic alarm threshold, trigger alarm event;In response to alarm event, automatically associate analysis corresponding log, slow query and lock competition information to locate abnormal root cause, generate repair suggestion;Quantitative evaluation alarm event influence degree on preset business index, according to influence degree, alarm service is prioritized and pushed.The application can effectively overcome the limitations of static threshold monitoring method, improve the accuracy, timeliness and effectiveness of database monitoring, guarantee the stable operation of database.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a method and system for dynamic monitoring of databases based on artificial intelligence. Background Technology

[0002] In the field of database operation and maintenance management, the stable operation of databases is crucial for the normal operation of various business systems. To ensure database stability, database monitoring is an indispensable key link, which can monitor the database's operating status in real time, promptly detect potential problems and trigger alarms, so that operation and maintenance personnel can take timely measures to deal with them.

[0003] Currently, static monitoring thresholds are commonly used to monitor the operational status of databases. Static monitoring thresholds refer to pre-setting a fixed alarm threshold for all database instances, such as triggering an alarm when CPU utilization reaches 90%. When a performance metric of a database instance reaches or exceeds this preset threshold, the system will issue an alarm message.

[0004] However, this static threshold setting method has revealed many significant limitations in practical applications, seriously affecting the effectiveness and timeliness of database monitoring, and consequently having an adverse impact on the stable operation of the database. Specifically, these limitations are reflected in the following aspects: 1. Inability to adapt to the actual usage of different database instances: Different database instances vary in their normal performance fluctuation range due to differences in the types of business they support, the volume of business, and their operating environments. For example, some database instances may normally have CPU utilization far below 90%, only 30%, while others may have higher CPU utilization, such as 70%, due to their business characteristics. For the former, when CPU utilization suddenly rises to 60%, although the absolute value does not reach the uniform threshold of 90%, this sudden increase in traffic has actually doubled, which is an abnormal situation; while for the latter, 70% CPU utilization may still be within the normal range. Static threshold settings cannot be flexibly adjusted according to these actual differences between different database instances. This results in some database instances failing to trigger alarms in a timely manner when abnormal situations occur, while other database instances may frequently trigger false alarms due to normal fluctuations, increasing the workload of operations and maintenance personnel and reducing the reliability of alarm information.

[0005] 2. Inability to effectively monitor and alert on sudden traffic spikes that do not reach a unified threshold: In actual operation, databases may encounter various sudden situations, such as peak business periods or malicious attacks, causing certain performance indicators to rise sharply in a short period of time, but not yet reaching the unified preset static threshold. For example, the database query response time is 100ms under normal circumstances, but may quickly rise to 500ms under sudden traffic spikes. However, if the unified preset alarm threshold is 1000ms, the system will not trigger an alarm. Nevertheless, this significant change in query response time has already had a substantial impact on database performance, potentially leading to slower response times in business systems and a degraded user experience. Static threshold settings cannot capture these sudden traffic changes that do not reach the unified threshold but have already affected database performance, thus making it impossible to take timely measures to ensure database stability.

[0006] 3. Lack of a mechanism for dynamically adjusting alarm thresholds: The operating status of a database is not static but dynamically changes with factors such as business development, changes in the system environment, and the database's own usage. Static thresholds, once set, remain fixed and cannot be flexibly adjusted according to these dynamic changes during actual database operation. For example, as business volume increases, the database's CPU utilization may gradually rise, and the originally set 90% threshold may no longer be applicable, requiring an appropriate increase to avoid frequent false alarms. Alternatively, during certain special periods, such as system maintenance or off-seasons, the fluctuation range of database performance indicators may change, necessitating corresponding adjustments to the alarm thresholds. The lack of a dynamic adjustment mechanism means that static thresholds cannot always accurately reflect the actual operating status of the database, making it difficult to meet the real-time and accuracy requirements of database monitoring. Summary of the Invention

[0007] The purpose of this invention is to provide a database dynamic monitoring method and system based on artificial intelligence, which can effectively overcome the limitations of static threshold monitoring methods, improve the accuracy, timeliness and effectiveness of database monitoring, and ensure the stable operation of the database, thereby solving at least one of the aforementioned problems in the prior art.

[0008] In a first aspect, the present invention provides a database dynamic monitoring method based on artificial intelligence, the method specifically comprising: Obtain performance time-series data of the database instance, dynamically adjust the sampling frequency based on the load fluctuation characteristics of the performance time-series data, and synchronously collect logs, slow queries and lock contention information associated with the performance time-series data; The data smoothing algorithm is used to process the performance time series data and its associated logs, slow queries and lock contention information to obtain clean data after noise reduction; Based on clean data, a threshold prediction model based on a federated learning framework is used to generate dynamic alarm thresholds for database instances. Monitor cleanliness data in real time and trigger an alarm event when the cleanliness data exceeds the dynamic alarm threshold; In response to alarm events, the system automatically correlates and analyzes the corresponding logs, slow queries, and lock contention information to locate the root cause of the anomaly and generate remediation suggestions. The impact of alarm events on preset business indicators is quantitatively assessed using a causal inference model, and alarm services are prioritized and pushed according to the degree of impact.

[0009] Secondly, the present invention provides a database dynamic monitoring system based on artificial intelligence, the system specifically comprising: The data acquisition module is used to acquire performance time-series data of the database instance, dynamically adjust the sampling frequency based on the load fluctuation characteristics of the performance time-series data, and synchronously collect logs, slow queries and lock contention information associated with the performance time-series data. The data denoising module is used to process performance time-series data and its associated logs, slow queries and lock contention information through data smoothing algorithms to obtain clean data after denoising. The threshold prediction module is used to generate dynamic alarm thresholds for database instances based on clean data and through a threshold prediction model based on a federated learning framework. The alarm triggering module is used to monitor clean data in real time and trigger an alarm event when the clean data exceeds the dynamic alarm threshold. The repair suggestion module is used to respond to alarm events, automatically correlate and analyze the corresponding logs, slow queries and lock contention information to locate the root cause of the anomaly, and generate repair suggestions; The business evaluation module is used to quantify the impact of alarm events on preset business indicators through a causal inference model, and to prioritize and push alarm services according to the degree of impact.

[0010] Thirdly, the present invention provides a computer device, comprising: a memory and a processor, and a computer program stored in the memory, wherein when the computer program is executed on the processor, it implements the artificial intelligence-based database dynamic monitoring method as described in any of the above methods.

[0011] Fourthly, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the AI-based dynamic database monitoring method as described in any of the above methods.

[0012] Compared with the prior art, the present invention has at least one of the following technical effects: 1. This invention can effectively overcome the limitations of static threshold monitoring methods, improve the accuracy, timeliness and effectiveness of database monitoring, and ensure the stable operation of the database.

[0013] 2. This invention can adjust the data acquisition frequency in real time according to the actual load of the database instance. When the load fluctuates significantly, the sampling frequency is increased to more accurately capture performance changes; when the load is stable, the sampling frequency is decreased to reduce resource consumption. This dynamic adjustment mechanism makes the monitoring data more representative and real-time, providing a reliable foundation for subsequent accurate analysis and alerts.

[0014] 3. This invention utilizes the Kalman filter algorithm to adaptively smooth performance time-series data, and combines the evidence weights of associated events to perform weighted correction on data points, effectively removing noise and abnormal interference in the data, improving data quality, making the generated dynamic alarm threshold more accurate and reliable, and reducing the occurrence of false alarms and missed alarms.

[0015] 4. This invention fully considers the individual differences and time-series data characteristics of different database instances. It uses federated learning to train a model using data from multiple database instances, generating personalized dynamic alarm thresholds. These thresholds can automatically adjust according to the actual operation of the database, better adapting to the characteristics of different database instances, promptly detecting potential anomalies, and improving the timeliness and accuracy of alarms.

[0016] 5. When an alarm event occurs, this invention can quickly and automatically correlate and analyze relevant information, accurately locate the root cause event leading to the anomaly, and generate targeted repair suggestions based on the root cause type and attributes. This greatly shortens the time for troubleshooting and repair, improves operational efficiency, and reduces the impact of database anomalies on business systems.

[0017] 6. This invention can accurately quantify the impact of alarm events on business indicators, enabling operation and maintenance personnel to clearly understand the importance and urgency of different alarm events, thereby rationally arranging the processing order, prioritizing the handling of alarm events with greater impact on business, improving the pertinence and effectiveness of database operation and maintenance, and ensuring the stable operation of business systems.

[0018] 7. This invention dynamically adjusts the sampling frequency based on load fluctuation characteristics, which can more accurately adapt to changes in database load, improve the rationality and effectiveness of data collection, and optimize monitoring results.

[0019] 8. This invention utilizes the Kalman filter algorithm to adaptively smooth performance time series data, which can dynamically adjust according to the short-term statistical characteristics of the data, thereby improving the accuracy and adaptability of data smoothing.

[0020] 9. This invention generates dynamic alarm thresholds based on a federated learning framework, which can achieve personalized and accurate alarm threshold settings while protecting data privacy.

[0021] 10. This invention initializes the threshold prediction global model based on the characteristics of database instances, thereby improving the applicability and initial accuracy of the model to different database instances.

[0022] 11. This invention automatically correlates and analyzes alarm event-related information to locate the root cause and generate repair suggestions, quickly and accurately resolving database anomaly issues and reducing manual analysis costs.

[0023] 12. This invention uses a causal inference model to quantitatively assess the impact of alarm events and prioritize their delivery, helping to prioritize alarms that have a significant impact on business operations and ensuring stable business operation. Attached Figure Description

[0024] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 This is a flowchart illustrating an artificial intelligence-based dynamic database monitoring method according to an embodiment of the present invention. Figure 2 This is a schematic diagram of the structure of a database dynamic monitoring system based on artificial intelligence, provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the structure of a computer device provided in an embodiment of the present invention. Detailed Implementation

[0026] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0027] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0028] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0029] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0030] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0031] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0032] In this application embodiment, the entity executing the process includes a terminal device. This terminal device includes, but is not limited to, devices capable of executing the methods disclosed in this application, such as servers, computers, smartphones, and tablets. Figure 1 A flowchart illustrating an embodiment of the artificial intelligence-based dynamic database monitoring method disclosed in this invention is shown below: S101: Obtain the performance time-series data of the database instance, dynamically adjust the sampling frequency based on the load fluctuation characteristics of the performance time-series data, and synchronously collect logs, slow queries and lock contention information associated with the performance time-series data. S102, the data smoothing algorithm is used to process the performance time series data and its associated logs, slow queries and lock contention information to obtain clean data after noise removal; S103, based on clean data, generates dynamic alarm thresholds for database instances through a threshold prediction model based on a federated learning framework. S104, monitors cleanroom data in real time and triggers an alarm event when the cleanroom data exceeds the dynamic alarm threshold; S105, in response to alarm events, automatically correlates and analyzes the corresponding logs, slow queries and lock contention information to locate the root cause of the anomaly and generate remediation suggestions; S106 uses a causal inference model to quantify the impact of alarm events on preset business indicators, and prioritizes and pushes alarm services according to the degree of impact.

[0033] In this embodiment, a data acquisition agent deployed on the database server continuously acquires performance time-series data of the database instance. This performance time-series data covers key performance indicators such as CPU utilization, memory utilization, disk I / O read / write speed, and query response time. Considering the differences in load fluctuation characteristics among different database instances—for example, some database instances experience a sharp increase in load during peak business periods and a significant decrease in load during off-peak periods—the system dynamically adjusts the sampling frequency based on the load fluctuation characteristics of the performance time-series data to more accurately capture these dynamic changes. When a large load fluctuation is detected, the sampling frequency is increased to obtain more detailed data changes; when the load is relatively stable, the sampling frequency is decreased to reduce data acquisition overhead. Simultaneously, logs, slow query information, and lock contention information associated with the performance time-series data are collected synchronously. Log information records various database operations and events, slow query information reflects query statements with long execution times, and lock contention information reflects the acquisition and release of locks in the database. This information is crucial for subsequent analysis of database performance problems and the root causes of anomalies.

[0034] The collected performance time-series data, along with associated logs, slow query data, and lock contention information, are input into the data smoothing algorithm module. The main function of the data smoothing algorithm is to denoise the raw data, eliminating random fluctuations and outliers to obtain clean, denoised data. For example, in performance time-series data such as CPU utilization, there may be sudden spikes in data due to instantaneous network fluctuations or temporary system tasks. The data smoothing algorithm uses certain calculation methods, such as moving averages or exponential smoothing, to correct these outliers, making the data smoother and more accurate. This provides a reliable data foundation for subsequent threshold prediction and anomaly detection.

[0035] A threshold prediction model based on a federated learning framework is used to analyze and process denoised clean data, generating dynamic alarm thresholds for each database instance. The federated learning framework allows the sharing of model training knowledge among different database instances while protecting the data privacy of each instance. The model comprehensively considers factors such as the database instance's historical performance data, current operating status, and business characteristics, using machine learning algorithms to learn a dynamic alarm threshold suitable for that instance. For example, for database instances with high business volume and frequent load fluctuations, the model generates relatively flexible thresholds to adapt to their dynamically changing performance needs; while for database instances with relatively stable business and small load changes, the model generates relatively stable thresholds. This results in dynamic alarm thresholds that more accurately reflect the actual operating status of the database instance, improving the accuracy and timeliness of alarms.

[0036] After generating dynamic alarm thresholds, the system monitors the denoised clean data in real time. A dedicated monitoring module continuously compares the clean data with the dynamic alarm thresholds. Once a performance indicator's clean data exceeds the corresponding dynamic alarm threshold, the monitoring module immediately triggers an alarm event. Alarm events can be notified to operations and maintenance personnel through various methods, such as SMS, email, and instant messaging tools, ensuring that they are promptly informed of any database anomalies.

[0037] When an alarm event is triggered, the system automatically correlates and analyzes the corresponding logs, slow queries, and lock contention information. By analyzing operation records and event information in the logs, it identifies operations that may cause performance anomalies; it performs detailed analysis of slow query information to identify queries with excessively long execution times and analyzes their execution plans and index usage; simultaneously, it checks lock contention information to determine if there are data access blockages caused by lock conflicts. Based on these analysis results, the system generates a detailed root cause report and provides corresponding remediation suggestions for different anomalies. For example, if performance degradation is caused by a query's improper use of indexes, the remediation suggestion might be to create appropriate indexes for the fields involved in the query; if data access blockage is caused by lock contention, the remediation suggestion might be to optimize transaction processing logic and reduce lock holding time.

[0038] To help operations and maintenance (O&M) personnel better handle alarm events, the system uses a causal inference model to quantify the impact of each alarm event on preset business metrics. These preset business metrics can be defined based on actual business needs, such as system response time, transaction success rate, and user activity. The causal inference model analyzes the causal relationship between alarm events and business metrics, calculating the specific impact of alarm events on these metrics through learning from historical and real-time data. Based on the magnitude of the impact, the system prioritizes alarms, pushing higher-impact alarm events to O&M personnel first. This ensures that O&M personnel can address the issues with the greatest impact on business operations first, improving fault handling efficiency and business recovery speed.

[0039] This embodiment enables real-time and accurate monitoring of the database, dynamic adjustment of alarm thresholds, timely detection and handling of database performance anomalies, ensuring stable database operation, and improving the work efficiency of maintenance personnel and the reliability of alarm information.

[0040] In some embodiments, step S101 above, which involves dynamically adjusting the sampling frequency based on the load fluctuation characteristics of performance time-series data, specifically includes: Based on performance time-series data, incremental sliding volatility is calculated for each performance indicator to obtain a real-time volatility quantification value for each performance indicator. Based on historical anomaly contribution and business importance, a corresponding performance weight coefficient is assigned to each performance indicator. The real-time fluctuation quantification values ​​of each performance indicator are then merged using the performance weight coefficient to generate a comprehensive fluctuation score that characterizes the overall load fluctuation status of the database instance. The system compares the comprehensive fluctuation score with the fluctuation threshold range in real time, and dynamically adjusts the sampling frequency of the next sampling period based on the comparison results and the preset multi-level frequency adjustment strategy. Analyze the monitoring effect data after the sampling frequency is adjusted, and dynamically optimize the boundary value of the fluctuation threshold range corresponding to the comprehensive fluctuation score based on the monitoring effect data.

[0041] In this embodiment, during database operation, time-series performance data for various performance metrics are continuously acquired. These metrics include CPU utilization, memory utilization, disk I / O read / write speed, and query response time. For each performance metric, an incremental sliding volatility calculation method is employed. Specifically, a sliding window is defined, containing a certain number of data points sequentially over time. Upon the arrival of each new data point, the volatility of the data within the current window is calculated. For example, for CPU utilization, the difference between the maximum and minimum values ​​within the window is calculated, or the average of the absolute values ​​of the differences between adjacent data points is calculated, thereby obtaining a real-time quantified value of the performance metric's volatility at the current moment. This method allows for timely capture of short-term fluctuations in each performance metric, providing foundational data for subsequent comprehensive analysis.

[0042] Considering the varying degrees of impact different performance metrics have on the overall database load, a corresponding performance weight coefficient is assigned to each performance metric based on its historical anomaly contribution and business importance. Historical anomaly contribution refers to the frequency and severity of database anomalies caused by the performance metric in the past. For example, if a performance metric has frequently caused database performance degradation or even failures in the past, its historical anomaly contribution is high and should be assigned a larger weight coefficient. Business importance is determined by how critical the performance metric is to the normal operation of the business system. For example, query response time directly affects the user experience and is crucial to the business system; therefore, its business importance is high and should also be assigned a larger weight coefficient.

[0043] After assigning performance weight coefficients to each performance metric, the real-time fluctuation metric of each performance metric is multiplied by its corresponding performance weight coefficient. Then, the products of all performance metrics are summed to generate a comprehensive fluctuation score that characterizes the overall load fluctuation status of the database instance. This comprehensive fluctuation score takes into account the fluctuation of each performance metric and their impact on the overall database load, thus more accurately reflecting the current load fluctuation status of the database instance.

[0044] The system compares the overall fluctuation score with a pre-defined fluctuation threshold range in real time. The fluctuation threshold range is set based on historical database data and business requirements; for example, it can be divided into low-fluctuation, medium-fluctuation, and high-fluctuation ranges. Based on the different ranges in which the overall fluctuation score falls, and in conjunction with a pre-defined multi-level frequency adjustment strategy, the sampling frequency for the next sampling period is dynamically adjusted.

[0045] Multi-level frequency adjustment strategies can be flexibly configured according to actual conditions. For example, when the overall fluctuation score is in the low fluctuation range, it indicates that the database instance load is relatively stable. At this time, the sampling frequency can be appropriately reduced to reduce the overhead of data collection and processing. When the overall fluctuation score is in the medium fluctuation range, the current sampling frequency should be kept unchanged, and the database load changes should continue to be closely monitored. When the overall fluctuation score is in the high fluctuation range, it indicates that the database instance load fluctuates greatly and there may be potential performance problems. At this time, the sampling frequency should be increased to obtain database performance data more timely and accurately, providing more detailed information for subsequent anomaly detection and handling.

[0046] After adjusting the sampling frequency, continuously analyze the monitoring performance data. This data includes, but is not limited to, the accuracy of anomaly detection, the timeliness of alarms, and the completeness of data collection. Analyzing this data helps assess the appropriateness of the current fluctuation threshold range setting.

[0047] If it is found that unreasonable boundary values ​​of the fluctuation threshold range lead to excessively frequent or untimely adjustments to the sampling frequency, affecting the effectiveness of database monitoring, then the boundary values ​​of the fluctuation threshold range corresponding to the comprehensive fluctuation score should be dynamically optimized based on monitoring performance data. For example, if the comprehensive fluctuation score frequently falls within the low fluctuation range but the database exhibits anomalies, it indicates that the upper limit of the low fluctuation range is set too high and should be appropriately lowered. Conversely, if the comprehensive fluctuation score falls within the high fluctuation range but the database operates normally, it indicates that the lower limit of the high fluctuation range is set too low and should be appropriately increased. By continuously and dynamically optimizing the boundary values ​​of the fluctuation threshold range, the adjustment of the sampling frequency becomes more reasonable and accurate, thereby improving the efficiency and accuracy of database monitoring.

[0048] This embodiment can dynamically adjust the sampling frequency based on the load fluctuation characteristics of performance time series data, making data collection more adaptable to the actual operation of the database instance and providing more accurate and timely data support for subsequent dynamic monitoring of the database.

[0049] In some embodiments, step S102 above, which involves processing the performance time-series data and its associated logs, slow queries, and lock contention information using a data smoothing algorithm to obtain denoised clean data, specifically includes: The Kalman filter algorithm is used to adaptively smooth the performance time series data to obtain a pre-smoothed performance data sequence. The associated logs, slow queries, and lock contention information are parsed and marked to extract the occurrence time and event type of the associated events; Based on the preset event type-weight mapping relationship, different evidence weights are assigned to related events of different event types; Based on the occurrence time of the associated events, the corresponding data points are located in the performance data sequence. Then, according to the event type of the associated events and the evidence weight, the smoothing result of the data points is weighted and corrected to generate the final clean data.

[0050] In this embodiment, after acquiring the performance time-series data of the database instance, an adaptive smoothing process is performed using the Kalman filter algorithm. The Kalman filter algorithm is an algorithm capable of optimally estimating the system state based on system input and output observation data. In the scenario of database performance time-series data processing, it can adaptively adjust the smoothing parameters based on historical performance data and currently collected performance data to adapt to the fluctuation characteristics of performance data at different times. For example, when the database is in a peak business period and the performance data fluctuates significantly, the Kalman filter algorithm automatically adjusts the parameters so that the smoothed data reflects both the overall trend of the data and retains a certain degree of fluctuation details, thereby obtaining a preliminarily smoothed performance data sequence. This step effectively removes high-frequency noise from the performance time-series data, making the data smoother and more stable, providing a more reliable foundation for subsequent processing.

[0051] Logs, slow queries, and lock contention information associated with performance time-series data are parsed and labeled. Logs record various events and operations during database operation, slow query information reflects queries with long execution times, and lock contention information reflects the contention for lock resources in the database. Using specific parsing rules, the occurrence time and event type of related events are extracted from this information. For example, the time point and type of a specific error event are identified from the logs; the start and end times of slow query statements and their types (e.g., complex queries, large data queries) are determined from the slow query information; and the start and end times of lock contention and its types (e.g., table lock contention, row lock contention) are identified from the lock contention information. The extraction of these key elements provides an important basis for subsequent data correction.

[0052] Based on a pre-defined event type-weight mapping, different evidence weights are assigned to related events of different event types. This mapping is pre-defined based on database operation and maintenance experience and business needs. Different types of related events have varying degrees of impact on database performance, thus requiring different weights. For example, severe error events may have a significant impact on database performance and should be assigned higher evidence weights; while minor warning events have a smaller impact on performance and can be assigned lower evidence weights. For slow query events, if the query involves processing a large amount of data, it may put significant pressure on database performance and should be given higher weights; while for simple slow queries, the weights can be appropriately reduced. In lock contention events, prolonged lock contention can cause database operation blocking and affect performance; therefore, appropriate weights should be assigned based on the intensity of lock contention. By reasonably allocating evidence weights, the degree of impact of different related events on database performance can be accurately reflected.

[0053] Based on the occurrence time of related events, corresponding data points are located in the initially smoothed performance data sequence. For example, if a related event occurs at a specific time point, the data point corresponding to that time point is found in the performance data sequence. Then, according to the event type of the related event and the previously assigned evidence weights, the smoothing result of that data point is weighted and corrected. If the evidence weight corresponding to the event type of the related event is high, it indicates that the event has a significant impact on database performance, so the value of that data point will be adjusted more significantly during correction; conversely, if the evidence weight is low, the adjustment will be smaller. In this way, the information of related events is integrated into the performance data, further optimizing the initially smoothed performance data and generating the final clean data. This clean data not only removes noise but also considers the impact of various performance-related events, enabling it to more accurately reflect the actual operating status of the database and providing reliable data support for subsequent dynamic alarm threshold generation and anomaly detection.

[0054] Furthermore, the step of adaptively smoothing the performance time-series data using the Kalman filter algorithm to obtain a preliminarily smoothed performance data sequence specifically includes: A Kalman filter state-space model is established to describe the state-observation relationship of the performance index, and the parameters of the Kalman filter state-space model are initialized. Real-time analysis of the short-term statistical characteristics of the input performance time-series observation data, and dynamic generation of observation noise level parameters based on the analysis results of the short-term statistical characteristics; In each iteration of the Kalman filter, dynamically generated observation noise level parameters are injected, and the adaptive Kalman gain is calculated in real time based on the observation noise level parameters and the current state prediction error. The actual observation value at the current moment is obtained, and the residual between the actual observation value and the state prediction value is weighted and corrected using adaptive Kalman gain. The optimal state estimate at the current moment is then updated to form a preliminary smoothed performance data sequence.

[0055] In this embodiment, a Kalman filter state-space model is constructed. This model aims to clearly describe the relationship between the actual state of database performance metrics and the data we observe. For example, the actual state of a database's CPU utilization may be influenced by various factors, while the observed data we collect is only an approximation of this actual state. By establishing this model, we can clarify the intrinsic connection between the actual state and the observed data. After completing the model construction, the parameters in the model need to be initialized. These parameters include the state transition matrix, observation matrix, etc. The initial values ​​can be determined based on the database's historical operating data and relevant experience, laying the foundation for subsequent Kalman filter processing.

[0056] In Kalman filtering, the observation noise level parameter is a crucial factor, reflecting the magnitude of noise present in the observed data. Since the performance time-series observation data of the database changes in real time, its short-term statistical characteristics also change accordingly. For example, during peak business periods, database performance data fluctuates significantly, and observation noise may be relatively high; while during off-peak periods, the data is relatively stable, and observation noise is low. Therefore, we need to analyze the short-term statistical characteristics of the input performance time-series observation data in real time, such as calculating statistics like variance within a certain time window. Based on the analysis results of these short-term statistical characteristics, an observation noise level parameter adapted to the current data conditions is dynamically generated. This ensures that the noise situation of the observation data at different times is fully considered in subsequent filtering processes, improving the accuracy of filtering.

[0057] Kalman gain is a crucial parameter in the Kalman filter algorithm, determining the degree to which observation data is utilized during state updates. In each iteration of the Kalman filter, a dynamically generated observation noise level parameter is incorporated into the calculation. Simultaneously, the adaptive Kalman gain is calculated in real-time, taking into account the current state prediction error—the difference between the actual observation and the state prediction based on the model. For example, when the observation noise level is high, indicating lower reliability of the observation data, the Kalman gain decreases accordingly to reduce the impact of the observation data on state updates; conversely, when the observation noise level is low, the Kalman gain increases to more fully utilize the observation data for state updates. In this way, the Kalman gain can adaptively adjust according to the actual situation, making the filtering process more flexible and accurate.

[0058] We obtain the actual observed values ​​of the database performance metrics at the current moment, and use the previously calculated adaptive Kalman gain to perform weighted correction on the residuals between the actual observed values ​​and the predicted state values. These residuals reflect the deviation between the actual observed values ​​and the model predictions; weighted correction can eliminate some of the influence of noise and error. After correction, we update the optimal state estimate for the current moment, which more accurately reflects the database performance metrics at that moment. We perform the above processing on the performance data at each moment in chronological order, ultimately forming a preliminarily smoothed performance data sequence. This sequence is smoother and more stable than the original observed data, removing most of the noise interference and providing a reliable data foundation for subsequent database monitoring and analysis.

[0059] Furthermore, the process of parsing and marking related logs, slow queries, and lock contention information to extract the occurrence time and event type of related events specifically includes: Based on the associated log stream, a pre-configured regular expression pattern is used to match with a keyword dictionary to capture error codes, description text, and related database object identifiers, forming a first initial field set; Based on the associated slow query list, the operation type, core table names involved and index features are extracted by parsing the query statement to form a second initial field set; Based on the associated lock contention states, a resource waiting relationship graph is constructed by analyzing the lock view to identify blocking chains and related session and resource identifiers, forming a third set of initial fields; Map the semantic content of events in the first initial field set, the second initial field set, and the third initial field set into structured associated event feature vectors; Based on the feature vectors of related events, the related events are classified into a multi-level event classification tree, and timestamps and type codes are assigned to the related events in combination with preset priority rules.

[0060] In this embodiment, regular expression patterns and keyword dictionaries are pre-configured. These configurations are derived from the analysis and summarization of a large number of database logs and can accurately match key information in the logs. When processing the log stream, the log content is matched one by one against the pre-configured regular expression patterns and keyword dictionary. Through this matching method, error codes in the logs can be captured. These error codes can intuitively reflect the abnormal situations that occur during database operation. At the same time, descriptive text can also be obtained, which details the specific manifestation of the abnormality and related background information. In addition, relevant database object identifiers, such as table names and column names, can be extracted, which helps to locate the specific database object where the problem occurs. The captured information is integrated to form a first initial field set, providing basic data for subsequent event analysis.

[0061] The associated slow query list is processed. Slow queries are those whose execution time exceeds a preset threshold, and they often have a significant impact on database performance. By parsing the queries in the slow query list, their structure is analyzed in depth. Operation types can be extracted from the query statements, such as SELECT, INSERT, UPDATE, and DELETE, and different types of operations have varying degrees of impact on database performance. The names of the core tables involved can also be obtained, clarifying the main data tables targeted by the query operations. Furthermore, index features can be extracted to understand whether the query utilizes appropriate indexes, which is crucial for optimizing query performance. This extracted information is summarized to form a second initial field set, providing a basis for analyzing the impact of slow queries on database performance.

[0062] Handling related lock contention states. Lock contention is a common performance issue in databases, occurring when multiple sessions simultaneously request operations on the same resource. By analyzing the lock view in the database, a resource wait relationship graph is constructed. This graph clearly shows the wait relationships between various resources, thus identifying blocking chains. A blocking chain refers to a series of session blocking phenomena caused by lock contention; identifying the blocking chain helps find the root cause of the problem. Simultaneously, it identifies the relevant session and resource identifiers, understanding which specific session is waiting for which resource. This information is then organized into a third initial field set, providing crucial information for resolving lock contention issues.

[0063] After obtaining the first, second, and third initial field sets, to facilitate subsequent unified processing and analysis, the semantic content of events in these sets is mapped into structured associated event feature vectors. These structured feature vectors can standardize the representation of different types of event information, enabling computers to process and analyze this information more efficiently.

[0064] Finally, based on the feature vectors of related events, the related events are categorized into a multi-level event classification tree. The multi-level event classification tree is designed hierarchically according to factors such as the nature and scope of impact of the events, enabling the orderly classification of various related events. During the classification process, a timestamp and type code are assigned to each related event based on preset priority rules. The timestamp accurately records the time when the event occurred, while the type code clarifies the type of event, such as a log error event, a slow query event, or a lock contention event. In this way, the parsing and labeling of related log, slow query, and lock contention information are completed, extracting the occurrence time and event type of related events, providing strong support for subsequent dynamic database monitoring and analysis.

[0065] In some embodiments, step S103 above, which involves generating dynamic alarm thresholds for database instances based on clean data using a threshold prediction model based on a federated learning framework, specifically includes: A threshold prediction global model is initialized based on the federated learning coordination server, and the initial parameters of the threshold prediction global model are distributed to the local agents of each database instance. Each local agent uses its corresponding local clean data to incrementally train the global threshold prediction model and obtain updated local model parameters. The updated local model parameters of each local agent are uploaded to the federated learning coordination server. The federated learning coordination server performs secure aggregation of local model parameters from multiple database instances to generate new global model parameters and distribute them to each local agent. Each local agent performs forward inference calculations based on the newly issued global model parameters and its own local clean data to generate personalized dynamic alarm thresholds for its respective database instance.

[0066] In this embodiment, the federated learning coordination server plays a crucial coordinating role in the overall architecture, initializing a global threshold prediction model. This global model serves as the foundational framework for generating dynamic alarm thresholds, possessing a degree of versatility and scalability to adapt to the basic characteristics of different database instances. After initialization, the federated learning coordination server distributes the initial parameters of the global threshold prediction model to the local agents of each database instance. The local agent is a program module deployed on each database instance, responsible for interacting with the federated learning coordination server and utilizing local data for model training and threshold generation. This ensures that each database instance receives the initial information of the global model, preparing it for subsequent local training.

[0067] Each local agent uses its corresponding local clean data for incremental training. Each database instance has its unique operating environment and business characteristics, resulting in different clean data. Based on the initial parameters of the global model, the local agent incrementally trains the global threshold prediction model using its local clean data. Incremental training is a method of gradually optimizing the model; it fine-tunes the model parameters according to the characteristics of the local data, making the model better adaptable to the local database instance. Through this training method, each local agent obtains updated local model parameters that reflect specific information about the local database instance, providing a basis for generating personalized dynamic alarm thresholds.

[0068] Each local agent uploads its updated local model parameters to the federated learning coordination server. Upon receiving the local model parameters from multiple database instances, the federated learning coordination server performs a secure aggregation operation. Secure aggregation is a crucial technique in federated learning, enabling the fusion of multiple local model parameters while protecting the data privacy of each database instance. Through secure aggregation, the federated learning coordination server can comprehensively consider information from different database instances to generate new global model parameters. This process fully leverages the data advantages of each database instance, allowing the global model to learn broader and more comprehensive features, thereby improving the model's accuracy and generalization ability. After generating the new global model parameters, the federated learning coordination server distributes them to each local agent for subsequent operations.

[0069] Each local agent performs forward inference calculations based on the newly issued global model parameters and its own local clean data. Forward inference calculation is the process of calculating the output result based on the model parameters and input data. In this process, the local agent uses its local clean data as input and performs calculations using the new global model parameters to generate personalized dynamic alarm thresholds for its respective database instance. Because the local data and operating conditions of each database instance are different, the generated dynamic alarm thresholds are also personalized, which can more accurately reflect the actual operating status of the database instance. This personalized dynamic alarm threshold can better adapt to the differences between different database instances, improve the effectiveness and timeliness of database monitoring, promptly detect potential problems and trigger alarms, and ensure the stable operation of the database.

[0070] Furthermore, the initialization of a threshold prediction global model based on the federated learning coordination server specifically includes: Collect configuration and historical statistical metadata from multiple database instances, and perform aggregation analysis and instance clustering based on the configuration and historical statistical metadata to obtain clustering results; Based on the time-series data characteristics of the database instances of each group in the clustering results, the threshold prediction model structure suitable for different groups is determined and constructed. Based on the domain knowledge extracted from configuration and historical statistical metadata, differentiated initialization is performed on the parameters of the threshold prediction model structure, and prior constraints are injected into the objective function of the threshold prediction model structure. By using privacy-preserving statistical features or synthetic data obtained from various database instances, the initialized threshold prediction model structure is trained to obtain a global threshold prediction model.

[0071] In this embodiment, the federated learning coordination server proactively collects configuration and historical statistical metadata from multiple database instances. This metadata covers a wealth of key information about the database instances, such as database type, version, hardware configuration (e.g., number of CPU cores, memory size), and historical performance metrics (e.g., average CPU utilization, maximum query response time). After collecting this data, the federated learning coordination server uses specific aggregation analysis algorithms to perform in-depth analysis. By analyzing the similarities and differences in configuration and historical operational statistics among different database instances, database instances with similar characteristics are grouped into the same group, ultimately obtaining the clustering results. This clustering process lays the foundation for subsequently building appropriate threshold prediction models for different groups, because database instances in different groups often have different operating modes and performance characteristics, requiring differentiated treatment.

[0072] Based on the clustering results obtained above, the federated learning coordination server carefully studies the time-series data characteristics of database instances in each group. These characteristics include data fluctuation patterns, trend changes, and periodicity. For example, some groups of database instances may experience business peaks at specific times each day, resulting in significant periodic changes in their performance metrics; while other groups may exhibit more random fluctuations in performance metrics due to the specific nature of their business. Based on a deep understanding of these time-series data characteristics, the federated learning coordination server determines and constructs threshold prediction model structures suitable for different groups. Different groups may employ model structures of varying complexity and type. For instance, groups with more obvious time-series data patterns may use a model structure including a time-series analysis module; while groups with more complex data fluctuations may employ more complex model structures such as deep neural networks to ensure that the model can accurately capture the performance change patterns of database instances in each group.

[0073] The federated learning coordination server extracts domain knowledge from the collected configuration and historical statistical metadata. Domain knowledge comprises long-term accumulated experience and patterns in database operations and maintenance, such as the impact of different types of business on database performance metrics and the reasonable performance range of a database under different hardware configurations. Based on this domain knowledge, differentiated initialization is performed on the parameters of the previously constructed threshold prediction model structure. Differentiated initialization means assigning different initial values ​​to the model parameters according to the characteristics of different groups, making the model closer to the actual situation of each group from the outset. Simultaneously, prior constraints are injected into the objective function of the threshold prediction model structure. Prior constraints are restrictions on the model's output results set based on domain knowledge, such as stipulating that the thresholds of certain performance metrics must be within a reasonable physical range, or setting limits on the magnitude of threshold changes according to business needs. By injecting prior constraints, the model can be guided to optimize in a direction that conforms to actual needs during training, improving the model's accuracy and reliability.

[0074] To protect the privacy of database instance data, the federated learning coordination server does not directly use raw data for training. Instead, it utilizes privacy-preserving statistical features or synthetic data obtained from each database instance. Privacy preservation can employ various techniques, such as data encryption and differential privacy, to ensure that sensitive information of the database instances is not leaked during data transmission and use. The federated learning coordination server uses this processed data to train the initialized threshold prediction model structure. During training, the model continuously adjusts its parameters to minimize the error between the objective function and the actual data. After multiple rounds of training and optimization, the training process ends when the model's performance on the validation dataset reaches a preset standard. The resulting model is the global threshold prediction model. This global model integrates information from multiple database instances and considers the characteristics and prior knowledge of different groups, providing strong support for generating accurate dynamic alarm thresholds.

[0075] In some embodiments, step S105 above, which involves automatically associating and analyzing corresponding logs, slow queries, and lock contention information in response to an alarm event to locate the root cause of the anomaly and generate remediation suggestions, specifically includes: Extract the alarm trigger time from the alarm event, and search for the corresponding logs, slow queries and lock contention information within a preset time window with the alarm trigger time as the center. Construct a local event relationship network based on a predefined causal graph. The local event relationship network is matched and compared with a pre-built root cause pattern library to locate one or more potential root cause events that lead to alarm events. Based on the type and attributes of the identified potential root cause events, a targeted database repair action plan is generated by matching them with a pre-built semantic repair rule base. Enter the specific parameters from the database repair action plan into a natural language template to generate structured repair suggestions.

[0076] In this embodiment, when the system detects an alarm event, it immediately extracts the alarm trigger time from the event. This time is a critical point for locating problem-related information. A comprehensive search is then performed within a preset time window centered on this alarm trigger time. The size of the preset time window is set based on the characteristics of the database system and past experience; for example, it can be set to 5 minutes before and after the alarm trigger time. Within this time window, the system collects corresponding logs, slow queries, and lock contention information. This information contains detailed operational data of the database before and after the alarm occurred, which is crucial for analyzing the cause of the problem. After collecting this information, the system constructs a local event relationship network based on a predefined causal graph. The predefined causal graph is derived from the analysis and summary of numerous database failure cases and normal operation conditions, clarifying the causal relationships between different types of information. For example, specific error information in the logs may be causally related to certain long-running query statements in slow queries, and lock waiting in lock contention information may be related to certain specific operations or queries. Based on these causal relationships, the collected logs, slow queries, and lock contention information are constructed into a local event relationship network, clearly showing the correlation and impact between various events.

[0077] The constructed local event relationship network is matched and compared with a pre-built root cause pattern library. This library, compiled through long-term research and analysis of database failures, summarizes various potential root cause event patterns that may lead to alarm events. These patterns cover various abnormal situations that may occur during database operation, such as specific log error patterns, slow query characteristic patterns, and typical lock contention patterns. During the matching and comparison process, the system carefully analyzes each event and relationship in the local event relationship network, comparing it one by one with the patterns in the root cause pattern library. Through this comparison, one or more potential root cause events leading to the alarm event are identified. For example, if a situation appears in the local event relationship network that matches a specific lock contention pattern in the root cause pattern library, then lock contention can be identified as a potential root cause of this alarm.

[0078] Based on the type and attributes of the identified potential root cause events, the system performs matching using a pre-built semantic remediation rule base. This pre-built rule base is a set of corresponding remediation rules tailored to different types of potential root cause events. These rules consider not only the type of the root cause event but also its specific attributes, such as the type of lock contention (table lock or row lock) and the complexity of the slow query. By matching with the semantic remediation rule base, the system can generate targeted database remediation action plans. For example, if the identified potential root cause is a specific slow query causing database performance degradation, the remediation action plan might include specific measures such as optimizing the SQL statement of the slow query and adjusting the database query optimizer parameters.

[0079] Enter the specific parameters from the database repair action plan into the natural language template. The natural language template is a pre-designed template used to generate easy-to-understand natural language repair suggestions. These templates are categorized and designed according to different repair action plan types and parameters to ensure that the generated repair suggestions are clear, accurate, and easy to execute. For example, for a repair action plan to optimize slow queries, the natural language template might contain content such as "It is recommended to optimize the SQL statement for [slow query name], with the specific optimization direction as [description of optimization direction]". After entering the specific slow query name and optimization direction parameters into the template, structured repair suggestions are generated. Operations personnel can quickly and accurately perform database repair operations based on these suggestions, ensuring the stable operation of the database.

[0080] In one possible implementation, the pre-construction steps of the root cause pattern library include: collecting historical operational data, including time-series performance data of database instances, log information generated during database operation, slow query information, and lock contention information; labeling anomalies in the historical operational data; and determining the specific time of occurrence, the database instances involved, and the affected business modules of these anomalies. After labeling the anomalies, in-depth root cause analysis is performed on each labeled anomaly, using various analysis methods such as log analysis, performance indicator correlation analysis, slow query and lock contention analysis, combined with the database architecture and operating mechanism, to identify the root causes of the anomalies. Based on the results of the root cause analysis, representative root cause patterns are extracted from a large number of anomalies. Each anomaly and its corresponding root cause are abstracted and summarized to identify common characteristics and patterns. When extracting patterns, the commonalities and differences of different types of anomalies should be considered, and similar root cause patterns should be merged and optimized to form a concise, accurate, and universal set of root cause patterns. Simultaneously, clear feature descriptions and judgment conditions are defined for each root cause pattern to ensure accurate identification and application in subsequent matching processes. Choose a suitable database management system to store the root cause pattern set, forming a root cause pattern library.

[0081] In one possible implementation, the pre-construction steps of the semantic remediation rule base include: collecting detailed information on each anomaly, including its manifestation, involved database instances, and business scenarios; recording the specific remediation measures taken by operations personnel and their effects; and categorizing and organizing this collected experience according to anomaly type to form a historical operations experience database. For this database, each type of anomaly is analyzed in depth. The characteristics of each anomaly type are studied, including its frequency of occurrence, common manifestations, and potential impact. Simultaneously, the remediation measures taken for each anomaly type are analyzed in detail, summarizing the applicable scenarios and effects of different remediation measures. This step clarifies the correspondence between different anomaly types and their corresponding remediation measures. Based on the analysis results of anomaly types and remediation measures, the initial extraction of semantic remediation rules begins. Operations personnel use natural language descriptions to transform each anomaly type and its corresponding remediation measures into general rule statements. To facilitate rule management and application, the rules need to be standardized and unified. The standardized and unified semantic remediation rules are stored in a dedicated database management system, forming the semantic remediation rule base.

[0082] Furthermore, the construction of a local event relationship network based on a predefined causal graph specifically includes: A predefined background graph is established to describe the static relationships between entities within the database. Logs, slow queries, and lock contention information found within a preset time window are mapped to event nodes on the predefined background graph. By analyzing the temporal and resource contention relationships between event nodes, and dynamically inferring and generating directed causal edges between event nodes based on predefined causal rules, a local event relationship network that integrates static entity relationships and dynamic event causality is constructed.

[0083] In this embodiment, a predefined background graph is established to describe the static relationships between entities within the database. The database contains various entities, such as different tables, stored procedures, and indexes, and these entities have static relationships such as inclusion and association. The predefined background graph is a graph structure formed after sorting out and defining these static relationships. When constructing this graph, the fixed relationships between each entity are clarified based on the database design documents, metadata information, and past operational experience. For example, it is clarified that there is a foreign key relationship between table A and table B, and that stored procedure C operates on table D, etc. In this way, a comprehensive and accurate predefined background graph is constructed, providing a foundation for subsequent event node mapping.

[0084] The system maps logs, slow queries, and lock contention information collected within a preset time window to event nodes on a predefined background graph. The preset time window is a period centered on the alarm trigger time; logs, slow queries, and lock contention information collected within this period are crucial for analyzing alarm events. For log information, key error messages and operation information are extracted and mapped as event nodes to their corresponding positions in the predefined background graph. For example, if the log records an abnormal update operation on table E, this operation information is mapped as an event node to the location of table E in the predefined background graph. For slow query information, the system analyzes the tables, stored procedures, and other entities involved in the query and maps the slow query as an event node to its corresponding position. For example, if a slow query involves table F and stored procedure G, the slow query event node is mapped to the relevant positions of table F and stored procedure G in the predefined background graph. For lock contention information, the system identifies the resource entities involved in the lock contention, such as a specific table or row, and maps the lock contention information as an event node to its corresponding position. In this way, all kinds of collected information are accurately mapped to event nodes on a predefined background graph.

[0085] This analysis examines the temporal and resource contention relationships between event nodes. Temporal sequence refers to the chronological order in which events occur, crucial for determining causal relationships. For example, if event A occurs earlier than event B, and there's a logical connection between them, a causal relationship may exist. Resource contention refers to the competition for database resources among different event nodes. For instance, multiple queries simultaneously vying for access to a table can also lead to causal relationships. The analysis will combine the database's operational mechanisms with past experience to conduct a detailed examination of the temporal and resource contention relationships between each event node.

[0086] By dynamically inferring and generating directed causal edges between event nodes based on predefined causal rules, a local event relationship network integrating static entity relationships and dynamic event causality is constructed. The predefined causal rules are derived from the analysis and summarization of numerous database failure cases and normal operation scenarios, clarifying the possible causal relationships between event nodes under different conditions. For example, if a slow query occupies a table's resources for an extended period, preventing other queries from acquiring those resources in a timely manner, then according to the predefined causal rules, a causal relationship can be inferred between the slow query event node and other affected query event nodes, generating directed causal edges. In this way, based on the analyzed temporal relationships and resource contention relationships, combined with the predefined causal rules, the causal relationships between various event nodes are dynamically inferred, and directed causal edges are generated, connecting these event nodes to ultimately construct a local event relationship network integrating static entity relationships and dynamic event causality. This network clearly demonstrates the correlation and impact between various events within the database before and after an alarm event occurs, providing strong support for subsequent anomaly root cause localization.

[0087] In some embodiments, step S106 above, which involves quantifying the impact of alarm events on preset business indicators using a causal inference model and prioritizing and pushing alarm services based on the degree of impact, specifically includes: Based on the root cause of the alarm event and the database object information, the affected preset business indicators are identified, and the time-series data of the preset business indicators during the abnormal period and the control period are obtained. Based on the time-series data of preset business indicators during abnormal and control periods, a feature set for causal inference is constructed. The feature set includes the processing group identifier representing the abnormal period, the control period identifier, and potential confounding factors. The feature set is input into a pre-trained causal inference model, and the quantitative causal impact of the alarm event on the preset business indicators is calculated through counterfactual estimation. By combining the quantitative causal impact value, the criticality weight of business indicators, and the scope of impact, the comprehensive priority score of alarm events is calculated, and all involved alarm services are sorted and pushed according to the comprehensive priority score.

[0088] In this embodiment, after an alarm event occurs and the root cause of the anomaly is located, the affected preset business indicators are first determined by associating the root cause of the alarm event with database object information. For example, if the root cause is a lock contention issue in a critical table, and this table is the core data storage table for the order processing flow in the business system, then the associated preset business indicators may include order processing success rate, order processing time, etc. After determining the affected preset business indicators, it is necessary to obtain the time-series data of these preset business indicators during the abnormal period and the control period. The abnormal period is the time period during which the alarm event occurred, and the control period is a normal time period that is comparable to the abnormal period in terms of business cycle, system operating status, etc. For example, if the abnormal period is the business peak period from 10:00 AM to 11:00 AM on the same day, the control period can be the same time period from 10:00 AM to 11:00 AM on the previous day, when the system operating status is normal. In this way, the time-series data of the preset business indicators in two different time periods are obtained, providing a data foundation for subsequent causal inference.

[0089] Based on the time-series data of the acquired preset business indicators during abnormal and control periods, a feature set for causal inference is constructed. The feature set contains several key elements, including a processing group identifier representing the abnormal period to clearly identify that the data belongs to an abnormal period, and a control period identifier to distinguish data from normal periods. Potential confounding factors refer to those factors that may simultaneously affect alarm events and preset business indicators, but are not directly caused by the alarm events. For example, network maintenance operations performed simultaneously by the system may affect database performance and business indicators, but it is not directly related to the root cause of the alarm event and is considered a potential confounding factor. When constructing the feature set, these potential confounding factors need to be comprehensively considered and included in the feature set to ensure the accuracy of causal inference. By constructing a reasonable feature set, comprehensive and accurate information can be provided to the causal inference model, enabling it to accurately calculate the impact of alarm events on preset business indicators.

[0090] The constructed feature set is input into a pre-trained causal inference model. This model, trained on a large amount of historical data, is capable of identifying relationships between various factors within the feature set and making accurate causal inferences. After inputting the feature set, the model calculates the quantified causal impact of the alarm event on preset business metrics using counterfactual estimation. Counterfactual estimation is a hypothetical scenario-based analysis method. It assumes the state of the preset business metrics in the absence of an alarm event and compares this assumption with the actual state of the business metrics when the alarm event occurs, thus deriving the quantified impact of the alarm event on the business metrics. For example, counterfactual estimation might calculate that in the case of a lock-free contention anomaly, the order processing success rate will be 15% higher than when the anomaly actually occurs. This 15% is the quantified causal impact of the alarm event on the preset business metric of order processing success rate.

[0091] By combining the quantified causal impact value, the criticality weight of business metrics, and the scope of impact, a comprehensive priority score for alarm events is calculated. The criticality weight of business metrics is pre-set based on the importance of the business system and the degree of impact of the metric on the business. For example, order processing success rate is very critical to the business system, so its criticality weight might be set to 0.8, while the impact of order processing time is relatively small, so its criticality weight might be set to 0.5. The scope of impact refers to the breadth of the alarm event's influence on the business metrics. For example, if a lock contention problem only affects the processing of some orders, then the scope of impact is relatively small; if it affects the order processing of the entire business system, then the scope of impact is large. By comprehensively considering these three factors, the comprehensive priority score of the alarm event is calculated according to certain calculation rules (such as weighted summation). Finally, all involved alarm businesses are sorted according to the comprehensive priority score, with alarm businesses having higher priority scores listed first. Alarm information is then pushed to operations personnel in this order, so that operations personnel can prioritize handling alarm events with a high impact on the business and high criticality, improving operational efficiency and database stability.

[0092] Furthermore, the pre-training steps of the causal inference model include: A business evaluation training sample set is constructed based on historical business indicator anomaly data. The business evaluation training sample set includes confounding factor features that characterize the environment and business status, processing variable labels that indicate whether database anomalies have occurred, and corresponding business indicator observations. A causal inference model is constructed, which includes a shared feature encoding layer and two parallel prediction branches corresponding to the treatment group and the control group, respectively. The business evaluation training sample set is input into the causal inference model for training. The model parameters are optimized by a multi-objective joint loss function, which includes a fact loss term for fitting the observed business indicators, a counterfactual regularization term for constraining the rationality of the intervention effect, and a distribution alignment term for balancing the feature representation distributions of the outputs of the two parallel prediction branches. During training, the labels of the processing variables are controlled to activate the corresponding prediction branches, enabling the causal inference model to learn and estimate the potential business indicator results under both the occurrence and non-occurrence of anomalies. The difference between the potential business indicator results under the two scenarios is used as an estimate of the intervention effect.

[0093] In this embodiment, a large amount of historical business indicator anomaly data is accumulated through long-term database operation and maintenance management practices. This data contains rich information and forms the basis for constructing a business evaluation training sample set. First, a comprehensive and detailed analysis of the historical business indicator anomaly data is conducted to extract confounding factor features representing the environment and business status. For example, network upgrades performed by the system within the same time period, peak business periods, etc., while not the direct cause of database anomalies, can affect business indicators and are considered confounding factor features. Simultaneously, processing variable labels are explicitly defined to indicate whether a database anomaly has occurred. If the database experiences lock contention or other anomalies during a certain period, it is marked as an anomaly occurring; if the database operates normally, it is marked as an anomaly not occurring. Furthermore, corresponding business indicator observations, such as order processing success rate and business response time, need to be obtained. Integrating these confounding factor features, processing variable labels, and business indicator observations constitutes the business evaluation training sample set, providing necessary data support for the subsequent training of the causal inference model.

[0094] Based on the understanding and requirements of causal inference tasks, a causal inference model with a specific structure is constructed. This model includes a shared feature encoding layer, which uniformly encodes the confounding factor features in the input business evaluation training sample set, extracting a general feature representation to provide a foundation for subsequent prediction branches. Simultaneously, the model also has two parallel prediction branches corresponding to the treatment group and the control group, respectively. The treatment group corresponds to the case where database anomalies occur, and the control group corresponds to the case where database anomalies do not occur. This structural design allows the model to process data under different states, thereby more accurately estimating the intervention effect and laying the foundation for subsequent quantitative assessment of the impact of alarm events on preset business indicators.

[0095] The constructed business assessment training sample set is input into the constructed causal inference model for training. During training, a multi-objective joint loss function is used to optimize the model parameters. This multi-objective joint loss function consists of several parts. The fact loss term is used to fit the observed business indicators, ensuring that the model can accurately learn the changes in actual business indicators and make the model output as close as possible to the true observations. The counterfactual regularization term is used to constrain the rationality of the intervention effect, preventing the model from producing unreasonable results when estimating the intervention effect, and ensuring the accuracy and reliability of the estimation results. The distribution alignment term is used to balance the feature representation distribution of the outputs of the two parallel prediction branches, making the feature representations of the treatment group and the control group comparable, and avoiding the impact of excessive differences in feature distribution on model performance. By comprehensively considering these three objectives, the multi-objective joint loss function can comprehensively optimize the model parameters and improve the model's performance and generalization ability.

[0096] During training, the corresponding prediction branches are activated by precisely controlling the processing variable labels. When the processing variable label indicates that a database anomaly has occurred, the prediction branch corresponding to the processing group is activated, and the model learns to estimate the potential business indicator results under the anomaly scenario. When the processing variable label indicates that a database anomaly has not occurred, the prediction branch corresponding to the control group is activated, and the model learns to estimate the potential business indicator results under the anomaly-free scenario. By continuously alternating the activation of different prediction branches and inputting corresponding training samples, the model can learn the change patterns of potential business indicators under the two scenarios. Finally, the difference between the potential business indicator results under the two scenarios of anomaly occurrence and non-occurrence is used as an estimate of the intervention effect. This estimate of the intervention effect can accurately reflect the degree of impact of database anomalies on business indicators, providing an important basis for subsequent prioritization and push based on the impact of alarm events on preset business indicators.

[0097] Reference Figure 2 An embodiment of the present invention provides a database dynamic monitoring system 2 based on artificial intelligence, wherein the system 2 specifically includes: The data acquisition module 201 is used to acquire the performance time-series data of the database instance, dynamically adjust the sampling frequency based on the load fluctuation characteristics of the performance time-series data, and synchronously collect logs, slow queries and lock contention information associated with the performance time-series data. The data denoising module 202 is used to process performance time-series data and its associated logs, slow queries and lock contention information through a data smoothing algorithm to obtain clean data after denoising. The threshold prediction module 203 is used to generate dynamic alarm thresholds for database instances based on clean data and through a threshold prediction model based on a federated learning framework. The alarm triggering module 204 is used to monitor clean data in real time and trigger an alarm event when the clean data exceeds the dynamic alarm threshold. The repair suggestion module 205 is used to respond to alarm events, automatically correlate and analyze the corresponding logs, slow queries and lock contention information to locate the root cause of the anomaly, and generate repair suggestions; The business evaluation module 206 is used to quantify the impact of alarm events on preset business indicators through a causal inference model, and to prioritize and push alarm services according to the degree of impact.

[0098] It is understandable that, such as Figure 1 The content of the AI-based database dynamic monitoring method embodiments shown herein is applicable to the AI-based database dynamic monitoring system embodiments. The specific functions implemented in the AI-based database dynamic monitoring system embodiments are the same as those described above. Figure 1 The illustrated embodiment of the AI-based database dynamic monitoring method is the same, and achieves the same beneficial effects as shown. Figure 1 The beneficial effects achieved by the AI-based database dynamic monitoring method embodiment shown are also the same.

[0099] It should be noted that the information interaction and execution process between the above systems are based on the same concept as the method embodiments of the present invention. For details on their specific functions and technical effects, please refer to the method embodiments section, which will not be repeated here.

[0100] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the system can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0101] Reference Figure 3The present invention also provides a computer device 3, including: a memory 302 and a processor 301, and a computer program 303 stored in the memory 302. When the computer program 303 is executed on the processor 301, it implements the database dynamic monitoring method based on artificial intelligence as described in any of the above methods.

[0102] The computer device 3 may be a desktop computer, laptop, handheld computer, or cloud server, etc. The computer device 3 may include, but is not limited to, a processor 301 and a memory 302. Those skilled in the art will understand that... Figure 3 The computer device 3 is merely an example and does not constitute a limitation on the computer device 3. It may include more or fewer components than shown in the figure, or combine certain components, or different components, such as input / output devices, network access devices, etc.

[0103] The processor 301 can be a Central Processing Unit (CPU), but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0104] In some embodiments, the memory 302 may be an internal storage unit of the computer device 3, such as a hard disk or memory of the computer device 3. In other embodiments, the memory 302 may be an external storage device of the computer device 3, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the computer device 3. Furthermore, the memory 302 may include both internal and external storage units of the computer device 3. The memory 302 is used to store the operating system, applications, boot loader, data, and other programs, such as the program code of the computer program. The memory 302 can also be used to temporarily store data that has been output or will be output.

[0105] This invention also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the AI-based dynamic database monitoring method as described in any of the above methods.

[0106] In this embodiment, if the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include at least: any entity or device capable of carrying computer program code to a photographing device / terminal device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.

[0107] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0108] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0109] In the embodiments disclosed in this application, it should be understood that the disclosed devices / terminal equipment and methods can be implemented in other ways. For example, the device / terminal equipment embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling or direct coupling or communication connection may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0110] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

Claims

1. A database dynamic monitoring method based on artificial intelligence, characterized in that, The method specifically includes: Obtain performance time-series data of the database instance, dynamically adjust the sampling frequency based on the load fluctuation characteristics of the performance time-series data, and synchronously collect logs, slow queries and lock contention information associated with the performance time-series data; The data smoothing algorithm is used to process the performance time series data and its associated logs, slow queries and lock contention information to obtain clean data after noise reduction; Based on clean data, a threshold prediction model based on a federated learning framework is used to generate dynamic alarm thresholds for database instances. Monitor cleanliness data in real time and trigger an alarm event when the cleanliness data exceeds the dynamic alarm threshold; In response to alarm events, the system automatically correlates and analyzes the corresponding logs, slow queries, and lock contention information to locate the root cause of the anomaly and generate remediation suggestions. The impact of alarm events on preset business indicators is quantitatively assessed using a causal inference model, and alarm services are prioritized and pushed according to the degree of impact.

2. The method according to claim 1, characterized in that, The dynamic adjustment of the sampling frequency based on the load fluctuation characteristics of performance time-series data specifically includes: Based on performance time-series data, incremental sliding volatility is calculated for each performance indicator to obtain a real-time volatility quantification value for each performance indicator. Based on historical anomaly contribution and business importance, a corresponding performance weight coefficient is assigned to each performance indicator. The real-time fluctuation quantification values ​​of each performance indicator are then merged using the performance weight coefficient to generate a comprehensive fluctuation score that characterizes the overall load fluctuation status of the database instance. The system compares the comprehensive fluctuation score with the fluctuation threshold range in real time, and dynamically adjusts the sampling frequency of the next sampling period based on the comparison results and the preset multi-level frequency adjustment strategy. Analyze the monitoring effect data after the sampling frequency is adjusted, and dynamically optimize the boundary value of the fluctuation threshold range corresponding to the comprehensive fluctuation score based on the monitoring effect data.

3. The method according to claim 1, characterized in that, The process involves using a data smoothing algorithm to process performance time-series data and its associated logs, slow queries, and lock contention information to obtain clean, denoised data. Specifically, this includes: The Kalman filter algorithm is used to adaptively smooth the performance time series data to obtain a pre-smoothed performance data sequence. The associated logs, slow queries, and lock contention information are parsed and marked to extract the occurrence time and event type of the associated events; Based on the preset event type-weight mapping relationship, different evidence weights are assigned to related events of different event types; Based on the occurrence time of the associated events, the corresponding data points are located in the performance data sequence. Then, according to the event type of the associated events and the evidence weight, the smoothing result of the data points is weighted and corrected to generate the final clean data.

4. The method according to claim 3, characterized in that, The step of adaptively smoothing the performance time series data using the Kalman filter algorithm to obtain a preliminarily smoothed performance data sequence specifically includes: A Kalman filter state-space model is established to describe the state-observation relationship of the performance index, and the parameters of the Kalman filter state-space model are initialized. Real-time analysis of the short-term statistical characteristics of the input performance time-series observation data, and dynamic generation of observation noise level parameters based on the analysis results of the short-term statistical characteristics; In each iteration of the Kalman filter, dynamically generated observation noise level parameters are injected, and the adaptive Kalman gain is calculated in real time based on the observation noise level parameters and the current state prediction error. The actual observation value at the current moment is obtained, and the residual between the actual observation value and the state prediction value is weighted and corrected using adaptive Kalman gain. The optimal state estimate at the current moment is then updated to form a preliminary smoothed performance data sequence.

5. The method according to claim 3, characterized in that, The process of parsing and marking related logs, slow queries, and lock contention information to extract the occurrence time and event type of related events specifically includes: Based on the associated log stream, a pre-configured regular expression pattern is used to match with a keyword dictionary to capture error codes, description text, and related database object identifiers, forming a first initial field set; Based on the associated slow query list, the operation type, core table names involved and index features are extracted by parsing the query statement to form a second initial field set; Based on the associated lock contention states, a resource waiting relationship graph is constructed by analyzing the lock view to identify blocking chains and related session and resource identifiers, forming a third set of initial fields; Map the semantic content of events in the first initial field set, the second initial field set, and the third initial field set into structured associated event feature vectors; Based on the feature vectors of related events, the related events are classified into a multi-level event classification tree, and timestamps and type codes are assigned to the related events in combination with preset priority rules.

6. The method according to claim 1, characterized in that, The process of generating dynamic alarm thresholds for database instances based on clean data and a threshold prediction model using a federated learning framework specifically includes: A threshold prediction global model is initialized based on the federated learning coordination server, and the initial parameters of the threshold prediction global model are distributed to the local agents of each database instance. Each local agent uses its corresponding local clean data to incrementally train the global threshold prediction model and obtain updated local model parameters. The updated local model parameters of each local agent are uploaded to the federated learning coordination server. The federated learning coordination server performs secure aggregation of local model parameters from multiple database instances to generate new global model parameters and distribute them to each local agent. Each local agent performs forward inference calculations based on the newly issued global model parameters and its own local clean data to generate personalized dynamic alarm thresholds for its respective database instance.

7. The method according to claim 6, characterized in that, The initialization of a threshold prediction global model based on the federated learning coordination server specifically includes: Collect configuration and historical statistical metadata from multiple database instances, and perform aggregation analysis and instance clustering based on the configuration and historical statistical metadata to obtain clustering results; Based on the time-series data characteristics of the database instances of each group in the clustering results, the threshold prediction model structure suitable for different groups is determined and constructed. Based on the domain knowledge extracted from configuration and historical statistical metadata, differentiated initialization is performed on the parameters of the threshold prediction model structure, and prior constraints are injected into the objective function of the threshold prediction model structure. By using privacy-preserving statistical features or synthetic data obtained from various database instances, the initialized threshold prediction model structure is trained to obtain a global threshold prediction model.

8. The method according to claim 1, characterized in that, In response to alarm events, the system automatically correlates and analyzes corresponding logs, slow queries, and lock contention information to pinpoint the root cause of the anomaly and generates remediation suggestions, specifically including: Extract the alarm trigger time from the alarm event, and search for the corresponding logs, slow queries and lock contention information within a preset time window with the alarm trigger time as the center. Construct a local event relationship network based on a predefined causal graph. The local event relationship network is matched and compared with a pre-built root cause pattern library to locate one or more potential root cause events that lead to alarm events. Based on the type and attributes of the identified potential root cause events, a targeted database repair action plan is generated by matching them with a pre-built semantic repair rule base. Enter the specific parameters from the database repair action plan into a natural language template to generate structured repair suggestions.

9. The method according to claim 1, characterized in that, The process of quantifying and evaluating the impact of alarm events on preset business metrics using a causal inference model, and prioritizing and pushing alarm services based on the degree of impact, specifically includes: Based on the root cause of the alarm event and the database object information, the affected preset business indicators are identified, and the time-series data of the preset business indicators during the abnormal period and the control period are obtained. Based on the time-series data of preset business indicators during abnormal and control periods, a feature set for causal inference is constructed. The feature set includes the processing group identifier representing the abnormal period, the control period identifier, and potential confounding factors. The feature set is input into a pre-trained causal inference model, and the quantitative causal impact of the alarm event on the preset business indicators is calculated through counterfactual estimation. By combining the quantitative causal impact value, the criticality weight of business indicators, and the scope of impact, the comprehensive priority score of alarm events is calculated, and all involved alarm services are sorted and pushed according to the comprehensive priority score.

10. A database dynamic monitoring system based on artificial intelligence, characterized in that, The system specifically includes: The data acquisition module is used to acquire performance time-series data of the database instance, dynamically adjust the sampling frequency based on the load fluctuation characteristics of the performance time-series data, and synchronously collect logs, slow queries and lock contention information associated with the performance time-series data. The data denoising module is used to process performance time-series data and its associated logs, slow queries and lock contention information through data smoothing algorithms to obtain clean data after denoising. The threshold prediction module is used to generate dynamic alarm thresholds for database instances based on clean data and through a threshold prediction model based on a federated learning framework. The alarm triggering module is used to monitor clean data in real time and trigger an alarm event when the clean data exceeds the dynamic alarm threshold. The repair suggestion module is used to respond to alarm events, automatically correlate and analyze the corresponding logs, slow queries and lock contention information to locate the root cause of the anomaly, and generate repair suggestions; The business evaluation module is used to quantify the impact of alarm events on preset business indicators through a causal inference model, and to prioritize and push alarm services according to the degree of impact.

Images