Python-based automatic database operation and maintenance management method and system

Abstract

The invention discloses a Python-based automatic database operation and maintenance management method and system, and the method comprises the steps: collecting and preprocessing a plurality of index data related to database operation and maintenance, building a multi-task learning model through a Python deep learning framework, achieving the precise prediction and priority classification of task execution time, and achieving the automatic database operation and maintenance. The time sequence data of the system load can be analyzed, so that more accurate load prediction is provided for task scheduling, and an intelligent scheduling strategy based on the load and the priority is realized. Besides, through optimization and continuous verification of a model training process, high efficiency and adaptability of the model in practical application are guaranteed, the intelligence level and response speed of database operation and maintenance management are finally and remarkably improved, unnecessary resource consumption is reduced, a large amount of manual operation time is saved, and the method is suitable for popularization and application. Therefore, the operation and maintenance personnel can concentrate energy on more complex technical problems, so that the problem of excessive manual intervention is solved.

Description

Technical Field

[0001] The present invention belongs to the technical field of data processing, and in particular relates to an automated database operation and maintenance management method and system based on Python. Background Art

[0002] Automated database operation and maintenance management refers to the management and maintenance of database systems through automated tools and scripts to reduce manual intervention, improve efficiency, reduce human errors, and ensure high availability, stability, and security of the database. It mainly relies on automation technology to perform a series of database management tasks, including but not limited to database monitoring, backup, recovery, performance optimization, security management, fault detection and repair, etc.

[0003] Most of the existing database operation and maintenance management tools are developed based on scripting languages ​​or specific management platforms. For example, traditional operation and maintenance tools such as Nagios, Zabbix, Prometheus, etc., although they can provide certain monitoring functions, Nagios is a widely used open source monitoring software, mainly used to monitor the availability and performance of IT infrastructure. It can monitor the status of the database through plug-ins. Nagios mainly focuses on monitoring and alarming.

[0004] When dealing with complex database management tasks, it relies on manual configuration and management, and has poor scalability. To address these problems, some database vendors have launched their own operation and maintenance solutions, such as Oracle's Enterprise Manager and SQL Server's SQL Server Management Studio (SSMS). However, these tools are often too large, have single functions, and have limitations in terms of operational complexity and customization.

[0005] The widespread use of cloud databases and distributed databases in recent years has made database operation and maintenance more complex and diversified. As the scale of databases continues to increase, the demand for automated operation and maintenance has become more urgent. The traditional manual operation and maintenance model can no longer meet the needs of efficient management and refined operations. Summary of the invention

[0006] The purpose of the present invention is to provide an automated database operation and maintenance management method and system based on Python, which saves a lot of manual operation time and enables operation and maintenance personnel to focus on more complex technical issues to solve the problem of excessive manual intervention.

[0007] To achieve the above object, the present invention, on the one hand, proposes an automated database operation and maintenance management method based on Python, comprising the following steps: Collect and pre-process multiple indicator data related to database operation and maintenance, the indicator data at least including system load, database connection status, disk space, network bandwidth, task execution history, query response time, health status, fault log, task priority, urgency, time sensitivity and business cycle; Based on the collected indicator data, a multi-task learning model is built using Python's deep learning framework, which extracts common information from all input features through a shared feature extraction layer, and sets an independent output layer for each task to optimize task requirements; The constructed multi-task learning model is used to predict task execution time and classify task priorities, taking into account the execution timing and resource allocation of tasks.

[0008] Preferably, the method further comprises the following steps: Recurrent neural networks are applied to analyze the time series data of system load to capture long-term dependencies and predict future load changes, thereby providing accurate load prediction for task scheduling.

[0009] Preferably, the method further comprises the following steps: The load prediction results are used to dynamically adjust the task execution frequency and timing to implement an intelligent scheduling strategy based on load and priority.

[0010] Preferably, the method further comprises the following steps: Implement a systematic training process to ensure that the task learning framework can effectively learn historical task data, and evaluate the model generalization ability by selecting appropriate loss functions and using cross-validation.

[0011] Preferably, the method further comprises the following steps: In the process of optimizing the performance of multi-task learning models, hyperparameter tuning, integrated learning methods, and early stopping strategy techniques are introduced to adapt to different data characteristics.

[0012] Preferably, the method further comprises the following steps: The multi-task learning model after step optimization is verified using a test dataset to ensure its effectiveness in the actual environment, and the model is continuously optimized based on feedback.

[0013] On the other hand, the present invention proposes an automated database operation and maintenance management system based on Python, including: a data acquisition module for collecting multiple indicator data related to database operation and maintenance; a data preprocessing module for cleaning and standardizing the collected indicator data; a task learning framework training module for training a multi-task learning model. An intelligent scheduling module for realizing intelligent scheduling of tasks to maximize resource utilization and task completion efficiency. A time series analysis module for performing time series analysis on system load to assist in achieving more accurate task scheduling. An automated operation and maintenance execution module for automatically executing database operation and maintenance tasks, including but not limited to scheduled backup, automated query, batch execution of SQL scripts, and regular cleaning of redundant data.

[0014] Technical effects and advantages of the present invention: Compared with the prior art, the present invention proposes a Python-based automated database operation and maintenance management method and system, which has the following advantages: The present invention collects and preprocesses multiple indicator data related to database operation and maintenance, and uses Python's deep learning framework to build a multi-task learning model, which achieves accurate prediction of task execution time and priority classification, and can analyze the time series data of system load, thereby providing more accurate load prediction for task scheduling and realizing intelligent scheduling strategies based on load and priority. In addition, by optimizing and continuously verifying the model training process, the efficiency and adaptability of the model in practical applications are guaranteed, and ultimately the intelligence level and response speed of database operation and maintenance management are significantly improved, unnecessary resource consumption is reduced, and a large amount of manual operation time is saved, so that operation and maintenance personnel can focus on more complex technical problems to solve the problem of excessive manual intervention, and improve the reliability of services and user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 This is a flow chart of the Python-based automated database operation and maintenance management method of the present invention; Figure 2 It is a data preprocessing flow chart of the present invention; Figure 3 The architecture diagram of the shared feature extraction layer and the independent output layer of multiple tasks of the present invention; Figure 4 This is a multi-task learning framework diagram of the present invention; Figure 5 This is a flow chart of the time series analysis in load forecasting and task scheduling of the present invention. DETAILED DESCRIPTION

[0016] The following will be combined with the accompanying drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. The specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0017] In an embodiment of the present invention, a Python-based automated database operation and maintenance management system is proposed, including automated task scheduling and execution; Through Python scripts, we can achieve scheduled backup of the database, automated query, batch execution of SQL scripts, and regular cleaning of redundant data. The above tasks can be automatically executed through the scheduled scheduling system, avoiding the tediousness and instability of manual operations.

[0018] Through Python's monitoring library, the database's performance, load, storage, response time and other indicators are monitored in real time, and alarms are automatically sent when abnormalities occur according to preset alarm rules to ensure the high availability of the database. By integrating Python with database management tools, the system automatically analyzes logs when failures occur, identifies common errors and automatically repairs them (such as restarting database services, cleaning up useless connections, etc.), reducing manual intervention. Based on Python's data processing and visualization library, the database operation status is deeply analyzed and customized operation and maintenance reports are generated to help operation and maintenance personnel understand the health of the database in real time.

[0019] The Python-based automated database operation and maintenance management system is an effective supplement and innovation to the traditional database operation and maintenance model. Through automation and combined with Python's powerful scripting capabilities, it can improve database operation and maintenance efficiency, reduce operation and maintenance costs, and improve system stability and reliability. Automatic task scheduling and execution, realize the scheduled scheduling and automatic execution of database operation and maintenance tasks, ensure that tasks such as regular backup, cleaning redundant data, and executing SQL scripts can be automatically run to avoid human omissions and operational errors. Use the scheduling library in Python to set scheduled tasks. Most current scheduling systems usually rely on fixed scheduling times or periodic task arrangements. In some cases, the execution time of tasks may need to be dynamically adjusted instead of sticking to the preset time point; Use machine learning algorithms or rule-based scheduling engines to predict the best time to execute tasks based on historical data (such as system load, task execution time, etc.), and automatically adjust the frequency and timing of task execution. For example, by analyzing the historical pattern of task execution, you can intelligently infer the best execution time in the future and optimize resource allocation; In order to implement intelligent scheduling strategies based on load and priority, and combine machine learning or rule engines to optimize the execution timing and frequency of tasks, the key lies in how to select and combine reasonable indicators (or features) to make the prediction model more accurate; Important indicators for automated task scheduling and execution include: system load (CPU, memory usage), number of database connections and connection pool status, disk space and storage pressure, network bandwidth and latency, task execution history data, database query response time and query volume, database health status and fault logs, task priority and urgency, time sensitivity and business cycle; Combine the above multiple indicators into one model to calculate the optimal execution time of the task. In order to combine all the identifiers into an effective scheduling model, a multi-task learning method is adopted to combine multiple tasks into one model, and use shared features to improve the overall performance of the model. The regression model and the classification model are combined, and the results of task execution time prediction and task priority classification are used to comprehensively consider the execution timing of tasks and resource allocation. The neural network is used to process the input and output of different tasks, and they are learned together through shared layers and different output layers. The details are as follows: In this embodiment, the Python-based automated database operation and maintenance management system is Figure 2-Figure 5 As shown, including: The data collection module is used to collect multiple indicator data related to database operation and maintenance; the data preprocessing module is used to clean and standardize the collected indicator data.

[0020] Specifically, data preprocessing is a key step in building an efficient scheduling model. It processes multiple indicators and extracts features related to task execution time and priority. It cleans data for each indicator, handles missing values, outliers, and noise data, and performs feature standardization and normalization for different types of data to ensure that each indicator is compared on the same scale.

[0021] For example, system load, CPU usage, memory usage, etc. can be standardized so that they are all between 0 and 1. For time series data (such as database query response time, number of database connections, etc.), time window methods are used to construct time series features (such as sliding average, trend analysis, etc.) to extract the historical change trend of system load. For classification features (such as task priority and urgency), one-hot encoding is used for processing. Based on feature selection, redundant features are removed and features most relevant to task execution time and priority are selected. For categorical features (such as task priority and urgency), each category is converted into a binary vector through One-Hot Encoding to ensure that each category is independent and has no sequential relationship.

[0022] For example, assuming that there are three categories of task priority: high, medium, and low, the one-hot encoded representation is: High: [1,0,0] Middle: [0,1,0] Low: [0,0,1] Feature Selection Correlation Coefficient The correlation coefficient measures the strength of the linear relationship between two variables. For regression tasks (task execution time prediction), the Pearson correlation coefficient between each feature and the target variable (task execution time) is calculated, and the feature with the larger correlation coefficient is selected; ; in, and are the sample values ​​of the two variables, and are their means respectively; For classification tasks with task priorities, the correlation between classification features and target variables was evaluated by calculating the point-biserial correlation coefficient between the features and task priorities; Mutual information is a nonlinear method that measures the dependency between two variables. It reflects the amount of information shared by a feature and the target variable. The relationship between classification features and target categories is evaluated through the mutual information method. The larger the mutual information, the stronger the relationship between the feature and the target variable. The most important features are selected by repeatedly training the model and removing the least important features.

[0023] Use Python's pandas library to efficiently read and process data sets, clean up missing values, outliers, and noisy data. For missing data, Python quickly detects and processes outliers. For features of different ranges (such as CPU usage, memory usage, etc.), you can use StandardScaler or MinMaxScaler in Python's sklearn.preprocessing to standardize or normalize these features so that they are in the same scale range. For time series data (such as database query response time, number of connections, etc.), use Python's pandas library to perform sliding average, trend analysis, and other processing to extract features within the time window. Use the statsmodels library in Python for trend analysis and seasonal analysis. For categorical features such as task priority and urgency, Python provides sklearn.preprocessing.OneHotEncoder for one-hot encoding, which facilitates the conversion of categorical data into numerical data.

[0024] Task learning framework training module, used to train multi-task learning models.

[0025] Specifically, in the multi-task learning framework, a joint model is set up to handle multiple tasks simultaneously, including task execution time prediction, task priority classification, and load prediction. The model learns representations of different tasks through shared layers and multiple output layers.

[0026] During the design process, first, a shared feature extraction layer is built to extract common features from all input indicators. This shared layer includes a multi-layer neural network to capture the nonlinear relationship between input features and ensure that different tasks can share this information. An independent output layer is set for each task. The output layer is optimized according to the goals of different tasks. The task execution time prediction can use the regression output layer, the task priority classification uses the classification output layer, and the load prediction can also be a regression task. Therefore, the model relies on shared feature information during the learning process of each task, while being able to perform special optimizations for the specific needs of the task. Multi-task learning framework,The network architecture of the multi-task learning framework includes shared layers and task-specific layers (output layers); Using Python's deep learning framework, it is convenient to build a multi-task learning model. In this model, first, a shared layer is used to extract common information from all input features, and then an independent output layer is designed for each task (such as task execution time prediction, priority classification, etc.). For regression tasks (task execution time prediction), the output layer uses a linear activation function to predict the time. For classification tasks (task priority prediction), the output layer uses a softmax activation function to predict the category. Python provides a variety of optimizers and loss functions to optimize multi-task learning models; Given the output layer of a neural network in is the number of categories, is the raw value (logits) output by the network; The Softmax activation function converts these raw values ​​into probabilities, as follows: ; in is the first Values ​​(logit); is the total number of categories; Yes Perform exponential transformation; Denominator is the sum of the exponentials of all output values; Each value of Softmax output will be between 0 and 1, and the sum of the probabilities of all categories is 1. In this way, the output of each category can be interpreted as the predicted probability of that category; The training process of the multi-task learning module specifically includes: (a) Forward propagation Feed all input features (such as task history data, load data, etc.) into the network; The shared layer extracts common features and passes them to the task-specific layer; The task-specific layers process the shared features separately for the output layer of each task and generate task-specific predictions.

[0027] For regression tasks, generate task execution time or load predictions; For classification tasks, generate prioritized class predictions for the task.

[0028] (b) Calculation of loss For each task, calculate the loss function (such as MSE or cross entropy loss) separately; The losses of the regression task and the classification task are weighted and combined into the final loss function.

[0029] (c) Backpropagation and optimization Use an optimizer (such as Adam or SGD) to calculate the gradient through the back-propagation algorithm and update the weights of the neural network; In each training cycle (epoch), the network parameters are updated according to the total loss function (the weighted sum of regression loss and classification loss).

[0030] (d) Hyperparameter Tuning During the training process, hyperparameters such as learning rate, regularization coefficient, and weight coefficient of task loss are tuned.

[0031] Cross-validation can be used to evaluate hyperparameter combinations and select the best hyperparameter configuration.

[0032] (e) Early stop Monitor the loss change on the validation set. If the loss does not improve over multiple training cycles, stop training to avoid overfitting.

[0033] In the multi-task learning framework, task execution time prediction and task priority classification are processed as regression tasks and classification tasks, respectively. First, for task execution time prediction, a regression model is used to estimate the resources and time required for the task. The input of the regression model includes the current load of the database (CPU, memory usage), task history data, disk space, etc. These input features are processed by a neural network to output the estimated execution time of the task. Second, for task priority classification, a classification model, such as a multi-layer perceptron (MLP) or a convolutional neural network (CNN), is used to predict the priority of the task. The input features include the urgency of the task, time sensitivity, historical execution data, etc. The classification model divides the tasks into different priorities, such as urgent, regular, low priority, etc. Through the joint training of regression and classification tasks, the model can not only predict the execution time of each task, but also determine its priority, so as to reasonably schedule tasks according to the priority and resource status.

[0034] In the multi-task learning framework, Python provides a wealth of libraries and methods to implement these two tasks: For task execution time prediction, Python's scikit-learn and TensorFlow / PyTorch are used to implement regression models. The input of the regression model includes database load data, historical execution data, etc., and the output predicts the execution time of the task. For task priority prediction, the neural network model in TensorFlow / PyTorch (such as multi-layer perceptron, MLP) is used for classification. The classification model predicts the task priority through input features (such as urgency, historical data). During the model training process, Python provides a multi-task learning framework to jointly train regression and classification tasks. Through shared layers and independent output layers, Python can simultaneously optimize the loss functions of multiple tasks (regression loss and classification loss).

[0035] The intelligent scheduling module is used to realize intelligent scheduling of tasks, maximize resource utilization and task completion efficiency. The time series analysis module is used to perform time series analysis on system load to assist in more accurate task scheduling.

[0036] Specifically, load prediction is a key issue in task scheduling, especially for the dynamic changes of database and system loads. In order to effectively predict the load situation, time series analysis methods and recursive neural networks (RNNs), such as LSTM (Long Short-Term Memory Networks), are used to model the time dependence of the load. Through the LSTM network, the time series characteristics of indicators such as system load, database response time, and network bandwidth can be captured to predict the load change trend in the future. These prediction results will be used as input for task scheduling to help determine the execution time of tasks under different load conditions. By combining with other features (such as task priority and health status), the LSTM model can not only provide prediction results for system load, but also assist regression models and classification models to more accurately determine the execution time and priority of tasks. In addition, by introducing sliding windows and historical pattern analysis, the model can dynamically adjust the prediction strategy based on historical data and gradually optimize the accuracy of task scheduling; In the load prediction task, Python's TensorFlow or PyTorch provides support for recursive neural networks such as LSTM (Long Short-Term Memory Network) for processing time series data. LSTM can effectively capture the long-term dependencies of indicators such as system load and query response time, and predict future load changes. Python can combine historical task execution data (such as load data of the past few days) and split historical data into different time periods through the sliding window method to construct input features. Using time series models (such as LSTM), Python can gradually optimize the accuracy of load prediction and help better perform task scheduling.

[0037] The automated operation and maintenance execution module is used to automatically execute database operation and maintenance tasks, including but not limited to scheduled backup, automated query, batch execution of SQL scripts, and regular cleaning of redundant data.

[0038] Database operation and maintenance usually involves repetitive tasks, such as regular backup, log cleaning, resource monitoring, performance optimization, etc. These tasks often take up a lot of time and energy of operation and maintenance personnel, and are easily missed or neglected without automation tools. The automation system automatically performs these repetitive tasks through Python scripts, performs regular backup, performance monitoring, log analysis, etc., saving a lot of manual operation time and allowing operation and maintenance personnel to focus on more complex technical issues to solve the problem of excessive manual intervention.

[0039] The present invention provides a Python-based automated database operation and maintenance management method, such as Figure 1 As shown, the following steps are included: Collect and pre-process multiple indicator data related to database operation and maintenance, the indicator data at least including system load, database connection status, disk space, network bandwidth, task execution history, query response time, health status, fault log, task priority, urgency, time sensitivity and business cycle; Based on the collected indicator data, a multi-task learning model is built using Python's deep learning framework, which extracts common information from all input features through a shared feature extraction layer, and sets an independent output layer for each task to optimize task requirements; The constructed multi-task learning model is used to predict task execution time and classify task priorities, taking into account the execution timing and resource allocation of tasks.

[0040] Furthermore, the following steps are included: applying a recursive neural network to analyze the time series data of the system load to capture long-term dependencies and predict future load changes, thereby providing accurate load prediction for task scheduling.

[0041] Furthermore, the method also includes the following steps: dynamically adjusting the task execution frequency and timing using the load prediction results to implement an intelligent scheduling strategy based on load and priority.

[0042] Furthermore, the following steps are included: implementing a systematic training process to ensure that the task learning framework can effectively learn historical task data, and introducing hyperparameter tuning, integrated learning methods, and early stopping strategy techniques in the process of optimizing the performance of multi-task learning models by selecting appropriate loss functions and using cross-validation to adapt to different data characteristics, and verifying the multi-task learning model after step optimization using a test data set to ensure its effectiveness in the actual environment, and continuously optimizing the model based on feedback; Specifically, the model training process includes multiple steps. First, the training data set needs to contain rich historical task data, and each piece of data should contain all relevant indicators (such as system load, task execution history, database connection status, etc.). When using these data for model training, select a suitable loss function to ensure the accuracy of task execution time prediction and priority classification. For regression tasks (task execution time prediction), the loss function usually uses mean square error (MSE); for classification tasks (task priority prediction), cross entropy loss is used. During the training process, the cross-validation method is used to evaluate the generalization ability of the model to prevent overfitting. At the same time, the early stopping strategy can be introduced to prevent over-optimization during the training process. In order to further improve the performance of the model, hyperparameter tuning (such as grid search, random search) is used to select the best learning rate, number of network layers, regularization coefficient and other parameters. During the model training process, ensemble learning (such as random forest, gradient boosting tree) and other methods can also be introduced to improve the stability and accuracy of the prediction. After the training is completed, the model is verified using the test data set to ensure its effectiveness in the actual environment, and the model is continuously optimized based on feedback; Using Python's scikit-learn library, the dataset is divided into training and test sets, and cross-validation is used to evaluate the generalization ability of the model. Python provides tools such as GridSearchCV and RandomizedSearchCV to perform hyperparameter tuning and optimize parameters such as learning rate and regularization coefficient to improve model performance. Python's TensorFlow and PyTorch provide a variety of optimization algorithms and loss functions to train models. For example, for regression tasks, the mean square error (MSE) loss function is used; for classification tasks, the cross entropy loss function is used. During the training process, different optimizers (such as Adam, SGD) can be selected to accelerate convergence. In order to improve the stability of the model, Python can combine ensemble learning (such as random forests, gradient boosting trees) with early stopping strategies to avoid overfitting and ensure the robustness of the model.

[0043] Individual Python scripts usually rely on task scheduling at fixed time points, which cannot automatically adapt to changes in system load, nor can they handle complex task priority and timing issues. Through machine learning and multi-task learning frameworks, the task scheduling system can not only predict execution time based on real-time data, but also intelligently schedule tasks based on factors such as task priority and load prediction, thereby maximizing resource utilization and task completion efficiency. Technologies such as multi-task learning, load prediction, and hyperparameter tuning bring higher prediction accuracy and scheduling efficiency to task scheduling, and can intelligently adjust execution time to avoid human errors and operational errors.

[0044] The present invention designs a multi-task learning framework so that multiple tasks (such as task execution time prediction, task priority classification and load prediction) can be processed simultaneously in the same model. By sharing the feature extraction layer, the model can effectively extract common information from all input features and capture the nonlinear relationship between features, thereby improving learning efficiency and generalization ability. The use of the shared layer not only promotes knowledge sharing between tasks, but also accelerates the training process of the model and reduces computational overhead. Each task has an independent output layer, which ensures that the model can be optimized according to the different properties of the task (regression or classification), further improves the accuracy and specialization of the task, and can improve the effect of the multi-task learning model, so that it can realize resource sharing and optimization between multiple tasks, reduce duplication of work, and improve the training efficiency and accuracy of the model.

[0045] Through the multi-task learning framework, the joint optimization of task execution time prediction and task priority classification is achieved, which improves the efficiency and accuracy of task scheduling. The combination of regression model and classification model enables the model to not only accurately predict the execution time of tasks, but also reasonably evaluate the priority of tasks, thereby realizing intelligent scheduling based on resources and priorities. Through the shared feature layer, the model can extract common input features and perform special optimization according to the needs of different tasks, thereby improving the learning efficiency and generalization ability of the model.

[0046] By combining time series analysis and recursive neural networks (such as LSTM), the time dependency of system load can be effectively captured to provide accurate load forecasting for task scheduling. The LSTM model can learn long-term dependencies in historical load data and predict future load changes, thereby helping the system determine the best time to execute tasks. By introducing sliding windows and historical pattern analysis, the model can dynamically adjust the forecasting strategy and continuously optimize the accuracy of load forecasting. Combined with other features such as task priority and health status, the load forecasting results can provide more accurate input for regression models and classification models, improve the prediction accuracy of task execution time and priority, enable the load forecasting model to efficiently process time series data, and provide a strong decision-making basis for task scheduling, thereby improving the intelligence of task scheduling and the overall performance of the system.

[0047] Through a systematic training process, we ensure that the model can effectively learn historical task data and improve its accuracy and generalization ability in task execution time prediction and priority classification. By reasonably selecting the loss function of regression and classification tasks (such as mean square error and cross entropy loss), and using cross-validation to evaluate the generalization ability of the model, we can effectively prevent overfitting and improve the robustness of the model. In addition, the introduction of technologies such as hyperparameter tuning, integrated learning methods, and early stopping strategies makes the model more efficient during training, able to adapt to different data features, and optimize the performance of the model.

[0048] Finally, it should be noted that the above is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, it is still possible for those skilled in the art to modify the technical solutions described in the aforementioned embodiments or to make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A Python-based automated database operation and maintenance management method, characterized in that: The following steps are involved: Collect and pre-process multiple indicator data related to database operation and maintenance, the indicator data at least including system load, database connection status, disk space, network bandwidth, task execution history, query response time, health status, fault log, task priority, urgency, time sensitivity and business cycle; Based on the collected indicator data, a multi-task learning model is built using Python's deep learning framework, which extracts common information from all input features through a shared feature extraction layer, and sets an independent output layer for each task to optimize task requirements; The constructed multi-task learning model is used to predict task execution time and classify task priorities, taking into account the execution timing and resource allocation of tasks.

2. According to the Python-based automated database operation and maintenance management method of claim 1, it is characterized in that: The following steps are also included: Recurrent neural networks are applied to analyze the time series data of system load to capture long-term dependencies and predict future load changes, thereby providing accurate load prediction for task scheduling.

3. According to the Python-based automated database operation and maintenance management method of claim 2, it is characterized in that: The following steps are also included: The load prediction results are used to dynamically adjust the task execution frequency and timing to implement an intelligent scheduling strategy based on load and priority.

4. The Python-based automated database operation and maintenance management method according to claim 3, characterized in that: The following steps are also included: Implement a systematic training process to ensure that the task learning framework can effectively learn historical task data, and evaluate the model generalization ability by selecting appropriate loss functions and using cross-validation.

5. The Python-based automated database operation and maintenance management method according to claim 4, characterized in that: The following steps are also included: In the process of optimizing the performance of multi-task learning models, hyperparameter tuning, integrated learning methods, and early stopping strategy techniques are introduced to adapt to different data characteristics.

6. The Python-based automated database operation and maintenance management method according to claim 5, characterized in that: The following steps are also included: The multi-task learning model after step optimization is verified using a test dataset to ensure its effectiveness in the actual environment, and the model is continuously optimized based on feedback.

7. A Python-based automated database operation and maintenance management system, characterized in that: include: Data collection module, used to collect multiple indicator data related to database operation and maintenance; Data preprocessing module, used to clean and standardize the collected indicator data; Task learning framework training module, used to train multi-task learning models.

8. The Python-based automated database operation and maintenance management system according to claim 7, characterized in that: Also includes: The intelligent scheduling module is used to realize intelligent scheduling of tasks and maximize resource utilization and task completion efficiency.

9. The Python-based automated database operation and maintenance management system according to claim 8, characterized in that: Also includes: The time series analysis module is used to perform time series analysis on system load to assist in achieving more accurate task scheduling.

10. The Python-based automated database operation and maintenance management system according to claim 9, characterized in that: Also includes: The automated operation and maintenance execution module is used to automatically execute database operation and maintenance tasks, including but not limited to scheduled backup, automated query, batch execution of SQL scripts, and regular cleaning of redundant data.

Images