Implementation method for highly autonomous operation and maintenance of cloud database

Abstract

The invention provides a method for realizing high autonomy of operation and maintenance of a cloud database. The method comprises the following steps: acquiring performance data of the cloud database; according to the performance data, analyzing the performance data based on a pre-constructed deep learning model, and outputting to obtain a prediction result; and formulating a database maintenance strategy based on the prediction result, and executing a maintenance task according to the formulated database maintenance strategy. According to the implementation method for highly autonomous operation and maintenance of the cloud database, the database can autonomously execute the maintenance task without manual intervention, so that the high availability and stability of the database are improved; meanwhile, data integration is performed based on the deep learning model, so that more efficient database management can be realized, manual operation is reduced, and intelligence and autonomy of database management are improved.

Description

Technical Field

[0001] The present application belongs to the field of database technology, and in particular, relates to a method for realizing highly autonomous operation and maintenance of a cloud database. Background Art

[0002] In the modern cloud computing environment, cloud databases have become a core component of enterprise information technology infrastructure. They provide high scalability, flexibility and cost-effectiveness, however, database management remains a complex and critical task. Database administrators need to constantly monitor and maintain databases to ensure their high availability and stability.

[0003] Currently, cloud database management usually requires a lot of manual intervention. Database administrators are responsible for performing tasks such as backup, recovery, performance optimization, and capacity planning. Although cloud platforms provide some automation tools, they are usually based on fixed rules and thresholds and lack intelligent decision-making capabilities. There are some problems and defects in current database management as follows:

[0004] (1) Reliance on manual operations can easily lead to human errors; (2) It is difficult to cope with rapidly changing workloads; (3) It lacks intelligent decision-making capabilities and is unable to perform maintenance tasks autonomously; (4) Backup and recovery operations may be time-consuming and lengthy, affecting business continuity; (5) It is difficult to issue warnings for impending anomalies based on historical database operation and maintenance experience. Summary of the invention

[0005] In view of this, the present application aims to propose a method for implementing highly autonomous cloud database operation and maintenance to solve at least one of the above problems.

[0006] To achieve the above purpose, the technical solution of this application is implemented as follows:

[0007] In a first aspect, the present application provides a method for implementing highly autonomous cloud database operation and maintenance, including:

[0008] Get performance data of cloud database;

[0009] Analyze the performance data based on the pre-built deep learning model and output a prediction result according to the performance data;

[0010] A database maintenance strategy is formulated based on the prediction result, and a maintenance task is performed according to the formulated database maintenance strategy.

[0011] Furthermore, the obtaining of the performance data of the cloud database includes:

[0012] Establish a connection with the cloud database and obtain performance data of the cloud database through an API function, wherein the performance data includes at least load, response time, and capacity utilization.

[0013] Furthermore, the method for constructing the deep learning model includes:

[0014] Preprocess the collected load data, response time data, and capacity utilization data, and integrate the preprocessed data into one data set;

[0015] The preprocessed data was subjected to feature selection using the Pearson correlation coefficient;

[0016] Perform model training on the deep learning model, where the model training is based on historical data sets, including various performance data as input variables and corresponding database states as output variables;

[0017] The performance analysis and prediction of the trained model are performed through cross-validation.

[0018] Further, the database status includes at least identifying potential performance bottlenecks, capacity shortages or failures.

[0019] Furthermore, formulating a database maintenance strategy based on the prediction result, and executing a maintenance task according to the formulated database maintenance strategy, includes:

[0020] Performing data analysis on the prediction results output by the deep learning model and classifying the data into different problem types;

[0021] According to the problem type obtained by the analysis, a corresponding maintenance strategy is formulated based on a preset logic, wherein the maintenance strategy at least includes data backup, data recovery, and capacity planning;

[0022] The execution engine automatically executes maintenance tasks, which at least include data backup, data migration, and performance optimization.

[0023] Furthermore, it also includes:

[0024] The maintenance tasks are monitored in real time, and the feedback information is passed to the deep learning model for periodic fine-tuning to optimize the prediction results and maintenance strategies.

[0025] In the second aspect, based on the same inventive concept, the present application also provides a cloud database operation and maintenance management system, including:

[0026] A data collection module, configured to collect performance data of the cloud database in real time;

[0027] A data integration module configured to perform integrated analysis and prediction on performance data;

[0028] a decision engine, configured to formulate database maintenance strategies;

[0029] The execution engine is configured to execute the maintenance task according to the formulated database maintenance strategy.

[0030] In a third aspect, based on the same inventive concept, the present application also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method described in the first aspect is implemented.

[0031] In a fourth aspect, based on the same inventive concept, the present application also provides a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the method described in the first aspect.

[0032] Compared with the prior art, the method for implementing highly autonomous cloud database operation and maintenance described in this application has the following beneficial effects:

[0033] The present application discloses a method for realizing highly autonomous cloud database operation and maintenance, wherein the method enables the database to autonomously perform maintenance tasks without human intervention, thereby improving the high availability and stability of the database. At the same time, data integration based on deep learning models can achieve more efficient database management, reduce manual operations, and improve the intelligence and autonomy of database management. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] The drawings constituting a part of the present application are used to provide a further understanding of the present application. The illustrative embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0035] Figure 1 A flow chart of a method for implementing highly autonomous cloud database operation and maintenance as described in an embodiment of the present application;

[0036] Figure 2 This is a schematic diagram of the structure of a cloud database operation and maintenance management system described in an embodiment of the present application;

[0037] Figure 3 This is an architecture diagram of a cloud database operation and maintenance management system described in an embodiment of the present application;

[0038] Figure 4 This is a schematic diagram of the hardware structure of the electronic device described in the embodiment of the present application. DETAILED DESCRIPTION

[0039] In order to make the objectives, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below in combination with specific embodiments and with reference to the accompanying drawings.

[0040] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present application should be the usual meanings understood by people with ordinary skills in the field to which the present application belongs. The "first", "second" and similar words used in the embodiments of the present application do not represent any order, quantity or importance, but are only used to distinguish different components. "Including" or "comprising" and similar words mean that the elements or objects appearing in front of the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0041] The embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0042] See also Figure 1 As shown, this embodiment provides a method for implementing highly autonomous cloud database operation and maintenance, including:

[0043] Step S101: Obtain performance data of the cloud database.

[0044] Specifically, in this embodiment, before using the cloud database API to obtain performance data, first establish a connection with the cloud database (set the database URL, authentication information, etc.), and after establishing the connection, use the API function to obtain the performance and status data of the cloud database, including load, response time, capacity utilization, I / O operation speed and other data.

[0045] Step S102: Analyze the performance data based on the pre-built deep learning model and output a prediction result.

[0046] Specifically, in this embodiment, the deep learning model adopts the LLAMA2 model as an example for explanation. Then, based on the LLAMA2 model architecture, the training method of the model includes:

[0047] Step S201: pre-process the collected load data, response time data, and capacity utilization data, and integrate the pre-processed data into one data set.

[0048] Specifically, the collected load data, response time data, and capacity utilization data are first preprocessed, including data cleaning, normalization and other operations. The preprocessed data are integrated to construct a data set, and the data set is randomly divided into training set, test set and validation set to train the LLAMA2 model.

[0049] Step S202: performing feature selection on the preprocessed data using the Pearson correlation coefficient.

[0050] Based on the preprocessed performance data, feature selection is performed using the Pearson correlation coefficient. For load data, response time, and capacity utilization, these features are directly related to the performance of the database, so the above data are selected for feature selection.

[0051] The goal of feature selection is to identify indicators that predict performance bottlenecks, response time delays, and other issues.

[0052] Step S203: perform model training on the deep learning model, wherein the model training is based on a historical data set, including various types of performance data as input variables, and corresponding database status as output variables, wherein the database status at least includes identifying potential performance bottlenecks, insufficient capacity, or failures.

[0053] Using the LLAMA2 model framework, the model is trained through the constructed historical data set, and the prediction results about the database status are output to complete the training of the model.

[0054] Step S204: Perform performance analysis and prediction on the trained model through cross-validation.

[0055] After training is completed, the prediction performance of the model is evaluated through cross-validation. According to the evaluation results, the model parameters are adjusted and the model structure is optimized to ensure that the model has high accuracy and generalization ability.

[0056] Step S103: formulate a database maintenance strategy based on the prediction result, and execute the maintenance task according to the formulated database maintenance strategy.

[0057] Specifically, in this embodiment, first, the prediction outputs are obtained from the LLAMA2 model, which include specific information about database performance bottlenecks, insufficient capacity, or potential failures. These data are parsed and classified into different problem types, such as capacity problems, performance problems, etc.

[0058] According to the analyzed problem type, the decision engine will formulate corresponding maintenance strategies according to the preset logic. For example:

[0059] If the forecast result shows "insufficient capacity", the decision is to increase storage capacity or optimize data distribution.

[0060] If the prediction results point to a "performance bottleneck", the decision is to perform query optimization, index adjustment, or hardware upgrade.

[0061] Each maintenance strategy is accompanied by specific operational steps and expected goals to ensure the specificity and enforceability of the decision.

[0062] The execution engine automatically performs various maintenance tasks according to the instructions of the decision engine, including at least:

[0063] Data backup: Automatically perform full or incremental backup of data to ensure data security.

[0064] Performance optimization: Automatically apply query optimization, reconfigure system parameters, or adjust index structure.

[0065] Capacity adjustment: Automatically expand the storage system or optimize the existing storage structure.

[0066] Data migration: Migrate data externally to ensure stable operation of the database.

[0067] In some embodiments, it further comprises:

[0068] Maintenance tasks are monitored in real time, and feedback information is passed to the deep learning model for periodic fine-tuning to optimize prediction results and maintenance strategies.

[0069] Specifically, in this embodiment, during the execution of the maintenance task, the execution results are monitored in real time, and if problems occur, corrective measures are taken immediately. In addition, the collected feedback information will be used to periodically fine-tune the LLAMA2 model, which helps to continuously optimize the prediction accuracy and decision quality of the model.

[0070] The method described in this embodiment enables the database to autonomously perform maintenance tasks without manual intervention, thereby improving the high availability and stability of the database. Data integration based on deep learning models can achieve more efficient database management, reduce manual operations, and improve the intelligence and autonomy of database management, which will help meet rapidly changing business needs and improve the efficiency and reliability of database management.

[0071] The specific implementation of the above solution is based on the Python language, which requires the driver connection package of the cloud database, the running package of the large model Llama2, the dependency package of langchain, etc., as follows:

[0072] 1. Data Collection

[0073] Data collection can use the monitoring API provided by the cloud database to obtain performance data. The following is a sample code snippet that demonstrates how to use the cloud database API to collect performance data:

[0074] import cloud_database_api

[0075] #Connect to the cloud database

[0076] db_connection=cloud_database_api.connect(database_url)

[0077] #Get performance data

[0078] performance_data=db_connection.get_performance_data()

[0079] 2. Data integration of LLAMA2 model

[0080] Integration of the LLAMA2 model can be achieved through API calls. The following is a sample code snippet that demonstrates how to use LLAMA2 to analyze performance data and generate predictions:

[0081] import llama2_api

[0082] #Connect to LLAMA2

[0083] llama2_connection=llama2_api.connect(llama2_url)

[0084] #Analyze performance data

[0085] performance_data=get_performance_data()

[0086] predictions=llama2_connection.analyze_performance(performance_data)

[0087] 3. Decision engine

[0088] The decision engine makes maintenance decisions based on the predictions from LLAMA2. Here is a sample code snippet that demonstrates how to make decisions based on the output of LLAMA2:

[0089] # Make decisions based on LLAMA2 predictions

[0090] if predictions["potential_issue"]=="capacity":

[0091]

[0092] 4. Execution Engine

[0093] The execution engine is responsible for performing maintenance tasks. The following is a sample code snippet that demonstrates how to perform maintenance tasks:

[0094]

[0095] The pseudo code above demonstrates how the key components of the solution actually work in a typical scenario. The specific implementation will vary depending on the type of cloud database engine and the specific integration method of the LLAMA2 model.

[0096] It should be noted that the above describes some embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the above embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0097] Based on the same inventive concept, corresponding to any of the above-mentioned embodiments and methods, the embodiments of the present application also provide an automated operation and maintenance management system for a cloud database, focusing on solving autonomous management of cloud database data backup, abnormal warning, etc., and integrating the open source AIGC large model LLAMA2 to improve the autonomous decision-making ability of the cloud database. Figure 2 As shown, the system will have the following core components:

[0098] Data collection module 11: configured to collect cloud database performance and status data in real time;

[0099] Data integration module 12: embed the LLAMA2 model into the system for integrated analysis and prediction of performance data;

[0100] Decision engine 13: Formulate database maintenance strategy based on the output of LLAMA2;

[0101] Execution engine 14: configured to execute maintenance tasks, including backup, recovery, capacity planning and other maintenance tasks.

[0102] Monitoring feedback module 15: is configured to monitor the system execution results in real time and pass feedback information to the LLAMA2 model for periodic fine-tuning to continuously improve prediction and decision-making.

[0103] The specific working process is:

[0104] 1. Data collection: The system regularly collects performance data of the cloud database, including load, response time, capacity utilization, etc.

[0105] 2. Data integration: The LLAMA2 model analyzes performance data to predict possible problems and needs of the database. This can include identifying potential performance bottlenecks, insufficient capacity, or failures.

[0106] 3. Decision making: The decision engine makes database maintenance decisions, such as backup, recovery, and capacity adjustment, based on the output of LLAMA2.

[0107] 4. Execute maintenance tasks: The execution engine automatically executes decisions, including tasks such as backup, data migration, and performance optimization.

[0108] 5. Monitoring and feedback: Monitor the execution results in real time and take corrective measures immediately if problems occur. At the same time, pass feedback information to LLAMA2 to improve the accuracy of the model.

[0109] The system has the following advantages:

[0110] 1. Autonomous decision-making: The system can make autonomous decisions based on LLAMA2’s predictions without human intervention.

[0111] 2. Improve efficiency: Automated operation and maintenance reduces manual operations and improves the efficiency of database management.

[0112] 3. High availability: Quickly respond to problems and reduce the impact of database failures on business.

[0113] 4. Intelligent decision-making: The LLAMA2 model provides the ability to make intelligent decisions, making database management more intelligent.

[0114] like Figure 3 As shown in the figure, this architecture diagram illustrates the main modules that the system relies on and the collaborative relationship between the modules. The core capabilities are mainly composed of the following parts:

[0115] Knowledge base capabilities: Knowledge base question-and-answer capabilities that support database operations and maintenance.

[0116] Large model management capability: Provides a large model operating environment based on LLAMA2.

[0117] Unified data vector storage and indexing: Provides a unified way to store and index various data types.

[0118] Connection module: used to connect different modules and data sources to achieve data flow and interaction.

[0119] Agents and plug-ins: Provide agent and plug-in mechanisms to allow users to customize and enhance the behavior of the system.

[0120] Prompt generation and optimization: Automatically generate high-quality prompts and optimize them to improve system response efficiency.

[0121] Multi-platform product interface: supports multiple client products such as Web, mobile applications, and desktop applications.

[0122] For the convenience of description, the above system is described by dividing the functions into various modules. Of course, when implementing the embodiments of the present application, the functions of each module can be implemented in the same or multiple software and / or hardware.

[0123] The system of the above embodiment is used to implement the corresponding method in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be repeated here.

[0124] Based on the same inventive concept, corresponding to any of the above-mentioned embodiments and methods, an embodiment of the present application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method described in any of the above embodiments is implemented.

[0125] Figure 4 A more specific schematic diagram of the hardware structure of an electronic device provided in this embodiment is shown, and the device may include: a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040 are connected to each other through the bus 1050 in the device.

[0126] The processor 1010 can be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0127] The memory 1020 may be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1020 may store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program codes are stored in the memory 1020 and are called and executed by the processor 1010.

[0128] The input / output interface 1030 is used to connect the input / output module to realize information input and output. The input / output module can be configured in the device as a component (not shown in the figure), or it can be externally connected to the device to provide corresponding functions. The input device may include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output device may include a display, a speaker, a vibrator, an indicator light, etc.

[0129] The communication interface 1040 is used to connect a communication module (not shown) to realize communication interaction between the device and other devices. The communication module can realize communication through a wired mode (such as USB, network cable, etc.) or a wireless mode (such as mobile network, WIFI, Bluetooth, etc.).

[0130] The bus 1050 includes a path that transmits information between the various components of the device (eg, the processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040).

[0131] It should be noted that, although the above device only shows the processor 1010, the memory 1020, the input / output interface 1030, the communication interface 1040 and the bus 1050, in the specific implementation process, the device may also include other components necessary for normal operation. In addition, it can be understood by those skilled in the art that the above device may also only include the components necessary for implementing the embodiments of the present specification, and does not necessarily include all the components shown in the figure.

[0132] The electronic device of the above embodiment is used to implement the corresponding method in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be described in detail here.

[0133] Based on the same inventive concept, corresponding to any of the above-mentioned embodiments, the present application also provides a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the method described in any of the above embodiments.

[0134] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.

[0135] The computer instructions stored in the storage medium of the above embodiments are used to enable the computer to execute the method described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0136] Those skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the present application (including the claims) is limited to these examples. In line with the concept of the present application, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of the different aspects of the embodiments of the present application as described above, which are not provided in detail for the sake of simplicity.

[0137] In addition, to simplify the description and discussion, and in order not to make the embodiments of the present application difficult to understand, the known power supply / ground connection with the integrated circuit (IC) chip and other components may or may not be shown in the provided drawings. In addition, the device can be shown in the form of a block diagram to avoid making the embodiments of the present application difficult to understand, and this also takes into account the fact that the details of the implementation of these block diagram devices are highly dependent on the platform to be implemented in the embodiments of the present application (that is, these details should be fully within the scope of understanding of those skilled in the art). In the case of elaborating specific details (e.g., circuits) to describe exemplary embodiments of the present application, it is obvious to those skilled in the art that the embodiments of the present application can be implemented without these specific details or when these specific details are changed. Therefore, these descriptions should be considered to be illustrative rather than restrictive.

[0138] Although the present application has been described in conjunction with specific embodiments of the present application, many replacements, modifications and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may use the embodiments discussed.

[0139] The embodiments of the present application are intended to cover all such substitutions, modifications and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present application should be included in the scope of protection of the present application.

Claims

1. A method for realizing highly autonomous operation and maintenance of a cloud database, characterized in that: include: Get performance data of cloud database; Analyze the performance data based on the pre-built deep learning model and output a prediction result according to the performance data; A database maintenance strategy is formulated based on the prediction result, and a maintenance task is performed according to the formulated database maintenance strategy.

2. The method according to claim 1, characterized in that The obtaining of the performance data of the cloud database includes: Establish a connection with the cloud database and obtain performance data of the cloud database through an API function, wherein the performance data includes at least load, response time, and capacity utilization.

3. The method according to claim 2, characterized in that The method for constructing the deep learning model includes: Preprocess the collected load data, response time data, and capacity utilization data, and integrate the preprocessed data into one data set; The preprocessed data was subjected to feature selection using the Pearson correlation coefficient; Perform model training on the deep learning model, where the model training is based on historical data sets, including various performance data as input variables and corresponding database states as output variables; The performance analysis and prediction of the trained model are performed through cross-validation.

4. The method according to claim 3, characterized in that: The database status includes at least identifying potential performance bottlenecks, capacity shortages, or failures.

5. The method according to claim 1, characterized in that The formulating a database maintenance strategy based on the prediction result, and executing the maintenance task according to the formulated database maintenance strategy, includes: Performing data analysis on the prediction results output by the deep learning model and classifying the data into different problem types; According to the problem type obtained by the analysis, a corresponding maintenance strategy is formulated based on a preset logic, wherein the maintenance strategy at least includes data backup, data recovery, and capacity planning; The execution engine automates maintenance tasks.

6. The method according to claim 1, characterized in that Also includes: The maintenance tasks are monitored in real time, and the feedback information is passed to the deep learning model for periodic fine-tuning to optimize the prediction results and maintenance strategies.

7. A cloud database operation and maintenance management system, characterized in that: include: A data collection module, configured to collect performance data of the cloud database in real time; A data integration module configured to perform integrated analysis and prediction on performance data; a decision engine, configured to formulate database maintenance strategies; The execution engine is configured to execute the maintenance task according to the formulated database maintenance strategy.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the method according to any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium, characterized in that: in, The non-transitory computer-readable storage medium stores computer instructions, and the computer instructions are used to cause a computer to execute the method according to any one of claims 1 to 6.

Images