Database autonomous service processing method and device based on machine learning, and terminal

Abstract

The invention discloses a database autonomous service processing method and device based on machine learning, a terminal and a medium, and belongs to the technical field of information. Analyzing and modeling the collected operation data by using the trained machine learning engine model, predicting and analyzing the database state, and outputting a prediction and analysis result; empirical knowledge and practical knowledge of database experts are obtained and stored as an expert empirical library for providing decision support for the system; according to the prediction analysis result of the machine learning engine, a specific execution plan is automatically generated in combination with corresponding comprehensive decision suggestions generated by an expert experience library, and corresponding autonomous operation is automatically executed according to the generated execution plan; wherein the autonomous operation comprises performance optimization, fault repair and safety protection. According to the method, self-sensing, self-repairing, self-optimizing, self-operation and self-safety of database faults can be achieved, and stability, safety and high efficiency of database services are effectively guaranteed.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of information technology, and in particular to a database autonomous service processing method and device based on machine learning, a server and a storage medium. BACKGROUND

[0002] With the rapid development of information technology, databases have been widely used in various fields. However, the management of the existing database technology faces many problems, such as inconvenient manual operation, easy to cause service failure, high management cost, and high security risk.

[0003] Therefore, the prior art still needs to be improved and developed. SUMMARY

[0004] In order to solve the above technical problems, the present application provides a database autonomous service processing method and device based on machine learning, a server and a storage medium, which innovatively solves the self-sensing, self-repairing, self-optimizing, self-operation and maintenance and self-security of database faults, and effectively guarantees the stability, security and efficiency of database service.

[0005] A database autonomous service processing method based on machine learning, comprising: Collecting the running data of the database, wherein the running data includes performance index running data, SQL statement data, transaction information data and log data; Using the trained machine learning engine model to analyze and model the collected running data, predicting and analyzing the database state, and outputting the prediction analysis result; Obtaining the experience and practical knowledge of database experts, and storing the expert experience library as a decision support system; According to the prediction analysis result of the machine learning engine and the corresponding comprehensive decision suggestion generated by the expert experience library, a specific execution plan is automatically generated, and the corresponding autonomous operation is automatically executed according to the generated execution plan; wherein the autonomous operation includes performance optimization, fault repair and security protection.

[0006] The database autonomous service processing method based on machine learning, wherein the step of automatically executing the corresponding autonomous operation according to the generated execution plan comprises: Real-time monitoring and evaluating the effect of the autonomous operation, and continuously optimizing the system performance according to the feedback information.

[0007] The database autonomous service processing method based on machine learning, wherein the step of collecting the running data of the database comprises: Identifying and connecting the target database instance, and establishing a communication link for data collection; According to a preset collection strategy, periodically or in real time, the running data of the database is collected and acquired; The collected running data is preprocessed to obtain preprocessed running data; the preprocessing includes formatting operation, cleaning operation; The preprocessed running data is stored in a temporary buffer.

[0008] The database autonomous service processing method based on machine learning, wherein the step of collecting the running data and using the trained machine learning engine model to analyze and model the running data to predict and analyze the database state and output the prediction analysis result comprises: Obtain the processed running data from the temporary buffer; According to the data characteristics of the running data, automatically select the corresponding machine learning engine model; Use the historical running data of the database to train and verify the selected machine learning engine model, optimize the model parameters, and obtain the trained machine learning engine model; Use the trained machine learning engine model to extract features and reduce dimensions of the obtained processed running data, and predict and analyze the current database state to generate a prediction analysis report.

[0009] The database autonomous service processing method based on machine learning, wherein the step of obtaining the experience and practical knowledge of database experts and storing the expert experience library as decision support provided by the system comprises: Collect and organize the experience and practical knowledge of database experts to form a structured knowledge base; Classify and index the knowledge base to store the expert experience library as decision support provided by the system.

[0010] The database autonomous service processing method based on machine learning, wherein the step of generating corresponding comprehensive decision suggestions according to the prediction analysis results of the machine learning engine and combining the expert experience library to automatically generate specific execution plans and automatically execute corresponding autonomous operations according to the generated execution plans comprises: According to the prediction analysis results of the machine learning engine, retrieve the corresponding relevant expert experience from the expert experience library; Fuse the retrieved expert experience with the prediction analysis results to generate corresponding comprehensive decision suggestions; According to the prediction analysis results of the machine learning engine, combine the expert experience library to generate corresponding comprehensive decision suggestions, automatically generate specific execution plans, and the execution plans include: SQL optimization script, configuration modification instruction; Before execution, the safety and feasibility of the execution plan are verified; The corresponding autonomous operation is automatically executed according to the verified execution plan, and the database state in the execution process is monitored in real time; When the execution plan is executed, an operation log is recorded for subsequent auditing and analysis.

[0011] The method for processing the autonomous service of the database based on machine learning, wherein the step of monitoring and evaluating the effect of the autonomous operation in real time and continuously optimizing the system performance according to the feedback information comprises: A monitoring index system is established in advance, and key performance indicators and security indicators are defined; The database running data after the corresponding autonomous operation is automatically executed according to the execution plan is collected in real time, and is compared and analyzed with the running data before the execution plan is executed; The operation effect is evaluated according to the comparison and analysis result, and an evaluation result is generated, wherein the operation effect includes the performance improvement amplitude and the fault repair situation; The evaluation result is fed back to the machine learning engine model and the expert experience library; The parameters and strategies of the machine learning engine model and the expert experience library are adjusted according to the feedback evaluation result, and the optimization of the machine learning engine model and the expert experience library is automatically performed.

[0012] An apparatus for processing the autonomous service of the database based on machine learning, wherein the apparatus comprises: A data collection module for collecting the running data of the database, wherein the running data includes performance indicator running data, SQL statement data, transaction information data, and log data; A machine learning engine prediction analysis module for analyzing and modeling the collected running data by using the trained machine learning engine model, predicting and analyzing the database state, and outputting a prediction analysis result; An expert experience library generation module for obtaining the experience knowledge and practical knowledge of database experts, and storing the expert experience library as decision support provided by the system; An autonomous execution module for generating a specific execution plan according to the corresponding comprehensive decision suggestion generated by the machine learning engine prediction analysis result in combination with the expert experience library, and automatically executing the corresponding autonomous operation according to the generated execution plan, wherein the autonomous operation includes performance optimization, fault repair, and security protection; A monitoring and evaluation module for monitoring and evaluating the effect of the autonomous operation in real time, and continuously optimizing the system performance according to the feedback information.

[0013] A server, comprising a memory and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by one or more processors, and the one or more programs comprise programs for executing any one of the methods.

[0014] A computer readable storage medium, wherein when instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute any one of the methods.

[0015] As can be seen, the application provides a database autonomous service processing method and device based on machine learning, a server and a storage medium, which can realize self-sensing, self-repairing, self-optimizing, self-operation and maintenance and self-security of database faults, and effectively guarantee the stability, security and efficiency of database services. BRIEF DESCRIPTION OF DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only some embodiments described in the present application, and for those skilled in the art, other drawings can also be obtained without creative labor on the basis of these drawings.

[0017] Figure 1 is a flowchart of the database autonomous service processing method based on machine learning of the embodiment 1 of the present application.

[0018] Figure 2 is a flowchart of the database autonomous service processing method based on machine learning of the embodiment 2 of the present application.

[0019] Figure 3 The principle block diagram of the database autonomous service processing device based on machine learning provided by the present application is provided.

[0020] Figure 4 is the internal structure principle block diagram of the server provided by the embodiment of the present application. DETAILED DESCRIPTION

[0021] In order to make the purpose, technical scheme and advantages of the present application more clear and definite, the following will further describe the present application in detail with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, and are not used to limit the present application.

[0022] It should be noted that if the embodiments of the present application involve directionality indication (such as up, down, left, right, front, back, …), the directionality indication is only used to explain the relative position relationship, motion condition, etc. between components in a certain specific posture (as shown in the drawings), if the specific posture changes, the directionality indication also changes accordingly.

[0023] With the rapid development of information technology, databases have been widely used in various fields. However, the management of the existing database technology faces many problems, such as inconvenient manual operation, easy to cause service failure, high management cost, and high security risk.

[0024] Embodiment one As shown in the figure, the database autonomous service processing method based on machine learning in the embodiment of the present application comprises the following steps: Figure 1 Step S100, collecting the running data of the database, wherein the running data includes performance index running data, SQL statement data, transaction information data, and log data; In the embodiment of the present application, the running data of the database refers to various information and records generated in the running process of the database, reflecting the working state and operation condition of the database. The performance index running data is the key data for measuring the performance of the database, such as response time, throughput, CPU usage, memory occupation, disk I / O rate, etc., which can reflect the running efficiency and resource consumption of the database. The SQL statement data is the structured query language (SQL) statement submitted by the user or application to the database, including the statement content and execution condition of the query, insertion, update, deletion, etc.

[0025] The transaction information data is the related information of the transaction in the database, such as the start and end time of the transaction, transaction ID, transaction operation object, transaction commit or rollback state, etc. The transaction is the basic unit of database operation.

[0026] The log data is various log information recorded in the running process of the database, including error log (records the errors occurred in the database), binary log (records the data modification operation, used for data recovery and replication), query log (records all executed SQL statements), etc.

[0027]

[0028] ​In the embodiment of the present step, various types of relevant data generated in the running process of the database are collected, specifically including key indicator data for measuring the performance of the database, SQL statements submitted by users or application programs and their execution data, relevant information data of transactions in the database, and various types of log information data recorded by the database during running. Through the collection of these data, data support can be provided for the analysis of the running state, performance bottleneck, and operation of the database in the subsequent steps, and basis can be provided for the optimization, maintenance, and troubleshooting of the database.

[0029] The step S100 specifically includes: S101, identify and connect the target database instance, and establish a communication link for data collection; In the embodiment of the present step, the target database instance refers to a specific database entity that needs to be collected, including specific data, table structure, configuration, etc. The communication link refers to a channel for transmitting data between two or more devices (here, the data collection end and the target database instance), which ensures stable data transmission between the two.

[0030] In the embodiment of the present step, the specific database instance that needs to be collected is first determined, and then connected with the instance through technical means, finally forming a channel that can stably transmit data, providing basic communication guarantee for subsequent data collection.

[0031] S102, according to the preset collection strategy, periodically or in real time collect and obtain the running data of the database; In the embodiment of the present step, the preset collection strategy refers to the rules and methods of data collection set in advance, including the time frequency of collection such as once an hour or real-time, the data range of collection, the trigger condition of collection, etc.

[0032] In the embodiment of the present step, according to the collection rules (such as collection frequency, range, etc.) set in advance, the periodic or real-time method is selected to collect various types of data generated in the running process of the target database instance that has been connected.

[0033] S103, pre-process the collected running data to obtain pre-processed running data; the pre-processing includes formatting operation and cleaning operation; In the embodiment of the present step, the pre-processing refers to the process of preliminary processing of the original data, the purpose of which is to make the data more standardized and accurate, and facilitate subsequent storage and analysis. The formatting operation of the pre-processing is to convert the collected data in different formats such as text and binary into a unified format such as JSON and CSV, ensuring consistent data structure. The cleaning operation is to remove or correct invalid information such as errors, duplicates, and missing information in the collected data, such as deleting duplicate SQL statement records and supplementing missing timestamps, to improve data quality.

[0034] In the embodiment of the present step, the original running data collected from the database is preliminarily processed, specifically including a formatting operation of converting the data into a unified format and a cleaning operation of removing invalid information, and finally obtaining standardized and accurate preprocessed data, thereby preparing for subsequent processing and analysis.

[0035] S104, store the preprocessed running data into a temporary buffer.

[0036] In the embodiment of the present application, the temporary buffer refers to a memory or a storage area for temporarily storing data.

[0037] In the embodiment of the present step, the standardized and accurate running data after preprocessing is temporarily stored in a special temporary storage area such as a temporary buffer, which facilitates subsequent analysis and extraction, and can reduce the pressure on the system caused by direct storage.

[0038] Step S200, using the trained machine learning engine model to analyze and model the collected running data, to predict and analyze the database state, and output the prediction analysis result; In the embodiment, the trained machine learning engine model refers to a computer model that can autonomously learn and analyze data rules after being trained by a large amount of historical running data. The database state refers to the running condition of the database at a certain time, including performance such as response speed, resource occupation, stability such as whether to freeze, whether to have errors, load such as concurrent request amount, and other state characteristics.

[0039] The prediction analysis result refers to the conclusive information about the future state of the database, such as whether performance degradation, fault risk, or current hidden problems such as potential bottlenecks may occur, obtained after model analysis.

[0040] In the embodiment of the present step, the collected database running data is input into the trained machine learning engine model, and the model analyzes the data and models the rules to judge the current state of the database and predict its future running trend, and finally outputs the results including database state evaluation, potential problem warning or trend judgment. With the help of the ability of machine learning, the hidden information in the data is more efficiently mined, which provides support for the optimization and fault prevention of the database.

[0041] Further, the step S200 specifically includes: S201, obtaining the processed running data from the temporary buffer; In the embodiment, the processed running data refers to the database running data after preprocessing operations such as formatting and cleaning, which has standardization and accuracy.

[0042] In this step, the pre-processed database running data is extracted from the temporary storage area, i.e., the temporary buffer, to provide high-quality input data for subsequent machine learning analysis.

[0043] S202, automatically selecting a corresponding machine learning engine model according to the data characteristics of the running data; In this step, the data characteristics refer to the attributes or characteristics of the running data itself, such as data type (including numerical type, text type), structure (including structured, unstructured), dimension (including the number of columns or the number of features of the data), distribution rule, etc. The machine learning engine model refers to a model that can autonomously learn data rules and perform analysis and prediction. Different models are suitable for different types of data or analysis targets, such as classification models for judging state types and regression models for predicting numerical trends.

[0044] In this step, according to the attributes of the running data to be analyzed, such as data type and structure, the machine learning engine model most suitable for processing such data is selected through an automated mechanism to improve the accuracy of analysis and prediction.

[0045] S203, training and verifying the selected machine learning engine model using historical running data of the database, optimizing model parameters, and obtaining a trained machine learning engine model; In this embodiment, the historical running data is running data generated and saved during the past running of the database, containing historical performance, operation records, and other information.

[0046] In this step, the historical running data accumulated by the database in the past is used to train the selected machine learning engine model, i.e., to let the model learn the data rules; and to verify the accuracy of the model, to optimize the performance of the model by adjusting the model parameters, and finally to obtain a trained model that can stably and accurately analyze data.

[0047] The model parameters in this embodiment are adjustable variables inside the model, which determine the calculation logic and output results of the model. Optimizing the parameters can improve the performance of the model.

[0048] S204, using the trained machine learning engine model to perform feature extraction and dimension reduction on the obtained processed running data, and predicting and analyzing the current database state to generate a prediction analysis report.

[0049] In this step, the processed running data is input into the trained machine learning engine model, and the data is first simplified through feature extraction and dimension reduction, and then analyzed by the model to predict the current and future state of the database, and finally a prediction analysis report containing state evaluation, trend prediction, etc. is generated, providing a decision basis for database management.

[0050] The feature extraction is to extract key information valuable for analysis from the original data, such as extracting core features such as execution frequency, time consumption, etc. from a large number of SQL statements, simplifying the data while retaining important rules. The dimension reduction is to reduce the dimension of the data, such as deleting redundant feature columns, to reduce the complexity of the data and improve the efficiency of model analysis, while avoiding analysis bias caused by excessive dimension. The current database state refers to the current running situation of the database, including performance, stability, load, etc.

[0051] The generated prediction analysis report is a document integrating the model analysis results, including the evaluation of the current state of the database, the prediction of future trends such as potential failure risks, and key conclusions.

[0052] Step S300, acquire the experience and practical knowledge of database experts, and store the expert experience library as the decision support of the system; In the embodiment of the present step, the experience and practical knowledge of database experts accumulated in long-term work are collected, and after the knowledge is sorted and stored, an expert experience library is formed. The role of this library is to provide professional knowledge basis for the subsequent decision support of the system, and to help the system to analyze and judge more in line with the actual scene.

[0053] The experience knowledge of database experts refers to the understanding and judgment accumulated by database experts in long-term practice and based on experience, such as the rule that a certain type of error log usually corresponds to a memory overflow problem. The practical knowledge of database experts refers to the methods, skills and processes mastered by experts in the process of actually operating the database, such as the specific steps of optimizing database indexes in high-concurrency scenarios, practical solutions for fault recovery, and other knowledge that can be directly applied to practice.

[0054] Further, the step S300 includes: S301, collect and sort the experience and practical knowledge of database experts, and form a structured knowledge base; In the embodiment of the present step, the experience and practical knowledge of database experts are collected, and after being sorted, they are made systematic and standardized, and finally a knowledge base with clear structure and convenient management is formed, laying a foundation for subsequent knowledge application. The structured knowledge base is an ordered collection formed by sorting scattered knowledge according to certain rules such as classification and format standard, which is convenient for storage, query and use, for example, classified by fault handling and performance optimization, and each piece of knowledge contains fixed fields such as problem description and solution method.

[0055] S302, classify and index the knowledge base, and store the expert experience library as the decision support of the system.

[0056] In the embodiments of the present application, the formed structured knowledge base is classified, arranged and indexed, so that it is easier to query and call, and then it is stored as an expert experience library. The core role of this library is to provide professional knowledge support for the subsequent decision-making process of the system, helping the system to more accurately analyze problems and develop solutions. Among them, the classification of the knowledge base is to divide the content in the knowledge base into different categories according to the attributes or uses of the knowledge, such as classifying by fault type and optimization scenario, which facilitates quick positioning of related knowledge. The index is to establish a retrieval identifier such as a keyword, a label for the content in the knowledge base, similar to a book index, which can greatly improve the query efficiency of the knowledge.

[0057] In step S400, a specific execution plan is automatically generated according to the prediction analysis result of the machine learning engine and the corresponding comprehensive decision suggestion generated by the expert experience library, and the corresponding autonomous operation is automatically executed according to the generated execution plan; wherein the autonomous operation includes performance optimization, fault repair, and security protection.

[0058] In the embodiments of the present application, the comprehensive decision suggestion is a comprehensive and feasible decision reference formed by combining the prediction analysis result (such as performance bottleneck warning and fault risk prompt) output by the machine learning engine and the professional knowledge (such as historical solutions and best practices) in the expert experience library, which takes into account data-driven analysis and artificially accumulated experience.

[0059] The execution plan is a specific operation step and process formulated according to the comprehensive decision suggestion, which clearly specifies what to do, how to do it, when to do it, etc., and is a specific scheme for converting the decision suggestion into actual action.

[0060] Among them, the autonomous operation adopted by the present application is a database management operation automatically executed by the system without human intervention, aiming to realize self-management and maintenance of the database.

[0061] In the embodiments of the present application, the system first generates comprehensive decision suggestions by combining the prediction analysis result of the machine learning engine and the professional knowledge provided by the expert experience library; then automatically formulates specific operation steps, i.e. specific execution plans, according to these suggestions; finally, according to the execution plan, automatically completes the autonomous operation of the database, including performance optimization, fault repair, security protection, etc., realizes the automated management and maintenance of the database, and reduces human intervention.

[0062] Among them, performance optimization refers to improving the running efficiency of the database by adjusting database parameters, optimizing SQL statements, adjusting indexes, etc.; and fault repair is to automatically take repair measures (such as restarting the service, rolling back the transaction, and restoring the backup data) for the faults (such as deadlock, connection timeout, and data damage) of the database, so as to make the database resume normal operation.

[0063] Further in the embodiment, the step S400 specifically comprises: S401, retrieving corresponding relevant expert experience from the expert experience library according to the prediction analysis result of the machine learning engine; In the embodiment, the prediction analysis result is the analysis and prediction of the machine learning engine on the database state, trend, etc., such as that the CPU usage will exceed the threshold within 1 hour, there is a risk of deadlock, etc.

[0064] In the embodiment, according to the prediction analysis result (such as problem warning, state judgment) output by the machine learning engine, the relevant expert experience knowledge (such as the historical solution to the corresponding problem, the processing idea) is searched from the expert experience library, providing experience reference for subsequent decision-making.

[0065] S402, fusing the retrieved expert experience and the prediction analysis result to generate corresponding comprehensive decision-making suggestions; In the embodiment, the relevant experience found from the expert experience library is combined with the prediction analysis result of the machine learning engine, and the information of the two is integrated to form a comprehensive decision-making suggestion, so that the suggestion conforms to the data rule and is suitable for the actual operation scene.

[0066] Among them, the fusion refers to combining the prediction analysis result (such as data-driven conclusion) of the machine learning and the expert experience (such as artificially accumulated knowledge) to form a unified and complementary analysis conclusion, avoiding the limitation of a single information source. The comprehensive decision-making suggestion generated is a comprehensive and feasible decision-making reference formed after fusion, which contains data-supported judgment and combines the practical experience of experts, and is more practical.

[0067] S403, automatically generating a specific execution plan according to the prediction analysis result of the machine learning engine and the corresponding comprehensive decision-making suggestion generated by combining the expert experience library, the execution plan comprising: SQL optimization script, configuration modification instruction; In the embodiment, according to the prediction analysis result of the machine learning and the comprehensive decision-making suggestion, a specific execution plan containing specific operations is automatically generated, including a script for optimizing SQL statements and an instruction for modifying database configuration, providing a basis for subsequent automatic execution.

[0068] Among them, the execution plan is the specific operation steps converted from the decision-making suggestion, which clearly shows how to execute.

[0069] The SQL optimization script is a code fragment for optimizing the execution efficiency of SQL statements in the database, such as adjusting the query statement structure, adding index hints, and is used to improve the query speed.

[0070] The configuration modification instruction is a command for adjusting parameters of the database system (such as modifying a memory allocation size, adjusting a connection number limit), and can optimize overall performance of the database.

[0071] S404, before execution, the execution plan is verified for safety and feasibility; In the embodiment of the application, before the execution plan is formally run, it is verified whether the execution plan is safe and feasible, so as to avoid database abnormities caused by problems of the plan.

[0072] S405, the execution plan after verification is automatically executed for corresponding autonomous operation, and a database state in an execution process is monitored in real time; In the embodiment of the step, the corresponding autonomous operation is automatically executed by the system through the verified execution plan, and the state of the database is monitored in real time in the execution process, so as to ensure that the operation is performed as expected, and the system can respond in time if an abnormality occurs. The autonomous operation refers to a database management operation automatically executed by the system, such as performance optimization, fault repair, and the like, and does not need manual intervention.

[0073] S406, when the execution plan is executed, an operation log is recorded, and is used for subsequent audit and analysis.

[0074] In the embodiment of the step, after the execution plan is executed, detailed information of the operation is recorded in the log, and the log can be used for subsequent compliance audit and effect analysis. The audit includes checking whether the operation is standard, and the effect analysis includes summarizing experience to improve the process.

[0075] Step S500, the effect of the autonomous operation is monitored and evaluated in real time, and the system performance is continuously optimized according to feedback information.

[0076] In the embodiment, the effect of the autonomous operation refers to a result generated after the autonomous operation (such as performance optimization, fault repair, security protection, and the like) automatically executed by the system is actually run, including whether an expected target (such as whether performance is improved, whether a fault is solved, and whether a security risk is reduced) is reached, and whether a side effect (such as a new performance problem, a data abnormality, and the like) exists.

[0077] The real-time monitoring of the effect of the autonomous operation refers to that, in the execution process of the autonomous operation and after the execution, key indicators (such as a database response time, a throughput, an error rate, a number of security events, and the like) related to the operation effect are continuously and dynamically collected, and an influence of the operation on the database state is tracked in real time.

[0078] The evaluation of the effect of the autonomous operation refers to that, based on the data collected by the real-time monitoring, whether the actual effect of the autonomous operation meets a preset target (such as reducing a query response time by 20%, and no recurrence within 1 hour after fault repair) is analyzed, and effectiveness and rationality of the operation are judged.

[0079] The embodiment of the present application is that the system will track and collect key indicators related to the operation effect (such as database running efficiency, whether the fault is reproduced, whether the security risk is eliminated, etc.) in real time after automatically performing autonomous operations such as performance optimization, fault repair, security protection, etc. Then, according to these indicators, it is evaluated whether the operation achieves the expected effect and whether there are problems by comparing with the preset target, and the evaluation result is taken as feedback information.

[0080] The feedback information is the result data generated in the evaluation process, such as the response time is only reduced by 5% after performance optimization, which does not meet the expectation, and a new connection timeout problem occurs after fault repair. These information reflects the effect and problems of autonomous operation, which is the basis for system improvement.

[0081] Finally, the present application uses these feedback information to continuously adjust and optimize the decision logic and execution strategy of the system, so as to continuously improve the autonomous ability and management effect of the system.

[0082] In further embodiments of the present application, the step S500 specifically comprises: S501, a monitoring index system is established in advance, and key performance indicators and security indicators are defined; This step is to establish a complete monitoring index system in advance, and to clearly define the key measurement standards (such as performance indicators and security indicators) for database performance and security status, so as to provide a unified basis for subsequent evaluation of autonomous operation effect. The key performance indicators are core indicators reflecting the running efficiency of the database, including response time, throughput, CPU usage, etc. The security indicators are key indicators for measuring the security status of the database, including the number of abnormal access times, permission change records, data leakage risks, etc.

[0083] S502, real-time collection of database running data after automatically performing corresponding autonomous operation according to the execution plan, and comparison and analysis with running data before execution of the execution plan; In the embodiment of the present application, after the autonomous operation is executed, the running data of the database is collected in real time, and compared with the data before the operation is executed. The specific changes brought by the operation are understood by analyzing the data difference.

[0084] S503, evaluating operation effect according to the comparison and analysis result, and generating evaluation result, the operation effect including performance improvement amplitude and fault repair situation; In the embodiment of the present application, based on the comparison and analysis result before and after the operation, the actual effect of the autonomous operation is evaluated, and the specific amplitude of performance improvement and the actual situation of fault repair are focused on, and finally the evaluation result is formed.

[0085] The performance improvement is measured by comparing performance metrics (such as the percentage reduction in response time and the percentage increase in throughput) before and after the operation to assess the actual effectiveness of the performance optimization. The fault repair status is determined by comparing fault-related data (such as whether the fault disappeared and the recurrence frequency) before and after the operation to assess whether the fault was effectively resolved.

[0086] S504. Feed the evaluation results back to the machine learning engine model and expert experience base; In the steps of this invention, the evaluation results of the autonomous operation effect are passed to the machine learning engine model and expert experience base in the system, providing a practical basis for the optimization of both.

[0087] S505. Adjust the parameters and strategies of the machine learning engine model and expert experience base based on the feedback evaluation results, and automatically optimize the machine learning engine model and expert experience base.

[0088] In this embodiment, based on the feedback evaluation results, the parameters of the machine learning engine model and the application strategy of the expert experience base are automatically adjusted to continuously optimize the accuracy and practicality of both, thereby improving the overall decision-making and execution capabilities of the system.

[0089] Among them, the parameters and strategies for adjusting the machine learning engine model and expert experience base are to modify the internal calculation parameters of the machine learning model and the application rules and priorities of the knowledge in the expert experience base based on the evaluation results.

[0090] The present invention will be further described in detail below through specific application examples: like Figure 2 As shown in the second specific application embodiment, a database autonomous service processing method based on machine learning is provided, including: S10, Start, then proceed to S11; S11, Data Acquisition, then proceed to S12.

[0091] In this embodiment of the invention, data acquisition is used to collect real-time operational data from the database, including but not limited to performance metrics, SQL statements, transaction information, and logs. In specific implementations of the invention, the collected data undergoes preliminary processing, such as formatting and cleaning, to ensure data accuracy and usability.

[0092] S12, intelligent analysis, and proceed to S13.

[0093] In this embodiment, intelligent analysis is based on the collected operational data, using machine learning algorithms for analysis and modeling, in order to achieve intelligent perception and prediction of the database status.

[0094] S13, Autonomous Decision-Making, then proceed to S14; In the embodiment of the present application, regarding autonomous decision, the experience knowledge and best practices of database experts are adopted to provide decision support for the system.

[0095] In the embodiment of the present application, the experience knowledge of database experts is collected and organized to form a structured knowledge base.

[0096] Then, according to the analysis result of the machine learning engine in the above steps, the relevant expert experience is retrieved from the knowledge base, and the retrieved expert experience is fused with the analysis result to generate a comprehensive decision suggestion, as described above.

[0097] S14, execute the operation, and then enter S15; In the embodiment of the present application, according to the analysis result of the machine learning engine and the suggestion of the expert experience base, the corresponding operation is automatically executed, including but not limited to performance optimization, fault repair, security protection, etc.

[0098] Specifically, in the embodiment of the present application, the comprehensive decision suggestion from the machine learning engine and the expert experience base is received, and then according to the suggestion content, a specific execution plan is generated, such as a SQL optimization script, a configuration modification instruction, etc.

[0099] And the present application verifies the safety and feasibility of the plan before execution, to avoid misoperation.

[0100] After the safety and feasibility verification, the present application automatically executes the operation according to the plan, monitors the database state in real time during the execution process, and records the operation log after the execution is completed, to facilitate subsequent audit and analysis.

[0101] S15, effect evaluation, and then enter S16; In the embodiment of the present application, when specifically implemented, a monitoring index system is established in advance, and key performance indicators and safety indicators are defined. Then, the database running data after the operation is executed is compared with the data before the operation, and the operation effect is evaluated according to the comparison result, such as the performance improvement amplitude, the fault repair situation, etc.

[0102] S16, continuous optimization, and then enter S17; In the embodiment of the present application, according to the real-time monitoring and evaluation feedback of the effect of autonomous operation, the system performance is continuously optimized, as described above.

[0103] S17, end.

[0104] From the above, the application innovatively proposes self-perception and repair of the database, adopts steps of data collection, machine learning, expert experience library, autonomous execution, monitoring and evaluation; the application realizes intelligent management of the database by introducing machine learning and expert experience, effectively reduces the risk of manual operation, and improves the stability and security of the database.

[0105] Exemplary device As Figure 3 shown, the embodiment of the application provides a database autonomous service processing device based on machine learning, which comprises: The data collection module 310 is configured to collect operation data of the database, wherein the operation data comprises performance index operation data, SQL statement data, transaction information data and log data. The machine learning engine prediction analysis module 320 is configured to analyze and model the collected operation data by using the trained machine learning engine model, predict and analyze the database state, and output prediction analysis results. The expert experience library generation module 330 is configured to obtain the experience knowledge and practical knowledge of database experts, and store the expert experience library as decision support provided by the system. The autonomous execution module 340 is configured to automatically generate a specific execution plan according to the prediction analysis results of the machine learning engine and the corresponding comprehensive decision suggestions generated by the expert experience library, and automatically execute corresponding autonomous operations according to the generated execution plan, wherein the autonomous operations comprise performance optimization, fault repair and security protection. The monitoring and evaluation module 350 is configured to monitor and evaluate the effect of the autonomous operation in real time, and continuously optimize the system performance according to the feedback information, as described above.

[0106] Based on the above embodiment, the application further provides a server, and a principle block diagram thereof can be as shown in Figure 4 The server comprises a processor, a memory, a network interface, a display screen and a database connected through a system bus.

[0107] The memory stores one or more programs configured to be executed by the processor to realize the machine learning-based database autonomous service processing method of the above embodiment.

[0108] The server refers to an intelligent computer with data processing capability. The memory can be a flash memory, a hard disk or a cloud storage space, which is used to store program codes and various data of the database. The processor can be a central processing unit, which is used to execute algorithm logic in the program, such as data collection, machine learning, expert experience library, autonomous execution, monitoring and evaluation steps. The program contains the machine learning-based database autonomous service processing method.

[0109] In further embodiments, a server of the present embodiments includes a memory and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by one or more processors, the one or more programs comprising instructions for: collecting running data of the database, wherein the running data includes performance index running data, SQL statement data, transaction information data, and log data; analyzing and modeling the collected running data using a trained machine learning engine model to predict and analyze the database state and output a prediction analysis result; acquiring expert experience and practical knowledge of the database and storing the expert experience as an expert experience library for providing decision support for the system; generating a specific execution plan automatically according to a corresponding comprehensive decision suggestion generated by combining the prediction analysis result of the machine learning engine and the expert experience library, and automatically performing a corresponding autonomous operation according to the generated execution plan; wherein the autonomous operation includes performance optimization, fault repair, and security protection.

[0110] The step of automatically performing a corresponding autonomous operation according to the generated execution plan further includes: real-time monitoring and evaluating the effect of the autonomous operation and continuously optimizing the system performance according to feedback information.

[0111] The step of collecting running data of the database includes: identifying and connecting the target database instance and establishing a communication link for data collection; periodically or in real time collecting and acquiring the running data of the database according to a preset collection strategy; preprocessing the collected running data to obtain preprocessed running data; the preprocessing includes formatting operation and cleaning operation; storing the preprocessed running data to a temporary buffer.

[0112] The step of analyzing and modeling the collected running data using a trained machine learning engine model to predict and analyze the database state and output a prediction analysis result includes: acquiring the processed running data from the temporary buffer; automatically selecting a corresponding machine learning engine model according to the data characteristics of the running data; training and verifying the selected machine learning engine model using the historical running data of the database, optimizing the model parameters, and obtaining a trained machine learning engine model; The trained machine learning engine model is used to extract features and reduce dimensions of the obtained processed operation data, and to predict and analyze the current database state, generating a prediction analysis report.

[0113] The step of obtaining the experience and practical knowledge of database experts and storing the expert experience library as decision support provided by the system comprises: Collect and organize the experience and practical knowledge of database experts to form a structured knowledge base; Classify and index the knowledge base to store the expert experience library as decision support provided by the system.

[0114] The step of automatically generating a specific execution plan according to the corresponding comprehensive decision suggestion generated by combining the prediction analysis result of the machine learning engine with the expert experience library, and automatically executing the corresponding autonomous operation according to the generated execution plan comprises: According to the prediction analysis result of the machine learning engine, retrieve the corresponding relevant expert experience from the expert experience library; Fuse the retrieved expert experience with the prediction analysis result to generate a corresponding comprehensive decision suggestion; According to the prediction analysis result of the machine learning engine, generate a corresponding comprehensive decision suggestion by combining the expert experience library, automatically generate a specific execution plan, and the execution plan includes: SQL optimization script, configuration modification instruction; Verify the safety and feasibility of the execution plan before execution; Automatically execute the corresponding autonomous operation on the verified execution plan, and monitor the database state in real time during execution; When the execution plan is executed, record the operation log for subsequent audit and analysis.

[0115] The step of real-time monitoring and evaluating the effect of autonomous operation, and continuously optimizing the system performance according to the feedback information comprises: Pre-establish a monitoring index system to define key performance indicators and safety indicators; Real-time collection of database operation data after the corresponding autonomous operation is automatically executed according to the execution plan, and comparison and analysis with the operation data before the execution plan is executed; Evaluate the operation effect according to the comparison and analysis result to generate an evaluation result, the operation effect includes performance improvement amplitude and fault repair situation; The evaluation result is fed back to the machine learning engine model and the expert experience library; Adjust the parameters and strategies of the machine learning engine model and the expert experience library according to the feedback evaluation result, automatically optimize the machine learning engine model and the expert experience library, as described above.

[0116] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when executed, can include the processes of the above-mentioned embodiment methods. Any reference to memory, storage, database or other medium used in the embodiments provided by the present application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct RAM bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0117] As can be seen from the above, the present application provides a database autonomous service processing method and device based on machine learning, a server and a storage medium. The present application can realize self-sensing, self-repairing, self-optimizing, self-operation and maintenance and self-security of database faults, and effectively guarantees the stability, security and efficiency of database services.

[0118] The above only describes the embodiments of the present application and is not used to limit the protection scope of the present application. For those skilled in the art, the present application can have various modifications and changes. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A database autonomous service processing method based on machine learning, characterized in that, include: Collect database operation data, including performance index operation data, SQL statement data, transaction information data, and log data; The collected operational data is analyzed and modeled using a trained machine learning engine model to predict and analyze the database status and output the prediction and analysis results. Acquire the experience and practical knowledge of database experts and store it as an expert experience base to provide decision support for the system; Based on the predictive analysis results of the machine learning engine and the corresponding comprehensive decision-making suggestions generated by the expert experience base, a specific execution plan is automatically generated, and corresponding autonomous operations are automatically executed according to the generated execution plan; wherein, the autonomous operations include performance optimization, fault repair, and security protection.

2. The database autonomous service processing method based on machine learning according to claim 1, characterized in that, The step of automatically executing the corresponding autonomous operation according to the generated execution plan includes: The effectiveness of autonomous operations is monitored and evaluated in real time, and system performance is continuously optimized based on feedback information.

3. The database autonomous service processing method based on machine learning according to claim 1, characterized in that, The steps for collecting operational data from the database include: Identify and connect to the target database instance to establish a communication link for data acquisition; According to the preset collection strategy, the database's operational data is collected periodically or in real time; The collected operational data is preprocessed to obtain preprocessed operational data; the preprocessing includes formatting and cleaning operations. The preprocessed runtime data is stored in a temporary buffer.

4. The database autonomous service processing method based on machine learning according to claim 2, characterized in that, The steps of analyzing and modeling the collected operational data using a trained machine learning engine model, predicting and analyzing the database state, and outputting the prediction and analysis results include: Retrieve the processed runtime data from the temporary buffer; Automatically select the corresponding machine learning engine model based on the data characteristics of the running data; The selected machine learning engine model is trained and validated using historical operating data from the database, and the model parameters are optimized to obtain a well-trained machine learning engine model. Using a trained machine learning engine model, feature extraction and dimensionality reduction are performed on the acquired and processed running data, and the current database status is predicted and analyzed to generate a predictive analysis report.

5. The database autonomous service processing method based on machine learning according to claim 1, characterized in that, The step of acquiring and storing the experiential and practical knowledge of database experts as an expert experience base to provide decision support for the system includes: Collect and organize the experience and practical knowledge of database experts to form a structured knowledge base; The knowledge base is classified and indexed, and stored as an expert experience base to provide decision support for the system.

6. The database autonomous service processing method based on machine learning according to claim 1, characterized in that, The steps of automatically generating a specific execution plan based on the predictive analysis results of the machine learning engine and the corresponding comprehensive decision suggestions generated by the expert experience base, and automatically executing the corresponding autonomous operations according to the generated execution plan, include: Based on the predictive analysis results of the machine learning engine, relevant expert experience is retrieved from the expert experience base; The retrieved expert experience is integrated with the predictive analysis results to generate corresponding comprehensive decision-making recommendations; Based on the predictive analysis results of the machine learning engine and the corresponding comprehensive decision-making suggestions generated by the expert experience base, a specific execution plan is automatically generated. The execution plan includes: SQL optimization scripts and configuration modification instructions. The execution plan should be verified for safety and feasibility before execution. The system automatically executes the corresponding autonomous operations on the verified execution plan and monitors the database status in real time during the execution process. Once the execution plan is completed, an operation log is recorded for subsequent auditing and analysis.

7. The database autonomous service processing method based on machine learning according to claim 2, characterized in that, The steps of real-time monitoring and evaluation of the effects of autonomous operations, and continuous optimization of system performance based on feedback information, include: Establish a monitoring indicator system in advance, and define key performance indicators and safety indicators; Real-time collection of database operation data after the automatic execution of corresponding autonomous operations according to the execution plan, and comparative analysis with the operation data before the execution plan was executed; The operational effectiveness is evaluated based on the comparative analysis results, and an evaluation result is generated. The operational effectiveness includes the performance improvement and fault repair status. The evaluation results are fed back to the machine learning engine model and the expert experience base. Based on the feedback evaluation results, the parameters and strategies of the machine learning engine model and expert experience base are adjusted to automatically optimize the machine learning engine model and expert experience base.

8. A database autonomous service processing device based on machine learning, characterized in that, The device includes: The data acquisition module is used to collect the database's operational data, which includes performance indicator operational data, SQL statement data, transaction information data, and log data. The machine learning engine predictive analysis module is used to analyze and model the collected runtime data using a trained machine learning engine model, predict and analyze the database status, and output the predictive analysis results. The expert experience base generation module is used to acquire the experience and practical knowledge of database experts and store it as an expert experience base to provide decision support for the system. The autonomous execution module is used to automatically generate specific execution plans based on the predictive analysis results of the machine learning engine and the corresponding comprehensive decision suggestions generated by the expert experience base, and to automatically execute corresponding autonomous operations according to the generated execution plans; wherein, the autonomous operations include performance optimization, fault repair, and security protection; The monitoring and evaluation module is used to monitor and evaluate the effectiveness of autonomous operations in real time, and continuously optimize system performance based on feedback information.

9. A server, characterized in that, It includes a memory and one or more programs, wherein one or more programs are stored in the memory and configured to be executed by one or more processors, wherein the one or more programs include methods for performing any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, When the instructions in the storage medium are executed by the processor of the electronic device, the electronic device is able to perform the method as described in any one of claims 1-7.

Images