A method and device for intelligent inspection of database

Abstract

The present application provides a method and device for intelligent inspection of a database, which belongs to the field of database technology. The method obtains a multimodal data set corresponding to the operating status of the inspection database by presetting a multi-source heterogeneous data collection method; based on the cross-modal attention mechanism and the self-learning model, a multidimensional data association relationship corresponding to the multimodal data set is established, and the database health value and abnormal warning information are determined by combining historical time series data with unsupervised anomaly detection; wherein the multidimensional data association relationship is established at least between SQL performance, resource consumption, transaction behavior and business load. According to the database health value, abnormal warning information and the preset feedback optimization mechanism, a database optimization plan is generated and sent to the user terminal. Through the above scheme, combined with the cross-modal attention mechanism, the self-learning model, the knowledge graph association reasoning and the time series prediction model, the database is comprehensively inspected to improve the automation, dynamic and intelligent level of the inspection.

Description

Technical Field

[0001] The present application relates to the field of database technology, and in particular to an intelligent inspection method and device for a database. Background Art

[0002] At present, database inspection has developed to a stage that combines static and dynamic methods, and most inspections rely on tools, which has improved inspection efficiency to a certain extent. However, the following problems still exist: Traditional inspections usually only rely on structured query language (SQL) logs or hardware resource monitoring data, lacking comprehensive analysis of multi-dimensional data such as transaction management and business load, resulting in one-sided inspection results.

[0003] In addition, the existing database inspection process relies too much on manual experience, lacks intelligent analysis and automatic repair capabilities, and is difficult to accurately locate performance bottlenecks; the inspection items are also relatively fixed and cannot be dynamically adjusted according to the real-time status of the database, making it difficult to warn of performance problems in advance. In addition, traditional inspections are mostly post-analysis, which cannot predict future performance bottleneck trends, resulting in passive responses. Summary of the invention

[0004] To solve the above problems, the embodiments of the present application provide a method and device for intelligent inspection of a database, which are used to comprehensively inspect the database and improve the automation, dynamism and intelligence level of database inspection.

[0005] On the one hand, an embodiment of the present application provides a method for intelligent inspection of a database, the method comprising:

[0006] By presetting a multi-source heterogeneous data collection method, a multimodal data set corresponding to the operating status of the inspection database is obtained;

[0007] Based on the cross-modal attention mechanism and the self-learning model, a multidimensional data association relationship corresponding to the multimodal data set is established, and the database health value and abnormal warning information are determined by combining historical time series data with unsupervised anomaly detection; wherein the multidimensional data association relationship is established at least between SQL performance, resource consumption, transaction behavior and business load;

[0008] A database optimization plan is generated based on the database health value, the abnormal warning information and a preset feedback optimization mechanism, and is sent to a user terminal.

[0009] In one implementation of the present application, after generating the database optimization plan, the method further includes:

[0010] Based on knowledge graph association reasoning, time series prediction model and preset historical health assessment data, the future performance bottleneck trend information of the inspection database and the optimization plan corresponding to the future performance bottleneck trend information are determined, and the future performance bottleneck trend information and the optimization plan corresponding to the future performance bottleneck trend information are sent to the user terminal so that the user terminal can determine whether to optimize again.

[0011] In one implementation of the present application, a multi-modal data set corresponding to the running state of the inspection database is obtained by presetting a multi-source heterogeneous data collection method, specifically including:

[0012] Through the database built-in system table, log analysis and external monitoring tools, multi-source heterogeneous data of the inspection database operation status are collected in real time, and the multi-source heterogeneous data at least includes: SQL execution data, resource consumption data, transaction behavior data, and business load data;

[0013] Each of the multi-source heterogeneous data is added to the multimodal dataset.

[0014] In one implementation of the present application, real-time collection of multi-source heterogeneous data of the operation status of the inspection database specifically includes:

[0015] Obtaining SQL execution data through the database built-in system table, wherein the SQL execution data at least includes query text, execution time and index hit rate;

[0016] Acquire the transaction behavior data by log analysis, wherein the transaction behavior data at least includes transaction commit and rollback rate, lock waiting time and number of concurrent transactions;

[0017] Collect the resource consumption data through an external monitoring tool, wherein the resource consumption data at least includes CPU load, memory usage and disk I / O;

[0018] The business load data is obtained by associating the database operation log with the business log, and the business load data includes the access peak time and the query request distribution.

[0019] In one implementation of the present application, based on the cross-modal attention mechanism and the self-learning model, a multidimensional data association relationship corresponding to the multimodal data set is established, specifically including:

[0020] Extracting features from each multi-source heterogeneous data in the multimodal data set, and converting the extracted data features of each dimension into a feature comparison vector;

[0021] According to each of the feature comparison vectors and the cross-modal attention mechanism, a corresponding correlation weight matrix between the SQL performance, the resource consumption, the transaction behavior, and the business load is calculated;

[0022] Determine, according to the self-learning model, association information of different data modes of the same multi-source heterogeneous data; the association information at least includes data mode grouping and abnormal mode characteristics;

[0023] The multi-dimensional data association relationship is generated according to the association weight matrix and the association information.

[0024] In one implementation of the present application, historical time series data and unsupervised anomaly detection are combined to determine the database health value and anomaly warning information, specifically including:

[0025] Modeling the historical time series data by using a preset Transformer model to generate a baseline model;

[0026] Calculating the degree of deviation between the current data corresponding to the multimodal data set and the baseline model through an autoencoder to determine an outlier according to the degree of deviation; wherein the degree of deviation is obtained based on a reconstruction error;

[0027] According to the abnormal value and the multi-dimensional data association relationship, the abnormal value is weightedly calculated to determine the health sub-value corresponding to each data pattern grouping, and the database health value is calculated according to each health sub-value;

[0028] Determine the cause of the abnormality according to the abnormal value and the multidimensional data association relationship;

[0029] The abnormal warning information is generated according to the database health value and the abnormal cause.

[0030] In one implementation of the present application, a database optimization plan is generated according to the database health value, the abnormal warning information, and a preset feedback optimization mechanism, specifically including:

[0031] Matching the database health value and the abnormal warning information with a preset optimization plan list to determine the database optimization plan according to the matching result; the database optimization plan includes at least one or more of the following: SQL optimization, resource allocation optimization, and concurrency control optimization;

[0032] According to the preset feedback optimization mechanism, the execution feedback information corresponding to the database optimization plan is determined, and when it is judged that the execution feedback information does not meet the preset optimization conditions, the database optimization plan is iteratively updated until the database optimization plan that meets the preset optimization conditions is generated.

[0033] In one implementation of the present application, the preset optimization solution list includes at least: rewriting SQL statements, switching Join algorithms, adjusting work_mem parameters, adjusting checkpoint_timeout parameters, and optimizing lock strategies.

[0034] In one implementation of the present application, based on knowledge graph association reasoning, time series prediction model and preset historical health assessment data, the future performance bottleneck trend information of the inspection database and the optimization scheme corresponding to the future performance bottleneck trend information are determined, specifically including:

[0035] According to the knowledge graph association reasoning, the root cause information of each performance bottleneck in the preset historical health assessment data is determined; wherein the knowledge graph association reasoning is based on the extracted multi-source heterogeneous data as entities and the connections between the multi-source heterogeneous data as relationships;

[0036] Input the preset historical health assessment data and the corresponding root cause information of each performance bottleneck into the time series prediction model to predict the performance indicator change trend of the inspection database within a preset time period and generate the future performance bottleneck trend information;

[0037] According to the historical health values, historical abnormal warning information and historical optimization plans in the preset historical health assessment data, an optimization plan corresponding to the future performance bottleneck trend information is generated.

[0038] On the other hand, an embodiment of the present application further provides an intelligent inspection device for a database, the device comprising:

[0039] An acquisition module is used to acquire a multimodal data set corresponding to the operation status of the inspection database through a preset multi-source heterogeneous data acquisition method;

[0040] Establish a determination module for establishing a multidimensional data association relationship corresponding to the multimodal data set based on a cross-modal attention mechanism and a self-learning model, and determining a database health value and abnormal warning information in combination with historical time series data and unsupervised anomaly detection; wherein the multidimensional data association relationship is established at least between SQL performance, resource consumption, transaction behavior and business load;

[0041] The generation module is used to generate a database optimization plan according to the database health value, the abnormal warning information and a preset feedback optimization mechanism, and send the plan to the user terminal.

[0042] Compared with the prior art, the present invention has the following significant effects:

[0043] Through the above technical solutions, the collection of multi-source heterogeneous data in the database is realized. Combined with the cross-modal attention mechanism, self-learning model, knowledge graph association reasoning and time series prediction model, the comprehensiveness, accuracy, intelligence and foresight of database intelligent inspection are achieved.

[0044] Specifically, this application integrates multi-dimensional data such as SQL performance, resource consumption, transaction behavior, and business load to achieve comprehensive inspections. The cross-modal attention mechanism can reveal the interactive impact of multi-dimensional data and accurately locate the root cause of performance bottlenecks without over-reliance on manual experience. At the same time, it can actively generate database optimization solutions, reduce manual intervention, and improve inspection efficiency.

[0045] In addition, this application also predicts future performance bottleneck trends through a time series prediction model, generates early warning information in advance, and supports proactive maintenance. The above solution effectively solves the problems of single data, lack of intelligent analysis, insufficient dynamic monitoring, and inability to provide early warnings in traditional database inspections, significantly improves the intelligence level and operation and maintenance efficiency of database inspections, and has broad application prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0047] Figure 1 A schematic diagram of a flow chart of an intelligent inspection method of a database in an embodiment of the present application;

[0048] Figure 2 Another flowchart of a method for intelligent inspection of a database in an embodiment of the present application is shown;

[0049] Figure 3 A schematic diagram of multimodal data in an intelligent inspection method of a database in an embodiment of the present application;

[0050] Figure 4 A schematic diagram of a process for deep analysis of inspection data in an intelligent inspection method for a database in an embodiment of the present application;

[0051] Figure 5 A schematic diagram of a process for generating a database optimization solution in a method for intelligent inspection of a database in an embodiment of the present application;

[0052] Figure 6 A schematic diagram of a process for predicting the future operation trend of a database in an intelligent inspection method of a database in an embodiment of the present application;

[0053] Figure 7This is a schematic diagram of the structure of an intelligent inspection device for a database in an embodiment of the present application. DETAILED DESCRIPTION

[0054] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described below in combination with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.

[0055] In the prior art, the inspection method based on SQL logs mainly analyzes SQL execution time, query mode and error conditions through SQL query logs. This method only focuses on SQL-level data and lacks multi-modal data support such as transaction management, system resources, and business traffic, resulting in one-sided inspection results.

[0056] Inspection tools based on resource monitoring: collect data such as CPU load, memory usage, disk I / O, and issue alarms. This method only monitors hardware resource consumption and cannot analyze SQL statement execution and transaction dependencies, making it difficult to accurately locate problems.

[0057] Based on this, the automation level of traditional database inspection is not enough. Most inspections still rely on DBA manual execution, lacking intelligent analysis and automatic repair capabilities. The dynamic inspection capabilities of traditional databases are insufficient and cannot warn of performance problems in advance. The inspection data type is single and cannot be comprehensively analyzed.

[0058] The embodiments of the present application provide a method and device for intelligent inspection of a database, which are used to solve the problems that the current database inspection data is single, the inspection results are one-sided, and there is a lack of intelligent analysis and dynamic monitoring capabilities, and it is impossible to predict the future performance bottlenecks of the database.

[0059] The following describes in detail various embodiments of the present application in conjunction with the accompanying drawings.

[0060] The present application embodiment provides a method for intelligent inspection of a database, such as Figure 1 As shown, the method may include steps S101-S103:

[0061] S101, the server obtains a multimodal data set corresponding to the operating status of the inspection database through a preset multi-source heterogeneous data collection method.

[0062] It should be noted that the server, as the executor of the intelligent inspection method of the database, is only for exemplary purposes. The executor is not limited to the server, and this application does not make any specific limitations on this.

[0063] In the embodiment of the present application, a multi-modal data set corresponding to the running status of the inspection database is obtained by presetting a multi-source heterogeneous data collection method, specifically including:

[0064] Through the database's built-in system tables, log analysis, and external monitoring tools, multi-source heterogeneous data on the operation status of the inspection database is collected in real time. The multi-source heterogeneous data includes at least: SQL execution data, resource consumption data, transaction behavior data, and business load data. Each multi-source heterogeneous data is added to the multimodal data set.

[0065] That is to say, this application uses a preset multi-source heterogeneous data collection method, including collection through database built-in system tables, collection through log analysis, and collection through external monitoring tools, to collect data from the inspection database selected or pre-specified by the user, and these data are related to the database operation status. Among them, the database built-in system table can read the system table or view (such as pg_stat_activity, sys.dm_exec_requests) provided by the database management system (Database Management System, DBMS) in real time, collect the current number of active connections, SQL execution status, lock waiting information and resource occupancy indicators; database log analysis can be a structured analysis of database transaction logs (such as RedoLog, UndoLog), slow query logs (SlowQueryLog) and error logs, extract SQL execution time, transaction commit / rollback records, deadlock events and abnormal alarm information; external monitoring tools can connect to third-party monitoring systems (such as Prometheus, Zabbix) through APIs or plug-ins, periodically collect server hardware resource data (CPU utilization, memory occupancy, disk I / O throughput) and network traffic indicators, and associate them with the database instance-level load status.

[0066] Figure 2 This is another flow chart of a method for intelligent inspection of a database according to an embodiment of the present application. The above step S101 corresponds to Figure 2 S201: Comprehensive data collection on the database operation status is carried out through the database internal system view, log analysis and external monitoring tools.

[0067] In the embodiment of the present application, the multi-source heterogeneous data of the real-time collection of the running status of the inspection database specifically includes:

[0068] Obtain SQL execution data through the database's built-in system tables. SQL execution data includes at least query text, execution time, and index hit rate. Obtain transaction behavior data through log analysis. Transaction behavior data includes at least transaction commit and rollback rates, lock wait time, and number of concurrent transactions. Collect resource consumption data through external monitoring tools. Resource consumption data includes at least CPU load, memory usage, and disk I / O. Obtain business load data by associating database operation logs with business logs. Business load data includes access peak time and query request distribution.

[0069] In other words, this application can collect multi-source heterogeneous data of different modalities, including text modalities such as query text, execution time, index hit rate, etc., time series modalities such as CPU load, memory usage and disk I / O, graph structure modalities such as transaction commit and rollback rate, lock waiting time and number of concurrent transactions, and spatiotemporal modalities such as access peak time and query request distribution. This breaks through the limitation of a single data source for database inspection, and can achieve a comprehensive evaluation of database performance, resource consumption, transaction integrity and business load based on the above data, providing a data foundation for intelligent optimization and predictive warning.

[0070] The multimodal data collected in step S201 is as follows: Figure 3 As shown, including 301-308:

[0071] 301, SQL execution data (text mode): query text, execution time, index hit rate, number of returned records. 302, Execution plan analysis (text mode): EXPLAINANALYZE results, including scanning mode, execution cost, cache hit rate. 303, Resource consumption data (time series mode): CPU load, memory usage, disk I / O, network traffic. 304, Storage management data (time series mode): cache hit rate, WAL log growth rate, write latency. 305, Concurrency control data (graph structure mode): number of active connections, number of concurrent transactions, lock waiting time. 306, Transaction execution data (graph structure mode): transaction commit / rollback rate, timeout and deadlock frequency. 307, Business load data (time and space mode): access peak time, batch import mode, query request distribution. 308, Business correlation data: identify high-impact SQL statements by correlating database logs with business logs.

[0072] S102, the server establishes a multidimensional data association relationship corresponding to the multimodal data set based on the cross-modal attention mechanism and self-learning model, and combines historical time series data with unsupervised anomaly detection to determine the database health value and abnormal warning information.

[0073] The multidimensional data association relationship is established at least between SQL performance, resource consumption, transaction behavior and business load.

[0074] The above step S102 can be generally understood as step S202, which introduces an intelligent analysis model to deeply analyze the inspection data, evaluate the database operation status from multiple dimensions, and generate optimization suggestions.

[0075] In the embodiment of the present application, based on the cross-modal attention mechanism and the self-learning model, a multi-dimensional data association relationship corresponding to the multi-modal data set is established, specifically including:

[0076] Feature extraction is performed on each multi-source heterogeneous data in the multimodal data set, and the extracted data features of each dimension are converted into feature comparison vectors. According to each feature comparison vector and the cross-modal attention mechanism, the corresponding association weight matrix between SQL performance, resource consumption, transaction behavior and business load is calculated. According to the self-learning model, the association information of different data modes of the same multi-source heterogeneous data is determined. The association information includes at least data mode grouping and abnormal mode features. According to the association weight matrix and the association information, a multidimensional data association relationship is generated. Among them, the abnormal mode feature can be used to identify abnormal modes, which is feature data related to abnormal identification.

[0077] That is to say, the present application first extracts features from various multi-source heterogeneous data, thereby converting different types of inspection data into a unified and comparable vector format. Among them, the present application can use different feature extraction algorithms to perform feature extraction, and the present application does not make specific restrictions on this. For example, "statistical feature extraction" models resource monitoring data (such as CPU usage, memory usage, disk I / O and other time series data) by calculating statistics; "time series feature extraction" models business layer data through sliding windows; "autoencoder" compresses high-dimensional data into low-dimensional representations to model SQL query logs.

[0078] Subsequently, a cross-modal attention mechanism is used to calculate the attention mechanism for each feature comparison vector corresponding to SQL performance, resource consumption, transaction behavior, and business load, so as to calculate the association weight between them, which contains cross-modal causal relationships. For example, a certain type of SQL with high concurrency causes CPU overload, and the association weight value between the two is relatively high, indicating that the two are strongly correlated. If the weights of "transaction submission delay" and "business access peak" are 0.8, then the second is. This application can extract the data modality combination that has the greatest impact on performance based on the weight sorting results. After calculating the various association weights, an association weight matrix can be formed for calculation and use.

[0079] The form of the association weight matrix is ​​shown in the following table:

[0080] Table 1. Association weight matrix form

[0081]

[0082] Subsequently, this application will also use a self-learning model to analyze different data patterns in multi-source heterogeneous data, and analyze the similarities and differences between different data patterns. Data patterns are regular features or behavior patterns implicit in data, such as: SQL query patterns: inefficient queries with high frequency calls, statements that miss indexes; resource consumption characteristics: periodic CPU overloads during business peaks, gradual occupancy increases caused by memory leaks; transaction behavior patterns: time aggregation of lock competition and deadlock events caused by batch updates. A multi-source heterogeneous data (i.e., multimodal data source) may contain one or more data patterns, such as "inefficient query patterns" and "high-frequency call patterns" in SQL logs in text mode; "periodic load patterns" and "sudden abnormal patterns" may be contained in resource data in time series mode. The self-learning model of this application can use machine learning algorithms such as clustering and contrastive learning to analyze the common features and differential features of different data patterns, generate data pattern groupings and abnormal pattern features, and obtain associated information. By analyzing the relationship between patterns through a self-learning mechanism, multi-dimensional data collaborative analysis can be achieved, which can be upgraded from "one-sided diagnosis" to "global root cause location", thereby improving the comprehensiveness of inspections and the accuracy of decision-making.

[0083] Finally, the present application adds both the association weight matrix and the association information to the multidimensional data association relationship, that is, the multidimensional data association relationship includes sub-items such as the association weight matrix and the association information.

[0084] Further, in the embodiment of the present application, the above-mentioned combination of historical time series data and unsupervised anomaly detection to determine the database health value and abnormal warning information specifically includes:

[0085] The historical time series data is modeled by the preset Transformer model to generate a baseline model. The degree of deviation between the current data corresponding to the multimodal data set and the baseline model is calculated by the autoencoder to determine the outlier according to the degree of deviation. The degree of deviation is obtained based on the reconstruction error. According to the association between the outlier and the multidimensional data, the outlier is weighted to determine the health sub-value corresponding to each data mode grouping, and the database health value is calculated according to each health sub-value. According to the association between the outlier and the multidimensional data, the cause of the anomaly is determined. According to the database health value and the cause of the anomaly, an abnormal warning information is generated.

[0086] The above-mentioned historical time series data can be data related to the historical operation status of the inspection database, which can be pre-stored in the server or input by the user in advance. This application does not make specific restrictions on this. This application can use Transformer to model historical time series data, capture long-term dependencies and trend changes, automatically identify SQL runtime anomalies, transaction submission delays, index failures and other problems, and finally output a baseline model of historical time series data. The baseline model contains the normal range intervals of various multimodal data, such as the normal range of SQL execution time.

[0087] The present application also uses an autoencoder to identify behaviors that deviate from the above baseline model. When behaviors that are significantly different from normal behaviors occur, the autoencoder will produce large fluctuations in the reconstruction error, and the system can detect anomalies through the reconstruction error. The present application can calculate the degree of fluctuation of the reconstruction error, that is, the degree of deviation, which can be obtained by calculating the difference of the fluctuations. By setting a fluctuation degree comparison table, the fluctuation degree is matched with the abnormal value in the fluctuation degree comparison table, and the autoencoder can output the score of the abnormal value.

[0088] Subsequently, the present application calculates the health sub-value according to the above-mentioned association weight matrix and the outliers and data mode grouping. Specifically, after obtaining the outliers, the present application can determine the data dimension corresponding to the outliers, that is, SQL performance, resource consumption, transaction behavior or business load, and determine the data mode grouping where each outlier is located. According to the data dimension, the present application queries the association weights corresponding to each outlier in the corresponding data mode grouping from the association weight matrix, and performs weighted calculation according to each association weight and each outlier to obtain the health sub-value corresponding to each data mode grouping under the same data dimension. An outlier corresponds to a data mode grouping. The present application can determine the data mode groupings corresponding to all current outliers according to each outlier, and determine the association weights corresponding to each data mode grouping under the corresponding data dimension according to the association weight matrix, to form the association weight group under the same data dimension, and perform weighted calculation on each outlier and the corresponding association weight group under the same data dimension to obtain the health sub-value. Then sum up each health sub-value to obtain the database health value.

[0089] For example, the outliers include: SQL performance outlier value = 0.8, resource consumption outlier value = 0.6, transaction behavior outlier value = 0.7, and business load outlier value = 0.5, which correspond to the SQL query pattern grouping in the data pattern grouping: high-frequency and inefficient query; resource consumption feature grouping: sudden increase anomaly; transaction behavior pattern grouping: abnormal rollback; business load data grouping: access peak.

[0090] Query the association weight matrix for each outlier in each data mode grouping in different data dimensions, such as the association weight of SQL performance:

[0091] The impact weight on SQL performance itself = 1.0; the impact weight on resource consumption = 0.7; the impact weight on transaction behavior = 0.5; the impact weight on business load = 0.3.

[0092] The associated weight of resource consumption:

[0093] The impact weight on SQL performance = 0.7; the impact weight on resource consumption itself = 1.0; the impact weight on transaction behavior = 0.6; the impact weight on business load = 0.4.

[0094] The associated weight of the transaction behavior:

[0095] The impact weight on SQL performance = 0.5; the impact weight on resource consumption = 0.6; the impact weight on transaction behavior itself = 1.0; the impact weight on business load = 0.2.

[0096] The associated weight of the business load:

[0097] The impact weight on SQL performance = 0.3; the impact weight on resource consumption = 0.4; the impact weight on transaction behavior = 0.2; the impact weight on the business load itself = 1.0.

[0098] This application performs weighted calculation on outliers in the same data dimension:

[0099]

[0100] in, Indicates The health sub-values ​​of the data pattern groups, For the outliers, Indicates The outlier pair The influence weight of each data pattern grouping, is the number of outliers.

[0101] For example, the healthy sub-values ​​for the SQL query pattern grouping (high-frequency, low-efficiency queries):

[0102] .

[0103] Health sub-values ​​for resource consumption feature groups (sudden spikes):

[0104] 0.8×0.7+0.6×1.0+0.7×0.6+0.5×0.4=0.56+0.6+0.42+0.2=1.78.

[0105] Then, the present application may also normalize the above health sub-values ​​to a range of 0 to 1, such as: ,Finally, the normalized health sub-values ​​are summed up, and the sum is used as the database health value.

[0106] In addition, the present application can also combine outliers, association weight matrices, data pattern groupings and abnormal pattern features to locate the root cause of abnormal behavior. The present application can filter out associations whose association weights are greater than a first predetermined value from the association weight matrix, and determine the data pattern groupings whose abnormal values ​​are greater than a second predetermined value in these filtered associations. For example, the association weight between SQL execution time and CPU load is 0.9, and SQL execution time 0.95 is strongly correlated with CPU load 0.85. The present application can perform reasoning to obtain the root cause of abnormal behavior. The reasoning process can be solved by knowledge graphs, or rule reasoning can be used, which is not specifically limited here. For example, through knowledge graph reasoning, the cause of the exception is output: "A certain type of complex query does not use an index, resulting in a full table scan and CPU overload." The above-mentioned first predetermined value and second predetermined value can be set by the user, such as the user setting it with reference to the abnormal pattern features, and the present application does not specifically limit this.

[0107] The server generates abnormal warning information based on the above database health value and abnormal reason. The abnormal warning information can also be sent to the user terminal for display to the user. The user terminal can be a mobile phone, computer or other device of the system user, and this application does not make specific restrictions on this.

[0108] Through the above solution, the database health value is grouped according to the data mode, which can more accurately inspect the database and improve the inspection quality.

[0109] The above step S202 introduces an intelligent analysis model to deeply analyze the inspection data, evaluate the database operation status from multiple dimensions, and generate optimization suggestions, and can also specifically include the following sub-steps, such as Figure 4As shown: 401, a feature extraction algorithm is used to uniformly model SQL operation logs, resource monitoring data and business layer data, and different types of inspection data are converted into comparable vector formats. 402, a cross-modal attention mechanism is used to establish associations between SQL statements, resource monitoring data, transaction management data and business traffic data to discover the core factors affecting database performance. 403, through a self-learning mechanism, the similarities and differences between different data patterns are analyzed, so that SQL query patterns, resource consumption characteristics, and transaction behavior patterns can work together in database health assessment to improve the comprehensiveness of inspections. 404, Transformer is used to model the historical change trend of the database operation status, and automatically identify problems such as SQL runtime anomalies, transaction submission delays, and index failures. 405, through intelligent analysis without preset rules, the system automatically analyzes abnormal behavior of the database without fixed standards, such as a sudden increase in lock contention and an abnormal increase in transaction rollback rate, and outputs specific abnormal causes and possible impact ranges. 406. Analyze SQL query plans (EXPLAIN ANALYZE), evaluate SQL execution efficiency, and make optimization suggestions, such as index optimization, JOIN method optimization, etc. 407. Based on the intelligent decision-making mechanism of feedback optimization, continuously adjust the optimization strategy of SQL statements so that the SQL tuning process can dynamically adapt to the database load. 408. Adopt the knowledge graph construction method, combine SQL queries, transaction dependencies and resource consumption, form a database health status graph, and perform association reasoning.

[0110] S103, the server generates a database optimization plan according to the database health value, abnormal warning information and a preset feedback optimization mechanism, and sends it to the user terminal.

[0111] In the embodiment of the present application, the above-mentioned database optimization scheme is generated according to the database health value, abnormal warning information and the preset feedback optimization mechanism, specifically including:

[0112] Match the database health value and abnormal warning information with the preset optimization plan list to determine the database optimization plan based on the matching results. The database optimization plan includes at least one or more of the following: SQL optimization, resource configuration optimization, and concurrency control optimization. According to the preset feedback optimization mechanism, determine the execution feedback information corresponding to the database optimization plan, and if it is determined that the execution feedback information does not meet the preset optimization conditions, iteratively update the database optimization plan until a database optimization plan that meets the preset optimization conditions is generated.

[0113] In other words, a preset optimization plan list is pre-set in the server, and the database optimization plan pre-set in the matching list can be performed according to the inspection analysis results, such as database health value and abnormal warning information. At the same time, when the present application uses the database optimization plan to perform optimization, it can monitor in real time whether the optimization results meet the preset optimization conditions. For example, if the optimization SQL execution time does not improve the efficiency, that is, the difference between the original SQL execution time and the optimized SQL execution time is less than the optimization time value pre-set by the user, and the preset optimization conditions are not met at this time, the server will iteratively update the database optimization plan, such as regenerating the query plan, adjusting the Join order, etc. The specific optimization strategy can be pre-set by the user, and this application does not make specific restrictions on this. Specifically, such as: defining the database system and its operating status; selecting appropriate SQL tuning operations to optimize query performance. Query the execution status of the current database or the execution status of the SQL query; optimization operations taken, such as regenerating the query plan, adjusting the JOIN order, etc. Give a reward value based on the performance of the SQL query. If the optimization strategy is effective, the reward will be higher. Multiple iterations and optimization.

[0114] The above preset optimization plan list at least includes: rewriting SQL statements, switching Join algorithms, adjusting work_mem parameters, adjusting checkpoint_timeout parameters, and optimizing lock strategies.

[0115] Furthermore, the above steps correspond to Figure 2 S203 in the embodiment automatically generates a database optimization plan based on the inspection and analysis results. Specifically, it includes the following sub-steps: Figure 5 As shown: 501. Through SQL execution plan analysis, SQL rewrite plans are automatically generated to optimize query structure and improve query efficiency. 502. When too many nested loops are used in SQL statements, HashJoin or MergeJoin is automatically recommended to improve execution efficiency. 503. When transaction execution timeout is found, work_mem parameter adjustment is automatically recommended to improve query cache capability. 504. When WAL logs are frequently written, checkpoint_timeout parameter is optimized to improve writing efficiency. 505. When long lock waits are found, it is recommended to optimize SQL statements and reduce the use of table-level locks, such as using row-level locks or transaction batching.

[0116] Through the above solution, optimization solutions can be automatically provided for the inspection results, effectively improving the efficiency of database inspection without over-reliance on manual experience to complete inspection optimization.

[0117] In addition, after the application generates a database optimization plan, the method further includes:

[0118] According to the multi-dimensional data association, database health value and abnormal warning information, a knowledge graph of the database health status of the current inspection is constructed, and the knowledge graph of the database health status is sent to the user terminal.

[0119] In other words, this application can use multi-dimensional data associations, database health values, and anomaly and warning information to construct a knowledge graph. Entities: SQL queries, transactions, resource indicators (CPU, memory, disk I / O), business operations, health values, anomaly causes, optimization suggestions. Relationships: Dependency between SQL queries and transactions; consumption relationship between transactions and resources; triggering relationship between business operations and SQL queries; correspondence between anomaly causes and optimization suggestions. The knowledge graph intuitively displays the health status, anomaly causes, and optimization suggestions of the database, helping users quickly understand database problems. The knowledge graph can also record historical data of inspection results to facilitate subsequent analysis and optimization effect evaluation.

[0120] In another embodiment of the present application, after the present application generates a database optimization plan, it also includes:

[0121] Based on knowledge graph association reasoning, time series prediction model and preset historical health assessment data, the server determines the future performance bottleneck trend information of the inspection database and the optimization plan corresponding to the future performance bottleneck trend information, and sends the future performance bottleneck trend information and the optimization plan corresponding to the future performance bottleneck trend information to the user terminal so that the user terminal can determine whether to optimize again.

[0122] That is to say, the database optimization plan, future performance bottleneck trend information and its corresponding optimization plan can be displayed on the user terminal so that the user can view or intervene in the inspection work, and optimize the above optimization plan again.

[0123] In the embodiment of the present application, the above-mentioned determination of the future performance bottleneck trend information of the inspection database and the optimization scheme corresponding to the future performance bottleneck trend information based on the knowledge graph association reasoning, the time series prediction model and the preset historical health assessment data specifically includes:

[0124] According to the knowledge graph association reasoning, the root cause information of each performance bottleneck in the preset historical health assessment data is determined. Among them, the knowledge graph association reasoning takes the extracted multi-source heterogeneous data as entities and the connections between the multi-source heterogeneous data as relationships. The preset historical health assessment data and its corresponding root cause information of each performance bottleneck are input into the time series prediction model to predict the performance indicator change trend of the inspection database within the preset time length and generate future performance bottleneck trend information. Based on the historical health values, historical abnormal warning information and historical optimization plans in the preset historical health assessment data, an optimization plan corresponding to the future performance bottleneck trend information is generated.

[0125] That is to say, the present application constructs a knowledge graph, which takes SQL queries, transactions, resource indicators (CPU, memory, disk I / O), business operations, etc. as entities, and takes the dependency relationship between SQL queries and transactions, the consumption relationship between transactions and resources, and the trigger relationship between business operations and SQL queries as relationships. The nodes in the knowledge graph represent entities, and the edges represent relationships. By presetting the knowledge graph corresponding to the historical health assessment data, the corresponding performance bottleneck root cause information can be obtained by association reasoning. Then, using a time series prediction model, such as a long short-term memory network (LSTM) model, the future performance indicators of the inspection database are predicted to change according to the above-mentioned preset historical health assessment data and performance bottleneck root cause information, so as to obtain future performance bottleneck trend information. At the same time, the present application matches the optimization strategy for the anomalies in the future performance bottleneck trend information according to the obtained historical health values, historical abnormal warning information and historical optimization schemes, as the optimization scheme corresponding to the future performance bottleneck trend information. The historical health value can be pre-marked by an expert or user, and the present application does not make specific restrictions on this.

[0126] Furthermore, it can accurately predict future performance bottleneck trends and generate early warning information in advance; generate targeted optimization solutions to support proactive maintenance; improve the intelligence level of database operation and maintenance, and achieve the transition from passive response to active optimization. This process significantly improves the accuracy and efficiency of database inspections, and solves the problems of single data, lack of intelligent analysis, and inability to provide early warnings in traditional inspections.

[0127] The above steps correspond to Figure 2 Step S204 in the example is to predict the future operation trend of the database based on the historical inspection data and provide early warning information. This includes the following sub-steps: Figure 6 As shown, it includes: 601, using time series analysis methods to predict future database storage needs, early warning of storage space shortage risks, and recommending data cleaning or storage expansion. 602, analyzing business query request patterns, predicting future database load peak periods, and providing resource allocation suggestions. 603, combining historical inspection data, through knowledge graph reasoning, predicting possible performance bottlenecks in the database, and providing optimization solutions in advance to prevent database failures.

[0128] Through the above technical solutions, the collection of multi-source heterogeneous data in the database is realized. Combined with the cross-modal attention mechanism, self-learning model, knowledge graph association reasoning and time series prediction model, the comprehensiveness, accuracy, intelligence and foresight of database intelligent inspection are achieved.

[0129] First, this application integrates multi-dimensional data such as SQL performance, resource consumption, transaction behavior, and business load to achieve comprehensive inspections. The cross-modal attention mechanism can reveal the interactive impact of multi-dimensional data, and can accurately locate the root cause of performance bottlenecks without over-reliance on manual experience. Secondly, it can actively generate database optimization plans, reduce manual intervention, and improve inspection efficiency. It also predicts future performance bottleneck trends through time series prediction models, generates early warning information in advance, and supports proactive maintenance. It effectively solves the problems of single data, lack of intelligent analysis, insufficient dynamic monitoring, and inability to provide early warnings in traditional database inspections, significantly improves the intelligence level and operation and maintenance efficiency of database inspections, and has broad application prospects.

[0130] Figure 7 A schematic diagram of the structure of an intelligent inspection device for a database provided in an embodiment of the present application is shown in FIG. Figure 7 As shown, the intelligent inspection device 700 of the database includes:

[0131] The acquisition module 701 is used to obtain a multimodal data set corresponding to the operating status of the inspection database through a preset multi-source heterogeneous data collection method. The establishment and determination module 702 is used to establish a multidimensional data association relationship corresponding to the multimodal data set based on a cross-modal attention mechanism and a self-learning model, and combine historical time series data with unsupervised anomaly detection to determine the database health value and abnormal warning information. Among them, the multidimensional data association relationship is established at least between SQL performance, resource consumption, transaction behavior and business load. The generation module 703 is used to generate a database optimization plan based on the database health value, abnormal warning information and a preset feedback optimization mechanism, and send it to the user terminal.

[0132] The intelligent inspection device 700 of the database also includes: a determination module 704, which is used to determine the future performance bottleneck trend information of the inspection database and the optimization plan corresponding to the future performance bottleneck trend information based on knowledge graph association reasoning, time series prediction model and preset historical health assessment data, and send the future performance bottleneck trend information and the optimization plan corresponding to the future performance bottleneck trend information to the user terminal, so that the user terminal can determine whether to optimize again.

[0133] The acquisition module 701 can specifically:

[0134] Through the database's built-in system tables, log analysis, and external monitoring tools, multi-source heterogeneous data on the operation status of the inspection database is collected in real time. The multi-source heterogeneous data includes at least: SQL execution data, resource consumption data, transaction behavior data, and business load data. Each multi-source heterogeneous data is added to the multimodal data set.

[0135] The acquisition module 701 can also specifically:

[0136] Obtain SQL execution data through the database's built-in system tables. SQL execution data includes at least query text, execution time, and index hit rate. Obtain transaction behavior data through log analysis. Transaction behavior data includes at least transaction commit and rollback rates, lock wait time, and number of concurrent transactions. Collect resource consumption data through external monitoring tools. Resource consumption data includes at least CPU load, memory usage, and disk I / O. Obtain business load data by associating database operation logs with business logs. Business load data includes access peak time and query request distribution.

[0137] The establishment determination module 702 can specifically:

[0138] Extract features from each multi-source heterogeneous data in the multimodal data set, and convert the extracted data features of each dimension into feature comparison vectors. According to each feature comparison vector and the cross-modal attention mechanism, calculate the corresponding association weight matrix between SQL performance, resource consumption, transaction behavior and business load. According to the self-learning model, determine the association information of different data modes of the same multi-source heterogeneous data. The association information at least includes data mode grouping and abnormal mode features. Generate multidimensional data association relationships based on the association weight matrix and association information.

[0139] The establishment determination module 702 can specifically:

[0140] The historical time series data is modeled by the preset Transformer model to generate a baseline model. The degree of deviation between the current data corresponding to the multimodal data set and the baseline model is calculated by the autoencoder to determine the outlier according to the degree of deviation. The degree of deviation is obtained based on the reconstruction error. According to the association between the outlier and the multidimensional data, the outlier is weighted to determine the health sub-value corresponding to each data mode grouping, and the database health value is calculated according to each health sub-value. According to the association between the outlier and the multidimensional data, the cause of the anomaly is determined. According to the database health value and the cause of the anomaly, an abnormal warning information is generated.

[0141] The generation module 703 can specifically:

[0142] Match the database health value and abnormal warning information with the preset optimization plan list to determine the database optimization plan based on the matching results. The database optimization plan includes at least one or more of the following: SQL optimization, resource configuration optimization, and concurrency control optimization. According to the preset feedback optimization mechanism, determine the execution feedback information corresponding to the database optimization plan, and if it is determined that the execution feedback information does not meet the preset optimization conditions, iteratively update the database optimization plan until a database optimization plan that meets the preset optimization conditions is generated.

[0143] In an embodiment of the present application, the preset optimization plan list includes at least: rewriting SQL statements, switching Join algorithms, adjusting work_mem parameters, adjusting checkpoint_timeout parameters, and optimizing lock strategies.

[0144] The determination module 704 can specifically:

[0145] According to the knowledge graph association reasoning, the root cause information of each performance bottleneck in the preset historical health assessment data is determined. Among them, the knowledge graph association reasoning is based on the multi-dimensional data association relationship to extract multi-source heterogeneous data entities as nodes and the relationship between each data as edges. The preset historical health assessment data and its corresponding root cause information of each performance bottleneck are input into the time series prediction model to predict the performance indicator change trend of the inspection database within the preset time length and generate future performance bottleneck trend information. According to the historical health value, historical abnormal warning information and historical optimization plan in the preset historical health assessment data, an optimization plan corresponding to the future performance bottleneck trend information is generated.

[0146] In the application embodiment, the device can also:

[0147] According to the multi-dimensional data association, database health value and abnormal warning information, a knowledge graph of the database health status of the current inspection is constructed, and the knowledge graph of the database health status is sent to the user terminal.

[0148] Each embodiment in this application is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0149] The device and method provided in the embodiments of the present application correspond one to one, and therefore, the device also has similar beneficial technical effects as the corresponding method. Since the beneficial technical effects of the method have been described in detail above, the beneficial technical effects of the device will not be repeated here.

[0150] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0151] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A method for intelligent inspection of a database, characterized in that: The method comprises: By presetting a multi-source heterogeneous data collection method, a multimodal data set corresponding to the operating status of the inspection database is obtained; Based on the cross-modal attention mechanism and the self-learning model, a multidimensional data association relationship corresponding to the multimodal data set is established, and the database health value and abnormal warning information are determined by combining historical time series data with unsupervised anomaly detection; wherein the multidimensional data association relationship is established at least between SQL performance, resource consumption, transaction behavior and business load; Generate a database optimization plan based on the database health value, the abnormal warning information and a preset feedback optimization mechanism, and send it to the user terminal; Among them, based on the cross-modal attention mechanism and the self-learning model, a multidimensional data association relationship corresponding to the multimodal data set is established, specifically including: Extracting features from each multi-source heterogeneous data in the multimodal data set, and converting the extracted data features of each dimension into a feature comparison vector; According to each of the feature comparison vectors and the cross-modal attention mechanism, a corresponding correlation weight matrix between the SQL performance, the resource consumption, the transaction behavior, and the business load is calculated; Determine, according to the self-learning model, association information of different data modes of the same multi-source heterogeneous data; the association information at least includes data mode grouping and abnormal mode characteristics; Generate the multidimensional data association relationship according to the association weight matrix and the association information; Among them, combining historical time series data with unsupervised anomaly detection to determine the database health value and abnormal warning information, specifically including: Modeling the historical time series data by using a preset Transformer model to generate a baseline model; Calculating the degree of deviation between the current data corresponding to the multimodal data set and the baseline model through an autoencoder to determine an outlier according to the degree of deviation; wherein the degree of deviation is obtained based on a reconstruction error; According to the abnormal value and the multi-dimensional data association relationship, the abnormal value is weightedly calculated to determine the health sub-value corresponding to each data pattern grouping, and the database health value is calculated according to each health sub-value; Determine the cause of the abnormality according to the abnormal value and the multidimensional data association relationship; The abnormal warning information is generated according to the database health value and the abnormal cause.

2. The intelligent inspection method of a database according to claim 1, characterized in that: After generating the database optimization plan, the method further includes: Based on knowledge graph association reasoning, time series prediction model and preset historical health assessment data, the future performance bottleneck trend information of the inspection database and the optimization plan corresponding to the future performance bottleneck trend information are determined, and the future performance bottleneck trend information and the optimization plan corresponding to the future performance bottleneck trend information are sent to the user terminal so that the user terminal can determine whether to optimize again.

3. The intelligent inspection method of a database according to claim 1, characterized in that: By presetting the multi-source heterogeneous data collection method, a multimodal data set corresponding to the operation status of the inspection database is obtained, including: Through the database built-in system table, log analysis and external monitoring tools, multi-source heterogeneous data of the inspection database operation status are collected in real time, and the multi-source heterogeneous data at least includes: SQL execution data, resource consumption data, transaction behavior data, and business load data; Each of the multi-source heterogeneous data is added to the multimodal dataset.

4. The intelligent inspection method of a database according to claim 3, characterized in that: Real-time collection of multi-source heterogeneous data on the operation status of the inspection database, specifically including: Obtaining SQL execution data through the database built-in system table, wherein the SQL execution data at least includes query text, execution time and index hit rate; Acquire the transaction behavior data by log analysis, wherein the transaction behavior data at least includes transaction commit and rollback rate, lock waiting time and number of concurrent transactions; Collect the resource consumption data through an external monitoring tool, wherein the resource consumption data at least includes CPU load, memory usage and disk I / O; The business load data is obtained by associating the database operation log with the business log, and the business load data includes the access peak time and the query request distribution.

5. The intelligent inspection method of a database according to claim 1, characterized in that: Generate a database optimization plan based on the database health value, the abnormal warning information and the preset feedback optimization mechanism, specifically including: Matching the database health value and the abnormal warning information with a preset optimization plan list to determine the database optimization plan according to the matching result; the database optimization plan includes at least one or more of the following: SQL optimization, resource allocation optimization, and concurrency control optimization; According to the preset feedback optimization mechanism, the execution feedback information corresponding to the database optimization plan is determined, and when it is judged that the execution feedback information does not meet the preset optimization conditions, the database optimization plan is iteratively updated until the database optimization plan that meets the preset optimization conditions is generated.

6. The intelligent inspection method of a database according to claim 5, characterized in that: The preset optimization scheme list includes at least: rewriting SQL statements, switching Join algorithms, adjusting work_mem parameters, adjusting checkpoint_timeout parameters, and optimizing lock strategies.

7. The intelligent inspection method of a database according to claim 2, characterized in that: Based on the knowledge graph association reasoning, time series prediction model and preset historical health assessment data, the future performance bottleneck trend information of the inspection database and the optimization scheme corresponding to the future performance bottleneck trend information are determined, specifically including: According to the knowledge graph association reasoning, the root cause information of each performance bottleneck in the preset historical health assessment data is determined; wherein the knowledge graph association reasoning is based on the extracted multi-source heterogeneous data as entities and the connections between the multi-source heterogeneous data as relationships; Input the preset historical health assessment data and the corresponding root cause information of each performance bottleneck into the time series prediction model to predict the performance indicator change trend of the inspection database within a preset time period and generate the future performance bottleneck trend information; According to the historical health values, historical abnormal warning information and historical optimization plans in the preset historical health assessment data, an optimization plan corresponding to the future performance bottleneck trend information is generated.

8. An intelligent inspection device for a database, characterized in that: The device comprises: An acquisition module is used to acquire a multimodal data set corresponding to the operation status of the inspection database through a preset multi-source heterogeneous data acquisition method; Establish a determination module for establishing a multidimensional data association relationship corresponding to the multimodal data set based on a cross-modal attention mechanism and a self-learning model, and determining a database health value and abnormal warning information in combination with historical time series data and unsupervised anomaly detection; wherein the multidimensional data association relationship is established at least between SQL performance, resource consumption, transaction behavior and business load; A generation module, used to generate a database optimization plan according to the database health value, the abnormal warning information and a preset feedback optimization mechanism, and send the plan to the user terminal; Among them, establishing a determination module can specifically: Extracting features from each multi-source heterogeneous data in the multimodal data set, and converting the extracted data features of each dimension into a feature comparison vector; According to each of the feature comparison vectors and the cross-modal attention mechanism, a corresponding correlation weight matrix between the SQL performance, the resource consumption, the transaction behavior, and the business load is calculated; Determine, according to the self-learning model, association information of different data modes of the same multi-source heterogeneous data; the association information at least includes data mode grouping and abnormal mode characteristics; Generate the multidimensional data association relationship according to the association weight matrix and the association information; Among them, establishing a determination module can specifically: Modeling the historical time series data by using a preset Transformer model to generate a baseline model; Calculating the degree of deviation between the current data corresponding to the multimodal data set and the baseline model through an autoencoder to determine an outlier according to the degree of deviation; wherein the degree of deviation is obtained based on a reconstruction error; According to the abnormal value and the multi-dimensional data association relationship, the abnormal value is weightedly calculated to determine the health sub-value corresponding to each data pattern grouping, and the database health value is calculated according to each health sub-value; Determine the cause of the abnormality according to the abnormal value and the multidimensional data association relationship; The abnormal warning information is generated according to the database health value and the abnormal cause.

Images