Visual table management method and system for different types of databases
By building a general database operating system and using deep reinforcement learning and graph neural network technology, the problems of complex cross-border database operations in the existing technology, inadequate query optimization and insufficient abnormal detection capabilities are solved, and efficient query performance and system stability are achieved.
Patent Information
- Application Number
- CN202510091350.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2025-01-21
- Publication Date
- 2025-04-22
- Estimated Expiration
- 2045-01-21
AI Technical Summary
The existing technology lacks a unified database operation interface, the query optimization strategy is not intelligent enough, and the abnormal detection and processing capabilities are insufficient, resulting in complex cross-database operations, low query performance and poor system stability.
Provides visual table management methods and systems for different types of databases. By building a common database operating system, including a common operation interface and dialect abstract base class, it realizes unified management across databases. Use deep reinforcement learning to predict the optimal database connection method, generate a multi-stage execution plan, and identify abnormal patterns and propagation paths through an abnormal feature knowledge base based on graph neural networks, and formulate preventive processing strategies.
It realizes a unified operation interface across databases, improves query efficiency and system stability, reduces database failures and data security risks, and provides a more friendly way to access data.
Smart Images

Figure CN119537424B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to computer technology, and in particular to a method and system for managing visualization tables of different types of databases. Background Art
[0002] Database visualization and management technology continues to develop with the growth of data scale and the complexity of application scenarios. The existing technology has the following defects and shortcomings:
[0003] Lack of a unified database operation interface: Different database systems usually use different APIs and query languages, which makes it difficult for users to learn and use, and also increases the complexity of cross-database operations. Existing visualization tools usually only support specific database types and cannot provide a unified management and operation interface across databases.
[0004] Query optimization strategies are not smart enough: Traditional database query optimizers are usually based on static rules and statistical information, which makes it difficult to adapt to complex query scenarios and dynamic data changes. In the case of large data volumes and high concurrency, query performance becomes a bottleneck, requiring more intelligent query optimization strategies.
[0005] Insufficient anomaly detection and processing capabilities: Database systems may encounter various anomalies during operation, such as hardware failures, software errors, malicious attacks, etc. Existing database systems usually lack effective anomaly detection and processing mechanisms, making it difficult to detect and handle anomalies in a timely manner, thus affecting the stability and reliability of the system. Summary of the invention
[0006] The embodiments of the present invention provide a method and system for managing visualized tables of different types of databases, which can solve the problems in the prior art.
[0007] According to a first aspect of the embodiments of the present invention,
[0008] Provides visual table management methods for different types of databases, including:
[0009] Construct a database universal operating system including a universal operation interface and a dialect abstract base class, wherein the universal operation interface includes a table structure manager, a data queryer and a data updater, and the dialect abstract base class includes an SQL syntax parser converter, a distributed transaction processor and a performance indicator collector; the SQL syntax parser converter performs syntax conversion based on an abstract syntax tree, the distributed transaction processor sets an operation checkpoint at a database operation node and records data status, and the performance indicator collector collects performance data in real time, wherein the performance data includes query response time, resource occupancy rate and concurrent processing number;
[0010] Establish a query performance evaluation system based on the performance data and obtain query cost evaluation results; based on the query cost evaluation results, predict the optimal database connection method through deep reinforcement learning and generate a multi-stage execution plan; construct an abnormal feature knowledge base based on a graph neural network, construct an abnormal prediction method based on the abnormal feature knowledge base, identify abnormal patterns through multi-dimensional feature similarity calculation, deduce abnormal propagation paths in combination with time series correlation analysis, formulate preventive processing strategies based on the abnormal prediction results, record the processing effects, and optimize the abnormal prediction methods and preventive processing strategies based on the processing effects;
[0011] The database operation request is sent to the SQL syntax parser converter for syntax adaptation, and the multi-stage execution plan is executed. During the execution process, the distributed transaction processor dynamically adjusts the transaction isolation level and concurrency control parameters based on the recommendation results of the abnormal feature knowledge base, and the performance indicator collector records the execution process data and updates the query performance evaluation system; when the query response is returned, the corresponding data display method is selected for visual presentation based on the terminal device type and the amount of query result data.
[0012] In an optional embodiment,
[0013] The steps of establishing a query performance evaluation system based on the performance data and obtaining a query cost evaluation result include:
[0014] A query performance evaluation system is established based on performance data, and the performance data is standardized. The query response time is converted by the quantile normalization method, the resource occupancy rate is processed by the minimum and maximum value normalization method, and the concurrent processing number is normalized by the standard score after logarithmic transformation; the difference coefficient of the performance data is calculated by information entropy, and the dynamic weight coefficient of the performance data is determined based on the difference coefficient; the performance data is subjected to continuous wavelet transformation to obtain time-frequency analysis results, and the performance fluctuation pattern is identified based on the time-frequency analysis results, and the performance fluctuation feature vector is constructed, and the time series change law of the performance data is extracted to obtain the performance time series characteristics;
[0015] A query cost evaluation function is constructed based on the dynamic weight coefficient, the performance fluctuation feature vector and the performance time series feature, wherein the weight of the performance fluctuation feature vector and the weight of the performance time series feature are determined to have an optimal configuration through a grid search method; a balance factor of the query cost evaluation function is adaptively adjusted based on the feature variance to obtain an initial query cost evaluation result;
[0016] A cross-validation method is used to evaluate the initial query cost evaluation result, the evaluation accuracy is verified by root mean square error and mean absolute percentage error, and a regularization term is introduced into the query cost evaluation function based on the verification result; the confidence interval of the initial query cost evaluation result is calculated, and the query cost evaluation function is optimized and adjusted according to the calculation result to obtain the final query cost evaluation result.
[0017] In an optional embodiment,
[0018] Based on the query cost evaluation result, the steps of predicting the optimal database connection method through deep reinforcement learning and generating a multi-stage execution plan include:
[0019] Construct a deep reinforcement learning framework including an Actor network and a Critic network. The Actor network adopts a multi-layer fully connected network structure. The input layer receives the query cost evaluation result, the table connection relationship diagram and the resource state vector. The hidden layer adopts the ReLU activation function. The output layer outputs the probability distribution of the connection mode through the Softmax function.
[0020] Construct a global state vector, the global state vector includes a query cost evaluation result vector, a structural feature vector generated based on the initial table connection relationship graph, and a resource state vector; the query cost evaluation result vector includes a standardized evaluation value and upper and lower bounds of a confidence interval used to characterize the relative uncertainty of the query cost evaluation result; the resource state vector includes processor utilization, memory utilization, input and output waiting time, and cache hit rate; the structural feature vector includes the number of connected tables, the selectivity of the connection predicate, the size of the table, and the cardinality of the connection key; design a comprehensive reward function, the comprehensive reward function includes an execution time reward, a resource consumption reward, and an intermediate result set reward, wherein the execution time reward is calculated based on the ratio of the actual execution time to the benchmark execution time, the resource consumption reward is calculated based on the resource utilization, and the intermediate result set reward is calculated based on the ratio of the actual result set size to the estimated value; dynamically adjust the weight of the execution time reward in the comprehensive reward function based on the relative uncertainty, and reduce the weight of the execution time reward when the relative uncertainty increases;
[0021] The input query statement is parsed into an initial connection graph, and based on the global state vector and the comprehensive reward function, a preliminary multi-stage execution plan is iteratively generated through a deep reinforcement learning framework. In each iterative step, the action probability distribution is obtained through the Actor network based on the current state, and the optimal connection method is selected using an adaptively adjusted exploration strategy. The connection graph and the global state vector are updated according to the selected optimal connection method; during the query execution, the deviation between the actual execution cost and the predicted cost is monitored, and re-optimization is triggered when the deviation exceeds a preset threshold, and the state space of the deep reinforcement learning framework is updated using the actual execution data and a final multi-stage execution plan including a query decomposition strategy, parallelism configuration, and resource allocation scheme is regenerated; the prediction error is calculated based on the actual execution time and resource consumption, and the loss function weights of the Actor network and the Critic network are adjusted according to the prediction error.
[0022] In an optional embodiment,
[0023] The steps of constructing an abnormal feature knowledge base based on graph neural network, constructing an abnormal prediction method based on the abnormal feature knowledge base, identifying abnormal patterns through multi-dimensional feature similarity calculation, and deducing abnormal propagation paths in combination with time series correlation analysis, and formulating preventive processing strategies according to abnormal prediction results include:
[0024] The monitoring indicators are organized into an attribute graph structure, where nodes represent monitoring objects and edges represent dependencies between monitoring objects. The monitoring indicators include the performance data and query cost evaluation results. The time series features, statistical features and topological features of the nodes are extracted to construct a multidimensional feature vector. The time series features include moving average, standard deviation and kurtosis. The statistical features include indicator distribution skewness and interquartile range. The topological features extract the connection mode and interaction relationship of the nodes in the local structure through the graph attention mechanism to form the local structure features of the nodes. The long short-term memory network is used to perform multi-scale temporal encoding on the historical multidimensional feature sequence of the nodes, and the multidimensional feature vectors are fused to obtain the comprehensive feature representation of the nodes. The comprehensive feature representation is input into the encoder-decoder network to learn the latent space representation of the abnormal pattern and construct an abnormal feature knowledge base.
[0025] The Mahalanobis distance is used to measure the similarity between the current system state and the abnormal pattern in the abnormal feature knowledge base, and the Markov state transition matrix is constructed in combination with the execution state of the multi-stage execution plan to describe the abnormal type transition probability; the abnormal patterns in the abnormal feature knowledge base are divided into groups using a density-based clustering method, and mixed abnormalities are identified by combining fast nearest neighbor search; the graph attention network based on the LeakyReLU activation function uses the local structural features of the nodes to calculate the influence weights between nodes, and uses the dynamic time warping algorithm to align the node state sequence, calculate the conditional mutual information and transfer entropy of the nodes to quantify the abnormal propagation intensity and direction, analyze the temporal correlation of the node state sequence to derive the abnormal propagation path, and construct the abnormal propagation probability graph in combination with the influence weights between nodes;
[0026] Based on the abnormal pattern recognition results and the abnormal propagation probability map, the abnormal prediction results are output, and the abnormal prediction results include the abnormal occurrence probability, abnormal type and impact range; a preventive processing strategy is formulated according to the abnormal prediction results, and a multi-objective reward function is constructed with the abnormal occurrence probability, performance impact and resource overhead. The decision optimization is performed by training the processing strategy selector through the deep Q network.
[0027] In an optional embodiment,
[0028] The steps of constructing a multi-objective reward function based on the probability of anomaly occurrence, performance impact, and resource overhead and optimizing the decision-making of the policy selector through deep Q network training include:
[0029] The multi-objective reward function integrates the abnormality occurrence probability reward, the performance impact reward and the resource expenditure reward in a linear weighted manner, wherein the abnormality occurrence probability reward is calculated based on the abnormality prediction probability, the performance impact reward is calculated based on the ratio of the performance change before and after the strategy execution to the performance benchmark value, and the resource expenditure reward is calculated based on the ratio of the resource consumption required for the strategy execution to the maximum resource consumption, and the weight coefficient of each reward is determined by grid search;
[0030] Construct a state space, which includes a discrete state space and a continuous state space, wherein the discrete state space contains an anomaly type set, an impact range level, and a resource allocation state, and the continuous state space is represented by a vector of normalized values of performance indicators; design an adaptive exploration strategy, adaptively adjust the decay rate based on the cumulative reward of a fixed number of historical steps, and introduce a priority experience replay mechanism, which calculates sample priority and determines sampling probability based on temporal difference error;
[0031] A dual-depth Q network architecture is used to train a processing strategy selector, wherein the dual-depth Q network includes a main network and a target network, both of which adopt a multi-layer fully connected structure, and the action output of the preventive processing strategy is obtained based on the input training of the state space; the processing strategy selector is trained based on the adaptive exploration strategy and the multi-objective reward function, the training samples are stored in an experience replay pool, the training samples are selected according to the sampling probability of the priority experience replay mechanism, and the target network parameters are updated regularly; based on the action sequence output by the processing strategy selector and the corresponding reward value evaluation, the optimal preventive processing strategy is selected and executed.
[0032] In an optional embodiment,
[0033] The steps of dynamically adjusting the transaction isolation level and concurrency control parameters of the distributed transaction processor based on the recommendation result of the abnormal feature knowledge base include:
[0034] Based on the abnormal pattern recognition results and historical processing effects in the abnormal feature knowledge base, a quantum genetic algorithm is used to predict transaction conflicts and adjust transaction isolation levels. The quantum genetic algorithm includes a ternary quantum bit encoding scheme to represent the transaction isolation state, reflects the probability distribution of different transaction isolation levels through quantum amplitude, maps transaction features to quantum states through quantum phase encoding, and designs an adaptive quantum rotation gate to adaptively update the ternary quantum bits.
[0035] Based on the transaction conflict prediction results, a hybrid optimization algorithm is used to optimize the concurrency control parameters. The hybrid optimization algorithm represents the parameter space through real number coding, combines the temperature control mechanism of the simulated annealing algorithm and the group search characteristics of the particle swarm algorithm, and uses the speed position update formula of the particle swarm algorithm to search the parameter space, in which the inertia weight of the particles is dynamically adjusted with the simulated annealing temperature; the Metropolis criterion is used to determine the acceptance probability of the particle position update, and the adjustment factor of the Metropolis criterion changes with the number of iterations; the temperature control mechanism is used to balance the global search and the local search;
[0036] Construct a parameter tuning closed loop based on online Bayesian optimization, including a probability regression model based on Gaussian process and a knowledge gradient acquisition function. The hyperparameters of the probability regression model are updated online through maximum likelihood estimation. The knowledge gradient acquisition function is used to optimize the selection of concurrent control parameters.
[0037] The transaction isolation level prediction result of the quantum genetic algorithm is used to guide the parameter search direction of the hybrid optimization algorithm. The optimization result of the hybrid optimization algorithm is used as the prior knowledge of the online Bayesian optimization. The feedback result of the online Bayesian optimization is used to adjust the evolution strategy of the quantum genetic algorithm to form a closed-loop optimization.
[0038] In an optional embodiment,
[0039] A quantum genetic algorithm is used to predict transaction conflicts and adjust transaction isolation levels. The quantum genetic algorithm includes a ternary quantum bit encoding scheme to represent the transaction isolation state, reflects the probability distribution of different transaction isolation levels through quantum amplitude, maps transaction characteristics to quantum states through quantum phase encoding, and designs an adaptive quantum revolving gate to adaptively update the ternary quantum bits. The steps include:
[0040] A ternary quantum bit encoding scheme is used to map the transaction isolation level to a complex Hilbert space. The ternary quantum bit encoding scheme uses three orthogonal ground states corresponding to read uncommitted, read committed, and serializable, respectively. The quantum amplitude of the ternary quantum bit is used to characterize the superposition state of the transaction isolation state. The quantum amplitude satisfies the normalization condition and is used to reflect the probability distribution of different transaction isolation levels. The historical statistical information of the database transaction is used as an initialization parameter, and the initial distribution of the quantum amplitude is set according to the transaction history data. The square of the modulus of the quantum amplitude represents the probability of the corresponding transaction isolation level occurring during the transaction execution process.
[0041] Mapping the transaction feature vector to the quantum phase space through quantum phase encoding to generate a feature quantum state, mapping the components of the transaction feature vector to quantum phase angles through trigonometric transformation, and constructing a composite state from the feature quantum state and the ternary quantum bit through a tensor product operation, wherein the composite state maintains the correlation between the transaction feature and the transaction isolation state through quantum entanglement;
[0042] An adaptive quantum revolving gate is designed to adaptively update a ternary quantum bit. The quantum revolving gate includes a rotation angle parameter and a phase parameter. The parameters of the quantum revolving gate are iteratively updated by a parameter optimization method based on gradient descent. The parameter optimization method adopts a multi-objective fitness function including conflict probability, performance index and deadlock risk. The quantum revolving gate is applied to the ternary quantum bit to obtain a new transaction isolation state. The probability distribution of the new transaction isolation state reflects the dynamic adjustment result of the transaction isolation level. The quantum state superposition result of the characteristic quantum state and the initial probability distribution determines the iterative optimization direction of the quantum revolving gate parameters through the gradient calculation of the multi-objective fitness function.
[0043] According to a second aspect of the embodiments of the present invention,
[0044] Provides different types of database visualization table management systems, including:
[0045] The first unit is used to construct a database universal operating system including a universal operation interface and a dialect abstract base class, wherein the universal operation interface includes a table structure manager, a data queryer and a data updater, and the dialect abstract base class includes an SQL syntax parser converter, a distributed transaction processor and a performance indicator collector; the SQL syntax parser converter performs syntax conversion based on an abstract syntax tree, the distributed transaction processor sets an operation checkpoint at a database operation node and records data status, and the performance indicator collector collects performance data in real time, wherein the performance data includes query response time, resource occupancy rate and concurrent processing number;
[0046] The second unit is used to establish a query performance evaluation system based on the performance data and obtain a query cost evaluation result; based on the query cost evaluation result, predict the optimal database connection method through deep reinforcement learning, and generate a multi-stage execution plan; construct an abnormal feature knowledge base based on a graph neural network, construct an abnormal prediction method based on the abnormal feature knowledge base, identify abnormal patterns through multi-dimensional feature similarity calculation, deduce abnormal propagation paths in combination with time series correlation analysis, formulate preventive processing strategies based on abnormal prediction results, and record the processing effects, and optimize abnormal prediction methods and preventive processing strategies based on the processing effects;
[0047] The third unit is used to send the database operation request to the SQL syntax parsing converter for syntax adaptation and execute the multi-stage execution plan. During the execution process, the distributed transaction processor dynamically adjusts the transaction isolation level and concurrency control parameters based on the recommendation results of the abnormal feature knowledge base, and the performance indicator collector records the execution process data and updates the query performance evaluation system; when the query response is returned, the corresponding data display method is selected for visual presentation based on the terminal device type and the amount of query result data.
[0048] According to a third aspect of the embodiments of the present invention,
[0049] An electronic device is provided, comprising:
[0050] processor;
[0051] a memory for storing processor-executable instructions;
[0052] The processor is configured to call the instructions stored in the memory to execute the aforementioned method.
[0053] According to a fourth aspect of the embodiments of the present invention,
[0054] A computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the aforementioned method is implemented.
[0055] The present invention predicts the optimal database connection mode and generates a multi-stage execution plan by establishing a query performance evaluation system and a query cost evaluation based on deep reinforcement learning, thereby significantly improving query efficiency.
[0056] The present invention constructs an abnormal feature knowledge base and an abnormality prediction method based on graph neural network, which can effectively identify abnormal patterns, predict abnormal propagation paths, and formulate preventive processing strategies, thereby reducing database failures and ensuring data security.
[0057] The present invention selects a corresponding data display method for visual presentation according to the terminal device type and the amount of query result data, thereby providing a more friendly and convenient data access method. BRIEF DESCRIPTION OF THE DRAWINGS
[0058] Figure 1 It is a flowchart of a method for managing visualized tables of different types of databases according to an embodiment of the present invention;
[0059] Figure 2 It is a schematic diagram of the structure of different types of database visualization table management systems according to an embodiment of the present invention. DETAILED DESCRIPTION
[0060] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0061] The technical solution of the present invention is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.
[0062] Figure 1 FIG. 1 is a flow chart of a method for managing visualized tables of different types of databases according to an embodiment of the present invention. Figure 1 As shown, the method includes:
[0063] S1. Construct a database general operating system including a general operation interface and a dialect abstract base class, wherein the general operation interface includes a table structure manager, a data queryer and a data updater, and the dialect abstract base class includes an SQL syntax parser converter, a distributed transaction processor and a performance indicator collector; the SQL syntax parser converter performs syntax conversion based on an abstract syntax tree, the distributed transaction processor sets an operation checkpoint at a database operation node and records data status, and the performance indicator collector collects performance data in real time, wherein the performance data includes query response time, resource occupancy rate and concurrent processing number;
[0064] S2. Establish a query performance evaluation system based on the performance data and obtain a query cost evaluation result; based on the query cost evaluation result, predict the optimal database connection method through deep reinforcement learning, and generate a multi-stage execution plan; construct an abnormal feature knowledge base based on a graph neural network, construct an abnormal prediction method based on the abnormal feature knowledge base, identify abnormal patterns through multi-dimensional feature similarity calculation, deduce abnormal propagation paths in combination with time series correlation analysis, formulate preventive processing strategies based on the abnormal prediction results, and record the processing effects, and optimize the abnormal prediction methods and preventive processing strategies based on the processing effects;
[0065] S3. Send the database operation request to the SQL syntax parser converter for syntax adaptation, and execute the multi-stage execution plan. During the execution process, the distributed transaction processor dynamically adjusts the transaction isolation level and concurrency control parameters based on the recommendation results of the abnormal feature knowledge base, and the performance indicator collector records the execution process data and updates the query performance evaluation system; when the query response is returned, the corresponding data display method is selected for visual presentation based on the terminal device type and the amount of query result data.
[0066] In an optional embodiment,
[0067] The steps of establishing a query performance evaluation system based on the performance data and obtaining a query cost evaluation result include:
[0068] A query performance evaluation system is established based on performance data, and the performance data is standardized. The query response time is converted by the quantile normalization method, the resource occupancy rate is processed by the minimum and maximum value normalization method, and the concurrent processing number is normalized by the standard score after logarithmic transformation; the difference coefficient of the performance data is calculated by information entropy, and the dynamic weight coefficient of the performance data is determined based on the difference coefficient; the performance data is subjected to continuous wavelet transformation to obtain time-frequency analysis results, and the performance fluctuation pattern is identified based on the time-frequency analysis results, and the performance fluctuation feature vector is constructed, and the time series change law of the performance data is extracted to obtain the performance time series characteristics;
[0069] A query cost evaluation function is constructed based on the dynamic weight coefficient, the performance fluctuation feature vector and the performance time series feature, wherein the weight of the performance fluctuation feature vector and the weight of the performance time series feature are determined to have an optimal configuration through a grid search method; a balance factor of the query cost evaluation function is adaptively adjusted based on the feature variance to obtain an initial query cost evaluation result;
[0070] A cross-validation method is used to evaluate the initial query cost evaluation result, the evaluation accuracy is verified by root mean square error and mean absolute percentage error, and a regularization term is introduced into the query cost evaluation function based on the verification result; the confidence interval of the initial query cost evaluation result is calculated, and the query cost evaluation function is optimized and adjusted according to the calculation result to obtain the final query cost evaluation result.
[0071] Exemplarily, first, the collected query performance data is standardized. Specifically, the query response time is converted using the quantile normalization method. For example, the response time is sorted from small to large, and then the data is divided into different intervals according to the set quantiles (such as 25%, 50%, 75%), and the values in each interval are mapped to between 0 and 1. The resource occupancy rate is processed using the minimum and maximum value normalization method, that is, the original data is linearly transformed into the range of 0 to 1. The number of concurrent processing is first logarithmically transformed, and then standardized by standard scores, that is, the mean value is subtracted from each data point and then divided by the standard deviation.
[0072] Next, calculate the coefficient of variance of the performance data. Use information entropy to measure the degree of variance of each performance indicator. The larger the information entropy, the more dispersed the data distribution of the indicator is and the greater the variance is. According to the calculated coefficient of variance, determine the dynamic weight coefficient of each performance indicator. The larger the coefficient of variance, the larger the corresponding weight coefficient is, which means that when evaluating the query cost, the more important performance indicators will occupy a larger proportion. For example, if the coefficient of variance of the number of concurrent processing is the largest, its weight coefficient is also the largest.
[0073] Then, the performance data is subjected to time-frequency analysis and time series analysis. Continuous wavelet transform is used to perform time-frequency analysis on the performance data, identify the performance fluctuation pattern, and construct the performance fluctuation feature vector. For example, the energy spectrum and frequency distribution after wavelet transform can be extracted to form the performance fluctuation feature vector. At the same time, the time series variation law of the performance data is extracted. For example, the moving average and autocorrelation coefficient of the performance index can be calculated to obtain the performance time series characteristics.
[0074] Based on the dynamic weight coefficient, performance fluctuation feature vector and performance timing characteristics obtained above, a query cost evaluation function is constructed. The weights of the performance fluctuation feature vector and the performance timing characteristics are used to determine the optimal configuration through the grid search method. For example, different weight combinations can be set, and then the accuracy of the query cost evaluation function under each combination can be evaluated, and finally the weight combination with the highest accuracy can be selected. The balance factor of the query cost evaluation function is adaptively adjusted based on the feature variance to obtain the initial query cost evaluation result. The larger the feature variance, the smaller the balance factor to avoid excessive influence of certain features on the evaluation results.
[0075] The cross-validation method is used to evaluate the initial results of the query cost evaluation. Indicators such as root mean square error and mean absolute percentage error are used to verify the evaluation accuracy. For example, the dataset is divided into a training set and a test set, the query cost evaluation function is trained with the training set data, and then its accuracy is evaluated with the test set data. Based on the verification results, a regularization term is introduced into the query cost evaluation function to prevent overfitting.
[0076] Finally, the confidence interval of the initial query cost evaluation result is calculated. The query cost evaluation function is optimized and adjusted according to the calculation result. For example, if the confidence interval is too wide, the function parameters need to be adjusted or the training data needs to be increased to improve the reliability of the evaluation result. Finally, a more accurate query cost evaluation result is obtained.
[0077] The present invention comprehensively considers various aspects of performance data and combines time-frequency characteristics and timing rules, which can evaluate the query cost more comprehensively and accurately; through dynamic weight coefficients and adaptive balancing factors, it can effectively reduce the impact of abnormal data and noise and improve the robustness of the model; through standardization processing and feature extraction, it can effectively reduce data dimensions and improve evaluation efficiency.
[0078] In an optional embodiment,
[0079] Based on the query cost evaluation result, the steps of predicting the optimal database connection method through deep reinforcement learning and generating a multi-stage execution plan include:
[0080] Construct a deep reinforcement learning framework including an Actor network and a Critic network. The Actor network adopts a multi-layer fully connected network structure. The input layer receives the query cost evaluation result, the table connection relationship diagram and the resource state vector. The hidden layer adopts the ReLU activation function. The output layer outputs the probability distribution of the connection mode through the Softmax function.
[0081] Construct a global state vector, the global state vector includes a query cost evaluation result vector, a structural feature vector generated based on the initial table connection relationship graph, and a resource state vector; the query cost evaluation result vector includes a standardized evaluation value and upper and lower bounds of a confidence interval used to characterize the relative uncertainty of the query cost evaluation result; the resource state vector includes processor utilization, memory utilization, input and output waiting time, and cache hit rate; the structural feature vector includes the number of connected tables, the selectivity of the connection predicate, the size of the table, and the cardinality of the connection key; design a comprehensive reward function, the comprehensive reward function includes an execution time reward, a resource consumption reward, and an intermediate result set reward, wherein the execution time reward is calculated based on the ratio of the actual execution time to the benchmark execution time, the resource consumption reward is calculated based on the resource utilization, and the intermediate result set reward is calculated based on the ratio of the actual result set size to the estimated value; dynamically adjust the weight of the execution time reward in the comprehensive reward function based on the relative uncertainty, and reduce the weight of the execution time reward when the relative uncertainty increases;
[0082] The input query statement is parsed into an initial connection graph, and based on the global state vector and the comprehensive reward function, a preliminary multi-stage execution plan is iteratively generated through a deep reinforcement learning framework. In each iterative step, the action probability distribution is obtained through the Actor network based on the current state, and the optimal connection method is selected using an adaptively adjusted exploration strategy. The connection graph and the global state vector are updated according to the selected optimal connection method; during the query execution, the deviation between the actual execution cost and the predicted cost is monitored, and re-optimization is triggered when the deviation exceeds a preset threshold, and the state space of the deep reinforcement learning framework is updated using the actual execution data and a final multi-stage execution plan including a query decomposition strategy, parallelism configuration, and resource allocation scheme is regenerated; the prediction error is calculated based on the actual execution time and resource consumption, and the loss function weights of the Actor network and the Critic network are adjusted according to the prediction error.
[0083] Exemplarily, first, a deep reinforcement learning framework is constructed. The core of the framework is the Actor network and the Critic network. The Actor network is responsible for outputting the probability distribution of various connection methods based on the input information. Its structure is a multi-layer fully connected network, and the input layer receives information such as query cost evaluation results, table connection relationship diagrams, and resource state vectors. The hidden layer uses the ReLU activation function, and the output layer uses the Softmax function to output the probability distribution of the connection method, indicating the possibility of selecting each connection method. The Critic network is responsible for evaluating the value of the current state, which is used to guide the learning of the Actor network.
[0084] Next, a global state vector is constructed to describe the overall environment of query execution. The global state vector consists of three parts: query cost evaluation result vector, structural feature vector, and resource state vector. The query cost evaluation result vector contains the standardized evaluation value and the upper and lower bounds of the confidence interval, which is used to characterize the uncertainty of the evaluation result. For example, the cost evaluation value of a certain join operation is 10, and the confidence interval is [8, 12], which means that the actual cost falls between 8 and 12. The structural feature vector describes the topological structure of the join graph, such as the number of join tables, the selectivity of the join predicate, the size of the table, and the cardinality of the join key. For example, if a join graph contains three tables and the selectivity of the join predicate is 0.1, the structural feature vector contains this information. The resource state vector describes the current system resource usage, such as processor usage, memory usage, input and output waiting time, and cache hit rate. For example, the current CPU usage is 80% and the memory usage is 50%. This information will be included in the resource state vector.
[0085] Then, a comprehensive reward function is designed to evaluate the quality of the execution plan. The comprehensive reward function consists of three parts: execution time reward, resource consumption reward, and intermediate result set reward. The execution time reward is calculated based on the ratio of the actual execution time to the benchmark execution time. For example, if the actual execution time is 10 seconds and the benchmark execution time is 20 seconds, the execution time reward is 0.5. The resource consumption reward is calculated based on the resource utilization rate. For example, the higher the CPU utilization rate, the lower the resource consumption reward. The intermediate result set reward is calculated based on the ratio of the actual result set size to the estimated value. For example, if the actual result set size is 1000 rows and the estimated value is 800 rows, the intermediate result set reward will be calculated based on this deviation. In addition, the system dynamically adjusts the weight of the execution time reward based on the relative uncertainty of the query cost evaluation result. When the relative uncertainty increases, the weight of the execution time reward is reduced to avoid over-reliance on uncertain evaluation results.
[0086] Next, the input query statement is parsed into an initial connection graph. Based on the global state vector and the comprehensive reward function, a preliminary multi-stage execution plan is iteratively generated through the deep reinforcement learning framework. In each iterative step, the Actor network outputs the action probability distribution according to the current state, such as the probability of selecting different connection methods. The system uses an adaptively adjusted exploration strategy to select the optimal connection method. For example, it is more inclined to explore different connection methods in the early stage, and more inclined to select known better connection methods in the later stage. The connection graph and global state vector are updated according to the selected optimal connection method.
[0087] During query execution, the system monitors the deviation between the actual execution cost and the predicted cost. When the deviation exceeds the preset threshold, the reoptimization mechanism is triggered. The system uses the actual execution data to update the state space of the deep reinforcement learning framework and regenerate the final multi-stage execution plan that includes the query decomposition strategy, parallelism configuration, and resource allocation scheme. For example, if the actual execution cost of a connection operation is much higher than the predicted cost, the system will adjust the strategy for subsequent connection operations.
[0088] Finally, the system calculates the prediction error based on the actual execution time and resource consumption, and adjusts the loss function weights of the Actor network and Critic network according to the prediction error to continuously improve the prediction accuracy of the model.
[0089] The present invention dynamically optimizes the execution plan through deep reinforcement learning, which can effectively reduce the execution time of the query, thereby improving the query efficiency. It can adaptively adjust the execution plan according to the query cost evaluation results and the system resource status to cope with different queries and system environments; by optimizing the execution plan, it can effectively reduce resource consumption during the query process, such as CPU utilization and memory utilization.
[0090] In an optional embodiment,
[0091] The steps of constructing an abnormal feature knowledge base based on graph neural network, constructing an abnormal prediction method based on the abnormal feature knowledge base, identifying abnormal patterns through multi-dimensional feature similarity calculation, and deducing abnormal propagation paths in combination with time series correlation analysis, and formulating preventive processing strategies according to abnormal prediction results include:
[0092] The monitoring indicators are organized into an attribute graph structure, where nodes represent monitoring objects and edges represent dependencies between monitoring objects. The monitoring indicators include the performance data and query cost evaluation results. The time series features, statistical features and topological features of the nodes are extracted to construct a multidimensional feature vector. The time series features include moving average, standard deviation and kurtosis. The statistical features include indicator distribution skewness and interquartile range. The topological features extract the connection mode and interaction relationship of the nodes in the local structure through the graph attention mechanism to form the local structure features of the nodes. The long short-term memory network is used to perform multi-scale temporal encoding on the historical multidimensional feature sequence of the nodes, and the multidimensional feature vectors are fused to obtain the comprehensive feature representation of the nodes. The comprehensive feature representation is input into the encoder-decoder network to learn the latent space representation of the abnormal pattern and construct an abnormal feature knowledge base.
[0093] The Mahalanobis distance is used to measure the similarity between the current system state and the abnormal pattern in the abnormal feature knowledge base, and the Markov state transition matrix is constructed in combination with the execution state of the multi-stage execution plan to describe the abnormal type transition probability; the abnormal patterns in the abnormal feature knowledge base are divided into groups using a density-based clustering method, and mixed abnormalities are identified by combining fast nearest neighbor search; the graph attention network based on the LeakyReLU activation function uses the local structural features of the nodes to calculate the influence weights between nodes, and uses the dynamic time warping algorithm to align the node state sequence, calculate the conditional mutual information and transfer entropy of the nodes to quantify the abnormal propagation intensity and direction, analyze the temporal correlation of the node state sequence to derive the abnormal propagation path, and construct the abnormal propagation probability graph in combination with the influence weights between nodes;
[0094] Based on the abnormal pattern recognition results and the abnormal propagation probability map, the abnormal prediction results are output, and the abnormal prediction results include the abnormal occurrence probability, abnormal type and impact range; a preventive processing strategy is formulated according to the abnormal prediction results, and a multi-objective reward function is constructed with the abnormal occurrence probability, performance impact and resource overhead. The decision optimization is performed by training the processing strategy selector through the deep Q network.
[0095] Exemplarily, first, collect various monitoring indicators of the system, including but not limited to CPU usage, memory occupancy, disk IO, network traffic, database query response time, and estimated query cost. Organize these monitoring indicators into a property graph structure. The nodes in the graph represent the monitored objects, such as servers, databases, applications, etc. The edges between nodes represent the dependencies between the monitored objects, such as service call relationships, data flows, etc. For example, there is an edge between a Web server node and a database server node, indicating that the Web server depends on the database server. Each node and edge has attributes, which are the corresponding monitoring indicator values.
[0096] Next, the multi-dimensional features of the nodes are extracted from the attribute graph to construct feature vectors. These features include: time series features, such as the moving average, standard deviation, kurtosis, etc. over a period of time, which reflect the changing trend and fluctuation of the indicator value; statistical features, such as the distribution skewness and interquartile range of the indicator value, which reflect the distribution form of the indicator value; topological features, which extract the connection mode and interaction relationship of the nodes in the local structure through the graph attention mechanism, such as the degree, centrality, and characteristics of neighbor nodes of the nodes. For example, the topological features of a database server node can include the number of applications it connects to, the average query response time it serves, etc. The graph attention mechanism assigns different weights according to the importance of neighbor nodes to better capture the local structural information of the nodes.
[0097] The long short-term memory network is used to perform multi-scale temporal encoding on the historical multi-dimensional feature sequence of the node, and the feature information of different time scales is fused. Then, the fused multi-dimensional feature vector is input into an encoder-decoder network to learn the latent space representation of the abnormal pattern. The encoder compresses the input feature vector into a low-dimensional vector, and the decoder attempts to reconstruct the original feature vector from this low-dimensional vector. By minimizing the reconstruction error, the network can learn the essential characteristics of the abnormal pattern. The learned abnormal patterns are stored to build an abnormal feature knowledge base.
[0098] The Mahalanobis distance is used to measure the similarity between the current system state and the abnormal patterns in the abnormal feature knowledge base. The Mahalanobis distance takes into account the correlation between different features and can measure the similarity more accurately. At the same time, combined with the execution state of the multi-stage execution plan, a Markov state transition matrix is constructed to describe the probability of transition between different abnormal types. For example, excessive database server load may cause the application to respond slowly. The abnormal patterns in the abnormal feature knowledge base are divided into groups using a density-based clustering method, and abnormal patterns with similar features are classified into one category. Combined with the fast nearest neighbor search technology, the abnormal pattern group to which the current system state belongs can be quickly identified, and even mixed abnormalities can be effectively identified.
[0099] Using the graph attention network based on the LeakyReLU activation function, the influence weights between nodes are calculated according to the local structural features of the nodes. The LeakyReLU activation function can avoid the gradient vanishing problem and improve the training efficiency of the network. The dynamic time warping algorithm is used to align the state sequences of different nodes to eliminate the influence of the difference in time series length. Then, the conditional mutual information and transfer entropy between nodes are calculated to quantify the intensity and direction of anomaly propagation. For example, if an anomaly occurs in the database server, its state change may affect the web server that depends on it. By analyzing the temporal correlation of the node state sequence, the path of anomaly propagation can be derived, such as from the database server to the web server and then to the client. Combined with the influence weights between nodes, an anomaly propagation probability graph is constructed to represent the possibility of anomalies propagating between different nodes.
[0100] Based on the anomaly pattern recognition results and anomaly propagation probability graph, the anomaly prediction results are output, including the probability of anomaly occurrence, anomaly type, and impact range. For example, the prediction results may indicate that the database server has an 80% probability of overload anomaly in the next hour, which may affect the performance of the Web server and the client. Formulate preventive processing strategies based on the anomaly prediction results. Construct a multi-objective reward function based on the probability of anomaly occurrence, performance impact, and resource overhead. For example, a good preventive processing strategy should be able to reduce the probability of anomaly occurrence while minimizing the impact on system performance and resource overhead. The processing strategy selector is trained through a deep Q network for decision optimization. The deep Q network can learn the ability to select the best processing strategy in different situations. For example, if it is predicted that the database server is about to be overloaded, the processing strategy selector may choose to increase the resources of the database server or limit the access frequency of the client.
[0101] By constructing an abnormal feature knowledge base and combining it with time series association analysis, the present invention can more accurately identify abnormal patterns and predict the probability, type and impact range of abnormal occurrence; by utilizing multi-dimensional feature similarity calculation and density-based clustering methods, it can effectively distinguish between normal and abnormal states and reduce false alarms; the processing strategy selector based on deep Q network training can automatically select the best preventive processing strategy according to the prediction results, improve processing efficiency and reduce manual intervention.
[0102] In an optional embodiment,
[0103] The steps of constructing a multi-objective reward function based on the probability of anomaly occurrence, performance impact, and resource overhead and optimizing the decision-making of the policy selector through deep Q network training include:
[0104] The multi-objective reward function integrates the abnormality occurrence probability reward, the performance impact reward and the resource expenditure reward in a linear weighted manner, wherein the abnormality occurrence probability reward is calculated based on the abnormality prediction probability, the performance impact reward is calculated based on the ratio of the performance change before and after the strategy execution to the performance benchmark value, and the resource expenditure reward is calculated based on the ratio of the resource consumption required for the strategy execution to the maximum resource consumption, and the weight coefficient of each reward is determined by grid search;
[0105] Construct a state space, which includes a discrete state space and a continuous state space, wherein the discrete state space contains an anomaly type set, an impact range level, and a resource allocation state, and the continuous state space is represented by a vector of normalized values of performance indicators; design an adaptive exploration strategy, adaptively adjust the decay rate based on the cumulative reward of a fixed number of historical steps, and introduce a priority experience replay mechanism, which calculates sample priority and determines sampling probability based on temporal difference error;
[0106] A dual-depth Q network architecture is used to train a processing strategy selector, wherein the dual-depth Q network includes a main network and a target network, both of which adopt a multi-layer fully connected structure, and the action output of the preventive processing strategy is obtained based on the input training of the state space; the processing strategy selector is trained based on the adaptive exploration strategy and the multi-objective reward function, the training samples are stored in an experience replay pool, the training samples are selected according to the sampling probability of the priority experience replay mechanism, and the target network parameters are updated regularly; based on the action sequence output by the processing strategy selector and the corresponding reward value evaluation, the optimal preventive processing strategy is selected and executed.
[0107] Exemplarily, first, a multi-objective reward function is constructed. This function comprehensively considers three aspects: the probability of anomaly occurrence, performance impact, and resource overhead. The reward for the probability of anomaly occurrence is calculated based on the probability of anomaly prediction. The higher the probability, the lower the reward. The performance impact reward is calculated based on the ratio of the performance change before and after the strategy is executed to the performance baseline value. The greater the performance improvement, the higher the reward. The resource overhead reward is calculated based on the ratio of the resource consumption required for strategy execution to the maximum resource consumption. The less the resource consumption, the higher the reward. The three reward items are integrated in a linear weighted manner, and the weight coefficient is determined by grid search. For example, the weight coefficients are taken as 0.2, 0.5, and 0.3 for combination testing, and finally the weight combination with the best performance in the test set is selected.
[0108] Next, construct the state space. The state space consists of two parts: discrete state space and continuous state space. The discrete state space includes a set of anomaly types, such as network delay, disk failure, etc.; impact range levels, such as mild, medium, and severe; and resource allocation states, such as CPU usage, memory usage, etc. The continuous state space is represented by a vector of normalized values of performance indicators. For example, performance indicators such as CPU utilization, memory utilization, and disk IO rate are normalized to between 0 and 1 to form a vector. For example, the current state is: the anomaly type is network delay, the impact range is medium, the CPU utilization is 70%, the memory utilization is 50%, and the disk IO rate is 30%. The discrete state is {network delay, medium, {70%, 50%, 30%}}, and the continuous state is {0.7, 0.5, 0.3}.
[0109] Then, an adaptive exploration strategy is designed. This strategy adaptively adjusts the decay rate of exploration based on the cumulative rewards of a fixed number of historical steps (for example, 100 steps). If the cumulative reward is high, it means that the current strategy is effective, then the decay rate of exploration is reduced to reduce the probability of exploration; otherwise, the decay rate of exploration is increased to increase the probability of exploration. At the same time, a priority experience replay mechanism is introduced. This mechanism calculates the sample priority based on the temporal difference error. The larger the error, the higher the priority and the higher the probability of being sampled. For example, if the temporal difference error of one sample is 0.8 and the temporal difference error of another sample is 0.2, the former is more likely to be sampled.
[0110] Afterwards, a dual-depth Q network architecture is used to train the processing strategy selector. The dual-depth Q network includes a main network and a target network, both of which have the same structure and use a multi-layer fully connected structure. The input is the state space, and the output is the action value corresponding to each preventive processing strategy. For example, the output layer has three nodes, corresponding to the three strategies of "restarting the service", "switching to the backup server", and "limiting traffic" respectively. Based on the input of the state space, the network outputs the Q value corresponding to each action, such as {0.6, 0.8, 0.5}, and selects the action with the largest Q value, namely "switching to the backup server". During the training process, the training samples are stored in the experience replay pool, the training samples are selected according to the sampling probability of the priority experience replay mechanism, and the target network parameters are updated regularly, for example, every 100 steps.
[0111] Finally, based on the action sequence output by the processing strategy selector and the corresponding reward value evaluation, the optimal preventive processing strategy is selected and executed. For example, multiple action sequences are simulated and executed, the cumulative reward of each sequence is calculated, and the strategy corresponding to the sequence with the highest cumulative reward is selected for execution.
[0112] The present invention uses a pre-trained processing strategy selector to quickly select an appropriate processing strategy when an exception occurs, avoiding delays caused by manual intervention, thereby improving decision-making efficiency; it comprehensively considers multiple factors such as the probability of exception occurrence, performance impact, and resource overhead, and can select a more reasonable processing strategy, thereby minimizing the negative impact of the exception and improving system stability; by optimizing resource overhead rewards, it can effectively reduce the resource consumption of the processing strategy and avoid resource waste.
[0113] In an optional embodiment,
[0114] The steps of dynamically adjusting the transaction isolation level and concurrency control parameters of the distributed transaction processor based on the recommendation result of the abnormal feature knowledge base include:
[0115] Based on the abnormal pattern recognition results and historical processing effects in the abnormal feature knowledge base, a quantum genetic algorithm is used to predict transaction conflicts and adjust transaction isolation levels. The quantum genetic algorithm includes a ternary quantum bit encoding scheme to represent the transaction isolation state, reflects the probability distribution of different transaction isolation levels through quantum amplitude, maps transaction features to quantum states through quantum phase encoding, and designs an adaptive quantum rotation gate to adaptively update the ternary quantum bits.
[0116] Based on the transaction conflict prediction results, a hybrid optimization algorithm is used to optimize the concurrency control parameters. The hybrid optimization algorithm represents the parameter space through real number coding, combines the temperature control mechanism of the simulated annealing algorithm and the group search characteristics of the particle swarm algorithm, and uses the speed position update formula of the particle swarm algorithm to search the parameter space, in which the inertia weight of the particles is dynamically adjusted with the simulated annealing temperature; the Metropolis criterion is used to determine the acceptance probability of the particle position update, and the adjustment factor of the Metropolis criterion changes with the number of iterations; the temperature control mechanism is used to balance the global search and the local search;
[0117] Construct a parameter tuning closed loop based on online Bayesian optimization, including a probability regression model based on Gaussian process and a knowledge gradient acquisition function. The hyperparameters of the probability regression model are updated online through maximum likelihood estimation. The knowledge gradient acquisition function is used to optimize the selection of concurrent control parameters.
[0118] The transaction isolation level prediction result of the quantum genetic algorithm is used to guide the parameter search direction of the hybrid optimization algorithm. The optimization result of the hybrid optimization algorithm is used as the prior knowledge of the online Bayesian optimization. The feedback result of the online Bayesian optimization is used to adjust the evolution strategy of the quantum genetic algorithm to form a closed-loop optimization.
[0119] Exemplarily, first, an abnormal feature knowledge base is constructed. The knowledge base stores the feature descriptions of various transaction abnormality patterns, such as deadlock, dirty read, non-repeatable read, etc., as well as the corresponding historical processing effects, including the adopted transaction isolation level, concurrency control parameters, and the final processing results (success or failure, as well as processing time, resource consumption and other indicators). For example, the knowledge base may contain such a record: "In the case of high concurrency updating the same data, deadlock occurs, the serialization isolation level is adopted, and the concurrency control parameters are set to a maximum number of connections of 10. The deadlock is finally successfully resolved, but the processing time is long."
[0120] Next, transaction conflict prediction and transaction isolation level adjustment are performed based on the abnormal feature knowledge base. When a new transaction arrives, the system extracts its features and matches them with the abnormal patterns in the knowledge base. The matching results and historical processing effects are input into the quantum genetic algorithm for transaction conflict prediction. The quantum genetic algorithm uses a ternary quantum bit encoding scheme to represent the transaction isolation state. For example, 0 represents read uncommitted, 1 represents read committed, and 2 represents repeatable read. The probability distribution of different transaction isolation levels is reflected by quantum amplitude. For example, if the probability of read committed is 0.6, the amplitude of the corresponding quantum bit is 0.6. The transaction features are mapped to quantum states through quantum phase encoding, and an adaptive quantum revolving gate is designed to adaptively update the ternary quantum bits. For example, if the transaction features show a high degree of concurrency, the revolving gate will increase the probability of a higher isolation level. Assuming that the features of a transaction have a high degree of match with the "high concurrency update" pattern in the knowledge base, and the historical records show that the repeatable read isolation level works well, the quantum genetic algorithm will output a transaction isolation level recommendation result, for example, the probability of repeatable read is 0.8 and the probability of read committed is 0.2.
[0121] Then, based on the transaction conflict prediction results, a hybrid optimization algorithm is used to optimize the concurrency control parameters. The hybrid optimization algorithm represents the parameter space through real number coding, such as the maximum number of connections, cache size, etc. Combining the temperature control mechanism of the simulated annealing algorithm and the group search characteristics of the particle swarm algorithm, the speed position update formula of the particle swarm algorithm is used to search the parameter space. The inertia weight of the particle is dynamically adjusted with the simulated annealing temperature. The Metropolis criterion is used to determine the acceptance probability of the particle position update, and the adjustment factor of the Metropolis criterion changes with the number of iterations. The temperature control mechanism is used to balance the global search and the local search. For example, assuming that the quantum genetic algorithm recommends the repeatable read isolation level, the hybrid optimization algorithm will search for the optimal concurrency control parameters in this direction, for example, the maximum number of connections is set to 50 and the cache size is set to 10MB.
[0122] A parameter tuning closed loop based on online Bayesian optimization is constructed. The closed loop includes a probabilistic regression model based on Gaussian processes and a knowledge gradient acquisition function. The hyperparameters of the probabilistic regression model are updated online through maximum likelihood estimation. The knowledge gradient acquisition function is used to optimize the selection of concurrent control parameters. The optimization results of the hybrid optimization algorithm are used as prior knowledge for online Bayesian optimization. The feedback results of online Bayesian optimization are used to adjust the evolution strategy of the quantum genetic algorithm to form a closed-loop optimization. For example, if the online Bayesian optimization finds that the current parameter settings cause the transaction processing time to be too long, it will be fed back to the quantum genetic algorithm to adjust its evolution strategy, for example, to increase the probability of a higher isolation level.
[0123] The transaction isolation level prediction results of the quantum genetic algorithm are used to guide the parameter search direction of the hybrid optimization algorithm. The optimization results of the hybrid optimization algorithm are used as prior knowledge for online Bayesian optimization. The feedback results of online Bayesian optimization are used to adjust the evolution strategy of the quantum genetic algorithm to form a closed-loop optimization.
[0124] The present invention can effectively avoid unnecessary conflicts and waiting by dynamically adjusting transaction isolation levels and concurrency control parameters, thereby improving transaction processing efficiency; through prediction and optimization based on an abnormal feature knowledge base, potential abnormal situations can be effectively avoided, thereby enhancing system stability; through the closed-loop feedback mechanism of online Bayesian optimization, the system can continuously adjust parameters according to actual operating conditions to achieve adaptive optimization, thereby better adapting to different loads and environments.
[0125] In an optional embodiment,
[0126] A quantum genetic algorithm is used to predict transaction conflicts and adjust transaction isolation levels. The quantum genetic algorithm includes a ternary quantum bit encoding scheme to represent the transaction isolation state, reflects the probability distribution of different transaction isolation levels through quantum amplitude, maps transaction characteristics to quantum states through quantum phase encoding, and designs an adaptive quantum revolving gate to adaptively update the ternary quantum bits. The steps include:
[0127] A ternary quantum bit encoding scheme is used to map the transaction isolation level to a complex Hilbert space. The ternary quantum bit encoding scheme uses three orthogonal ground states corresponding to read uncommitted, read committed, and serializable, respectively. The quantum amplitude of the ternary quantum bit is used to characterize the superposition state of the transaction isolation state. The quantum amplitude satisfies the normalization condition and is used to reflect the probability distribution of different transaction isolation levels. The historical statistical information of the database transaction is used as an initialization parameter, and the initial distribution of the quantum amplitude is set according to the transaction history data. The square of the modulus of the quantum amplitude represents the probability of the corresponding transaction isolation level occurring during the transaction execution process.
[0128] Mapping the transaction feature vector to the quantum phase space through quantum phase encoding to generate a feature quantum state, mapping the components of the transaction feature vector to quantum phase angles through trigonometric transformation, and constructing a composite state from the feature quantum state and the ternary quantum bit through a tensor product operation, wherein the composite state maintains the correlation between the transaction feature and the transaction isolation state through quantum entanglement;
[0129] An adaptive quantum revolving gate is designed to adaptively update a ternary quantum bit. The quantum revolving gate includes a rotation angle parameter and a phase parameter. The parameters of the quantum revolving gate are iteratively updated by a parameter optimization method based on gradient descent. The parameter optimization method adopts a multi-objective fitness function including conflict probability, performance index and deadlock risk. The quantum revolving gate is applied to the ternary quantum bit to obtain a new transaction isolation state. The probability distribution of the new transaction isolation state reflects the dynamic adjustment result of the transaction isolation level. The quantum state superposition result of the characteristic quantum state and the initial probability distribution determines the iterative optimization direction of the quantum revolving gate parameters through the gradient calculation of the multi-objective fitness function.
[0130] Exemplarily, first, a ternary quantum bit encoding scheme is constructed. Three mutually orthogonal basis states are used to represent three transaction isolation levels: read uncommitted, read committed, and serializable. The isolation state of the transaction is represented by the superposition of these three basis states, and the coefficient before each basis state is the quantum amplitude. The square of the modulus of the quantum amplitude represents the probability of the occurrence of the isolation level. The initial values of these probabilities can be set according to the historical transaction statistics of the database.
[0131] Next, the transaction features are quantum phase encoded. The features of the transaction, such as the number of data items involved in the transaction, the duration of the transaction, etc., are extracted to form a feature vector. Each component of the feature vector is mapped into an angle value through trigonometric transformation as the rotation angle of the quantum phase. For example, assuming that the feature vector is (2, 5), the first component 2 can be mapped to an angle of 2π / 10, and the second component 5 can be mapped to an angle of 5π / 10. These angle values are applied to a series of quantum bits to generate characteristic quantum states. Then, the characteristic quantum state is tensor-producted with the previously constructed ternary quantum bit to construct a composite state. In this way, the transaction features are associated with the transaction isolation state through quantum entanglement.
[0132] Then, an adaptive quantum revolving gate is designed. The quantum revolving gate contains rotation angle parameters and phase parameters, which are used to update the state of the ternary quantum bit. The initial values of these parameters can be set randomly. A multi-objective fitness function is defined, which comprehensively considers the conflict probability, performance indicators (such as throughput), and deadlock risk. For example, different weights can be assigned to the conflict probability, performance indicators, and deadlock risk, and they can be linearly combined into a single fitness function value.
[0133] Next, perform parameter optimization. Use the gradient descent method to iteratively update the parameters of the quantum rotating gate. Specifically, calculate the gradient of the fitness function with respect to the rotation angle parameter and the phase parameter. Apply the quantum rotating gate to the ternary quantum bit to obtain a new transaction isolation state. Substitute the conflict probability, performance index, and deadlock risk under the new isolation state into the fitness function to calculate the new fitness function value. Adjust the rotation angle parameter and phase parameter according to the gradient direction to change the fitness function value in a more optimal direction. Repeat this process until the fitness function value converges or reaches the preset number of iterations. Finally, determine the final transaction isolation level based on the optimized ternary quantum bit state.
[0134] The present invention can effectively reduce the probability of transaction conflicts by adaptively adjusting the transaction isolation level, thereby improving the throughput and concurrency performance of the database. The deadlock risk is taken into account during the optimization process, which can effectively avoid the occurrence of deadlock and improve the stability of the system. Compared with the traditional static isolation level setting, the isolation level can be dynamically adjusted according to the specific characteristics of the transaction, achieving more refined control, thereby better balancing performance and data consistency.
[0135] In an optional embodiment,
[0136] A method for visual management of a database includes: creating a database interaction general interface and a database dialect abstract base class, wherein the database interaction general interface is used to define database operation methods, and the database dialect abstract base class is used as a basic class for different database implementations; matching database operation requests to corresponding database dialect strategy implementation classes according to different database types; obtaining database metadata, including physical table structure, partition information, and index information; converting the database metadata into structured information and visually presenting it on a front-end interface; receiving a user's database operation instruction through the front-end interface, and automatically generating and executing corresponding database operation commands according to the current database type. Create and save connection information of different database types; test the availability of database connections; provide a visual table creation operation interface, support table information selection, field type setting, and field value configuration; generate and display table creation statements corresponding to the current database type. Among them, the database dialect abstract base class includes a general method for inheritance implementation; and a customized implementation method for different database characteristics.
[0137] Exemplary system architecture design: create a database resource table service interface to define a set of methods for front-end database operations; create a database dialect abstract base class as a parent class for different database implementations; and implement support for different databases through the coordination of the interface implementation class and the concrete subclasses of the abstract class.
[0138] Database operation implementation: The interface implementation class automatically matches the corresponding database dialect strategy implementation class according to the current database type; inherits the common methods in the abstract class and implements customized functions according to the specific database characteristics; obtains database metadata information, including table structure, partitions, indexes, etc.; converts complex database metadata into structured information that is easy for users to understand.
[0139] Visual interface implementation: Provide a unified operation interface to display database table structure, index and other information; support operations such as adding, modifying and deleting table structures through the interface; automatically generate operation commands that match the current database; provide database connection management, visual table creation and other functional modules.
[0140] Specific application process: users create and save database connection information; the system tests connection availability; users create tables through a visual interface; the system automatically generates corresponding table creation statements; and supports importing existing table structures and performing visual editing.
[0141] The present invention significantly reduces the learning cost of different databases for users, making it more convenient for users to use databases. By simplifying the operation process and providing an intuitive visual interface, users can easily get started and quickly manage and operate databases, thereby improving work efficiency.
[0142] Figure 2 Schematic diagram of the structure of different types of database visualization table management systems according to embodiments of the present invention. Figure 2 As shown, the system comprises:
[0143] The first unit is used to construct a database universal operating system including a universal operation interface and a dialect abstract base class, wherein the universal operation interface includes a table structure manager, a data queryer and a data updater, and the dialect abstract base class includes an SQL syntax parser converter, a distributed transaction processor and a performance indicator collector; the SQL syntax parser converter performs syntax conversion based on an abstract syntax tree, the distributed transaction processor sets an operation checkpoint at a database operation node and records data status, and the performance indicator collector collects performance data in real time, wherein the performance data includes query response time, resource occupancy rate and concurrent processing number;
[0144] The second unit is used to establish a query performance evaluation system based on the performance data and obtain a query cost evaluation result; based on the query cost evaluation result, predict the optimal database connection method through deep reinforcement learning, and generate a multi-stage execution plan; construct an abnormal feature knowledge base based on a graph neural network, construct an abnormal prediction method based on the abnormal feature knowledge base, identify abnormal patterns through multi-dimensional feature similarity calculation, deduce abnormal propagation paths in combination with time series correlation analysis, formulate preventive processing strategies based on abnormal prediction results, and record the processing effects, and optimize abnormal prediction methods and preventive processing strategies based on the processing effects;
[0145] The third unit is used to send the database operation request to the SQL syntax parsing converter for syntax adaptation and execute the multi-stage execution plan. During the execution process, the distributed transaction processor dynamically adjusts the transaction isolation level and concurrency control parameters based on the recommendation results of the abnormal feature knowledge base, and the performance indicator collector records the execution process data and updates the query performance evaluation system; when the query response is returned, the corresponding data display method is selected for visual presentation based on the terminal device type and the amount of query result data.
[0146] According to a third aspect of the embodiments of the present invention,
[0147] An electronic device is provided, comprising:
[0148] processor;
[0149] a memory for storing processor-executable instructions;
[0150] The processor is configured to call the instructions stored in the memory to execute the aforementioned method.
[0151] According to a fourth aspect of the embodiments of the present invention,
[0152] A computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the aforementioned method is implemented.
[0153] The present invention may be a method, an apparatus, a system and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for executing various aspects of the present invention.
[0154] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.
Claims
1. A method for managing visual tables of different types of databases, characterized in that: include: Constructing a database universal operating system including a universal operation interface and a dialect abstract base class, wherein the universal operation interface includes a table structure manager, a data queryer, and a data updater, and the dialect abstract base class includes an SQL syntax parsing converter, a distributed transaction processor, and a performance indicator collector; The SQL syntax parser converter performs syntax conversion based on the abstract syntax tree, the distributed transaction processor sets an operation checkpoint at the database operation node and records the data status, and the performance indicator collector collects performance data in real time, and the performance data includes query response time, resource occupancy rate and concurrent processing number; Establish a query performance evaluation system based on the performance data and obtain query cost evaluation results; based on the query cost evaluation results, predict the optimal database connection method through deep reinforcement learning and generate a multi-stage execution plan; construct an abnormal feature knowledge base based on a graph neural network, construct an abnormal prediction method based on the abnormal feature knowledge base, identify abnormal patterns through multi-dimensional feature similarity calculation, deduce abnormal propagation paths in combination with time series correlation analysis, formulate preventive processing strategies based on the abnormal prediction results, record the processing effects, and optimize the abnormal prediction methods and preventive processing strategies based on the processing effects; Sending the database operation request to the SQL syntax parsing converter for syntax adaptation, executing the multi-stage execution plan, during which the distributed transaction processor dynamically adjusts the transaction isolation level and concurrency control parameters based on the recommendation results of the abnormal feature knowledge base, and the performance indicator collector records the execution process data and updates the query performance evaluation system; When the query response is returned, the corresponding data display method is selected for visualization based on the terminal device type and the amount of query result data.
2. The method according to claim 1, characterized in that The steps of establishing a query performance evaluation system based on the performance data and obtaining a query cost evaluation result include: A query performance evaluation system is established based on performance data, and the performance data is standardized. The query response time is converted by the quantile normalization method, the resource occupancy rate is processed by the minimum and maximum value normalization method, and the concurrent processing number is normalized by the standard score after logarithmic transformation; the difference coefficient of the performance data is calculated by information entropy, and the dynamic weight coefficient of the performance data is determined based on the difference coefficient; the performance data is subjected to continuous wavelet transformation to obtain time-frequency analysis results, and the performance fluctuation pattern is identified based on the time-frequency analysis results, and the performance fluctuation feature vector is constructed, and the time series change law of the performance data is extracted to obtain the performance time series characteristics; A query cost evaluation function is constructed based on the dynamic weight coefficient, the performance fluctuation feature vector and the performance time series feature, wherein the weight of the performance fluctuation feature vector and the weight of the performance time series feature are determined to have an optimal configuration through a grid search method; a balance factor of the query cost evaluation function is adaptively adjusted based on the feature variance to obtain an initial query cost evaluation result; A cross-validation method is used to evaluate the initial query cost evaluation result, the evaluation accuracy is verified by root mean square error and mean absolute percentage error, and a regularization term is introduced into the query cost evaluation function based on the verification result; the confidence interval of the initial query cost evaluation result is calculated, and the query cost evaluation function is optimized and adjusted according to the calculation result to obtain the final query cost evaluation result.
3. The method according to claim 1, characterized in that Based on the query cost evaluation result, the steps of predicting the optimal database connection method through deep reinforcement learning and generating a multi-stage execution plan include: Construct a deep reinforcement learning framework including an Actor network and a Critic network. The Actor network adopts a multi-layer fully connected network structure. The input layer receives the query cost evaluation result, the table connection relationship diagram and the resource state vector. The hidden layer adopts the ReLU activation function. The output layer outputs the probability distribution of the connection mode through the Softmax function. Construct a global state vector, the global state vector includes a query cost evaluation result vector, a structural feature vector generated based on the initial table connection relationship graph, and a resource state vector; the query cost evaluation result vector includes a standardized evaluation value and upper and lower bounds of a confidence interval used to characterize the relative uncertainty of the query cost evaluation result; the resource state vector includes processor utilization, memory utilization, input and output waiting time, and cache hit rate; the structural feature vector includes the number of connected tables, the selectivity of the connection predicate, the size of the table, and the cardinality of the connection key; design a comprehensive reward function, the comprehensive reward function includes an execution time reward, a resource consumption reward, and an intermediate result set reward, wherein the execution time reward is calculated based on the ratio of the actual execution time to the benchmark execution time, the resource consumption reward is calculated based on the resource utilization, and the intermediate result set reward is calculated based on the ratio of the actual result set size to the estimated value; dynamically adjust the weight of the execution time reward in the comprehensive reward function based on the relative uncertainty, and reduce the weight of the execution time reward when the relative uncertainty increases; The input query statement is parsed into an initial connection graph, and based on the global state vector and the comprehensive reward function, a preliminary multi-stage execution plan is iteratively generated through a deep reinforcement learning framework. In each iterative step, the action probability distribution is obtained through the Actor network based on the current state, and the optimal connection method is selected using an adaptively adjusted exploration strategy. The connection graph and the global state vector are updated according to the selected optimal connection method; during the query execution, the deviation between the actual execution cost and the predicted cost is monitored, and re-optimization is triggered when the deviation exceeds a preset threshold, and the state space of the deep reinforcement learning framework is updated using the actual execution data and a final multi-stage execution plan including a query decomposition strategy, parallelism configuration, and resource allocation scheme is regenerated; the prediction error is calculated based on the actual execution time and resource consumption, and the loss function weights of the Actor network and the Critic network are adjusted according to the prediction error.
4. The method according to claim 1, characterized in that: The steps of constructing an abnormal feature knowledge base based on graph neural network, constructing an abnormal prediction method based on the abnormal feature knowledge base, identifying abnormal patterns through multi-dimensional feature similarity calculation, and deducing abnormal propagation paths in combination with time series correlation analysis, and formulating preventive processing strategies according to abnormal prediction results include: The monitoring indicators are organized into an attribute graph structure, where nodes represent monitoring objects and edges represent dependencies between monitoring objects. The monitoring indicators include the performance data and query cost evaluation results. The time series features, statistical features and topological features of the nodes are extracted to construct a multidimensional feature vector. The time series features include moving average, standard deviation and kurtosis. The statistical features include indicator distribution skewness and interquartile range. The topological features extract the connection mode and interaction relationship of the nodes in the local structure through the graph attention mechanism to form the local structure features of the nodes. The long short-term memory network is used to perform multi-scale temporal encoding on the historical multidimensional feature sequence of the nodes, and the multidimensional feature vectors are fused to obtain the comprehensive feature representation of the nodes. The comprehensive feature representation is input into the encoder-decoder network to learn the latent space representation of the abnormal pattern and construct an abnormal feature knowledge base. The Mahalanobis distance is used to measure the similarity between the current system state and the abnormal pattern in the abnormal feature knowledge base, and the Markov state transition matrix is constructed in combination with the execution state of the multi-stage execution plan to describe the abnormal type transition probability; the abnormal patterns in the abnormal feature knowledge base are divided into groups using a density-based clustering method, and mixed abnormalities are identified by combining fast nearest neighbor search; the graph attention network based on the LeakyReLU activation function uses the local structural features of the nodes to calculate the influence weights between nodes, and uses the dynamic time warping algorithm to align the node state sequence, calculate the conditional mutual information and transfer entropy of the nodes to quantify the abnormal propagation intensity and direction, analyze the temporal correlation of the node state sequence to derive the abnormal propagation path, and construct the abnormal propagation probability graph in combination with the influence weights between nodes; Based on the abnormal pattern recognition results and the abnormal propagation probability map, the abnormal prediction results are output, and the abnormal prediction results include the abnormal occurrence probability, abnormal type and impact range; a preventive processing strategy is formulated according to the abnormal prediction results, and a multi-objective reward function is constructed with the abnormal occurrence probability, performance impact and resource overhead. The decision optimization is performed by training the processing strategy selector through the deep Q network.
5. The method according to claim 4, characterized in that The steps of constructing a multi-objective reward function based on the probability of anomaly occurrence, performance impact, and resource overhead and optimizing the decision-making of the policy selector through deep Q network training include: The multi-objective reward function integrates the abnormality occurrence probability reward, the performance impact reward and the resource expenditure reward in a linear weighted manner, wherein the abnormality occurrence probability reward is calculated based on the abnormality prediction probability, the performance impact reward is calculated based on the ratio of the performance change before and after the strategy execution to the performance benchmark value, and the resource expenditure reward is calculated based on the ratio of the resource consumption required for the strategy execution to the maximum resource consumption, and the weight coefficient of each reward is determined by grid search; Construct a state space, which includes a discrete state space and a continuous state space, wherein the discrete state space contains an anomaly type set, an impact range level, and a resource allocation state, and the continuous state space is represented by a vector of normalized values of performance indicators; design an adaptive exploration strategy, adaptively adjust the decay rate based on the cumulative reward of a fixed number of historical steps, and introduce a priority experience replay mechanism, which calculates sample priority and determines sampling probability based on temporal difference error; A dual-depth Q network architecture is used to train a processing strategy selector, wherein the dual-depth Q network includes a main network and a target network, both of which adopt a multi-layer fully connected structure, and the action output of the preventive processing strategy is obtained based on the input training of the state space; the processing strategy selector is trained based on the adaptive exploration strategy and the multi-objective reward function, the training samples are stored in an experience replay pool, the training samples are selected according to the sampling probability of the priority experience replay mechanism, and the target network parameters are updated regularly; based on the action sequence output by the processing strategy selector and the corresponding reward value evaluation, the optimal preventive processing strategy is selected and executed.
6. The method according to claim 1, characterized in that The steps of dynamically adjusting the transaction isolation level and concurrency control parameters of the distributed transaction processor based on the recommendation result of the abnormal feature knowledge base include: Based on the abnormal pattern recognition results and historical processing effects in the abnormal feature knowledge base, a quantum genetic algorithm is used to predict transaction conflicts and adjust transaction isolation levels. The quantum genetic algorithm includes a ternary quantum bit encoding scheme to represent the transaction isolation state, reflects the probability distribution of different transaction isolation levels through quantum amplitude, maps transaction features to quantum states through quantum phase encoding, and designs an adaptive quantum rotation gate to adaptively update the ternary quantum bits. Based on the transaction conflict prediction results, a hybrid optimization algorithm is used to optimize the concurrency control parameters. The hybrid optimization algorithm represents the parameter space through real number coding, combines the temperature control mechanism of the simulated annealing algorithm and the group search characteristics of the particle swarm algorithm, and uses the speed position update formula of the particle swarm algorithm to search the parameter space, in which the inertia weight of the particles is dynamically adjusted with the simulated annealing temperature; the Metropolis criterion is used to determine the acceptance probability of the particle position update, and the adjustment factor of the Metropolis criterion changes with the number of iterations; the temperature control mechanism is used to balance the global search and the local search; Construct a parameter tuning closed loop based on online Bayesian optimization, including a probability regression model based on Gaussian process and a knowledge gradient acquisition function. The hyperparameters of the probability regression model are updated online through maximum likelihood estimation. The knowledge gradient acquisition function is used to optimize the selection of concurrent control parameters. The transaction isolation level prediction result of the quantum genetic algorithm is used to guide the parameter search direction of the hybrid optimization algorithm. The optimization result of the hybrid optimization algorithm is used as the prior knowledge of the online Bayesian optimization. The feedback result of the online Bayesian optimization is used to adjust the evolution strategy of the quantum genetic algorithm to form a closed-loop optimization.
7. The method according to claim 6, characterized in that A quantum genetic algorithm is used to predict transaction conflicts and adjust transaction isolation levels. The quantum genetic algorithm includes a ternary quantum bit encoding scheme to represent the transaction isolation state, reflects the probability distribution of different transaction isolation levels through quantum amplitude, maps transaction characteristics to quantum states through quantum phase encoding, and designs an adaptive quantum revolving gate to adaptively update the ternary quantum bits. The steps include: A ternary quantum bit encoding scheme is used to map the transaction isolation level to a complex Hilbert space. The ternary quantum bit encoding scheme uses three orthogonal ground states corresponding to read uncommitted, read committed, and serializable, respectively. The quantum amplitude of the ternary quantum bit is used to characterize the superposition state of the transaction isolation state. The quantum amplitude satisfies the normalization condition and is used to reflect the probability distribution of different transaction isolation levels. The historical statistical information of the database transaction is used as an initialization parameter, and the initial distribution of the quantum amplitude is set according to the transaction history data. The square of the modulus of the quantum amplitude represents the probability of the corresponding transaction isolation level occurring during the transaction execution process. Mapping the transaction feature vector to the quantum phase space through quantum phase encoding to generate a feature quantum state, mapping the components of the transaction feature vector to quantum phase angles through trigonometric transformation, and constructing a composite state from the feature quantum state and the ternary quantum bit through a tensor product operation, wherein the composite state maintains the correlation between the transaction feature and the transaction isolation state through quantum entanglement; An adaptive quantum revolving gate is designed to adaptively update a ternary quantum bit. The quantum revolving gate includes a rotation angle parameter and a phase parameter. The parameters of the quantum revolving gate are iteratively updated by a parameter optimization method based on gradient descent. The parameter optimization method adopts a multi-objective fitness function including conflict probability, performance index and deadlock risk. The quantum revolving gate is applied to the ternary quantum bit to obtain a new transaction isolation state. The probability distribution of the new transaction isolation state reflects the dynamic adjustment result of the transaction isolation level. The quantum state superposition result of the characteristic quantum state and the initial probability distribution determines the iterative optimization direction of the quantum revolving gate parameters through the gradient calculation of the multi-objective fitness function.
8. A visual table management system for different types of databases, used to implement the method described in any one of claims 1 to 7, characterized in that: include: The first unit is used to construct a database general operating system including a general operation interface and a dialect abstract base class, wherein the general operation interface includes a table structure manager, a data queryer and a data updater, and the dialect abstract base class includes an SQL syntax parsing converter, a distributed transaction processor and a performance indicator collector; The SQL syntax parser converter performs syntax conversion based on the abstract syntax tree, the distributed transaction processor sets an operation checkpoint at the database operation node and records the data status, and the performance indicator collector collects performance data in real time, and the performance data includes query response time, resource occupancy rate and concurrent processing number; The second unit is used to establish a query performance evaluation system based on the performance data and obtain a query cost evaluation result; based on the query cost evaluation result, predict the optimal database connection method through deep reinforcement learning, and generate a multi-stage execution plan; construct an abnormal feature knowledge base based on a graph neural network, construct an abnormal prediction method based on the abnormal feature knowledge base, identify abnormal patterns through multi-dimensional feature similarity calculation, deduce abnormal propagation paths in combination with time series correlation analysis, formulate preventive processing strategies based on abnormal prediction results, and record the processing effects, and optimize abnormal prediction methods and preventive processing strategies based on the processing effects; The third unit is used to send the database operation request to the SQL syntax parsing converter for syntax adaptation, execute the multi-stage execution plan, and during the execution process, the distributed transaction processor dynamically adjusts the transaction isolation level and concurrency control parameters based on the recommendation results of the abnormal feature knowledge base, and the performance indicator collector records the execution process data and updates the query performance evaluation system; When the query response is returned, the corresponding data display method is selected for visualization based on the terminal device type and the amount of query result data.
9. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the method described in any one of claims 1 to 7.
10. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 7 is implemented.
Citation Information
Patent Citations
Database query optimization method and system based on self-attention and reinforcement learning
CN117827889A
Database anti-ransomware method and device and storage medium
CN118468273A