A cardinality estimation optimization method based on attention mechanism and data awareness

By combining the ADAL framework with a dual attention model and a Bayesian neural network, model uncertainty is quantified and active learning is performed, which solves the problems of accuracy and adaptability of cardinality estimation and improves the efficiency and accuracy of database query optimization.

CN118964403BActive Publication Date: 2026-11-17UNIV OF ELECTRONICS SCI & TECH OF CHINA
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202410991235.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-07-23
Publication Date
2026-11-17
Estimated Expiration
2044-07-23

AI Technical Summary

Technical Problem

Existing cardinality estimation methods suffer from insufficient accuracy, limited model adaptability, and high computational efficiency and storage requirements in database query optimization.

Method used

We employ an attention-based and data-aware active learning framework (ADAL), which combines a dual-attention model, a Bayesian neural network, and an active learning strategy with positional attention and cross-attention modules to quantify model uncertainty. Furthermore, we optimize cardinality estimation through diversity sampling and ensemble model techniques.

Benefits of technology

It significantly improves the accuracy and computational efficiency of cardinality estimation, and enhances the adaptability of the model and the overall efficiency of query optimization.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118964403B_ABST
    Figure CN118964403B_ABST
Patent Text Reader

Abstract

This invention belongs to the field of database management system query optimization technology, specifically involving a cardinality estimation optimization method based on attention mechanisms and data awareness. This invention proposes a data-aware dual-attention model, integrating positional attention and cross-attention modules. It then employs a Bayesian neural network-based learning strategy, using Monte Carlo dropout to quantify model uncertainty, effectively improving the model's adaptability and generalization ability to diverse data distributions. Finally, an active learning data sampling strategy is designed to intelligently select the most critical data samples for model training, optimizing the learning process and improving prediction accuracy. Through an ensemble learning strategy, multiple models obtained through different data sampling strategies are integrated to construct a powerful ensemble model, further enhancing the model's generalization ability and stability under various query workloads.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention belongs to the field of database management system query optimization technology, specifically involving a cardinality estimation optimization method based on attention mechanism and data awareness. Background Technology

[0002] Cardinality estimation is a crucial step in database query optimization, directly impacting the efficiency of query plan generation and execution. Traditional cardinality estimation methods primarily rely on statistical information and heuristic algorithms, such as histogram-based methods. These methods typically assume that the data follows a specific statistical distribution and that attributes are independent of each other. However, real-world datasets often exhibit high levels of randomness and complexity, failing to meet these assumptions and leading to insufficient accuracy of traditional methods in certain situations.

[0003] With the development of machine learning technology, cardinality estimation methods based on machine learning have emerged. These methods can be divided into three categories: data-driven, query-driven, and hybrid-driven.

[0004] 1. Data-driven approach: This approach treats cardinality estimation as an unsupervised learning problem, estimating it by modeling the joint distribution of relation tables. For example, DeepDB estimates cardinality by modeling relation and product networks (RSPN).

[0005] 2. Query-driven approach: This approach treats cardinality estimation as a regression problem, mapping the query content to cardinality values. For example, MSCN uses multi-set convolutional networks to process query feature vectors and map them to cardinality estimation.

[0006] 3. Hybrid Approach: Combining data and query information for learning to improve estimation accuracy. For example, UAE uses a unified deep autoregressive model to learn the joint distribution of the table in an unsupervised manner, and uses query content as auxiliary information for training in a supervised manner.

[0007] Despite the progress made in some aspects of machine learning-based cardinality estimation techniques, several challenges remain:

[0008] 1. Mapping between data and query: Existing methods often lack a reasonable mapping mechanism when mapping query features to the actual cardinality, resulting in insufficient information extraction and affecting the accuracy of estimation.

[0009] 2. Model adaptability: Although neural networks have achieved success in many fields, their adaptability in cardinality estimation remains limited, and a single model struggles to capture all types of query features.

[0010] 3. Computational efficiency and storage requirements: Some machine learning-based methods may not be as efficient as traditional query-driven methods in terms of computational efficiency and storage requirements. Summary of the Invention

[0011] This invention aims to address the limitations of existing cardinality estimation methods in database query optimization. By proposing an Active Learning Framework (ADAL) based on attention mechanisms and data awareness to optimize cardinality estimation, it significantly improves the accuracy and computational efficiency of cardinality estimation.

[0012] The technical solution adopted in this invention is:

[0013] A cardinality estimation optimization method based on attention mechanism and data awareness includes the following steps:

[0014] S1. Characterize the database data and query statements to construct training data. Specifically, the database state is characterized. For each attribute A of the database relation R, a histogram is constructed by dividing the value range of A into multiple buckets and calculating the frequency of the value of A in each bucket. Then, the histogram is normalized and combined into a feature vector set X of the database state. Query characterization includes join characterization and filter characterization. Join characterization converts the join conditions in the query into binary vectors, representing the relationship between the two joined attributes. Filter characterization converts the filter conditions in the query into vectors representing the boundary points of the search hyperrectangle. Finally, these feature vectors are combined into the feature vector q of the SQL query.

[0015] S2. Construct a cardinality estimation model based on a dual attention model. The dual attention model refers to a dual attention model that includes a positional attention module and a cross attention module. The feature vector of the database state is input into the positional attention module for processing. The positional attention module captures the correlation between attributes in the database state through a self-attention mechanism. The feature vector of the SQL query and the output of the positional attention module are simultaneously input into the cross attention module. The cross attention module discovers the correlation between the database data and the query through cross attention and obtains the cardinality estimate.

[0016] The initial cardinality estimation model is obtained by offline training of the constructed cardinality estimation model using training data;

[0017] S3. Quantify the uncertainty of the output of the preliminary cardinality estimation model using a Bayesian neural network, and perform data sampling through an active learning strategy, including uncertainty sampling, sampling combining upper confidence bound and uncertainty, and diversity sampling combining the two.

[0018] The formula for uncertainty sampling is:

[0019]

[0020] Where, n B f is the number of times the Bayesian neural network makes predictions for the same sample point q.B (q|W B ) is the prediction of sample q by a Bayesian neural network. It is the average of multiple predictions made by the network for the same sample;

[0021] The formula for combining the upper confidence bound with uncertainty sampling is as follows:

[0022]

[0023] Where, q i It corresponds to f B (q|W B The q-error;

[0024] The diversity sampling is achieved by clustering the data using the k-means clustering algorithm, and then selecting samples with greater uncertainty or upper confidence thresholds from each cluster.

[0025] S4. Incremental training is performed on the cardinality estimation model. Specifically, different models are trained using samples obtained through uncertainty sampling, upper confidence bound and uncertainty combined sampling, and diversity sampling combining the two. Then, multiple models are combined to form an ensemble model.

[0026] S5. Optimize the database cardinality estimate using the obtained ensemble model.

[0027] The beneficial effects of this invention are that it realizes a novel cardinality estimation method, which not only improves the accuracy of cardinality estimation, but also significantly enhances the adaptability of the model and the overall efficiency of query optimization through active learning strategies and ensemble model technology. Attached Figure Description

[0028] Figure 1 This is a query feature example diagram.

[0029] Figure 2 This is a schematic diagram of the data-aware dual attention model (DDAM) architecture.

[0030] Figure 3 This is a schematic diagram of the ADAL framework. Detailed Implementation

[0031] The present invention will now be described in detail with reference to the accompanying drawings:

[0032] This invention includes the following key points:

[0033] 1. Data-Aware Dual Attention Model (DDAM): Innovatively combines the Position Attention Module (PAM) and the Cross Attention Module (CAM) to gain a deeper understanding of the relationship between queries and data.

[0034] 2. Learning strategy based on Bayesian neural network: Introduce Bayesian neural network to quantify the uncertainty of the model and optimize the data sampling process through active learning strategy.

[0035] 3. Ensemble model construction: By integrating models obtained from different sampling strategies, the generalization ability and stability of the model are enhanced.

[0036] 4. End-to-end execution efficiency improvements: Optimize model structure and learning strategies, reduce the computational overhead of the query optimizer, and improve the overall efficiency of query processing.

[0037] The present invention specifically includes the following steps:

[0038] S1, Data and Query Feature Coding

[0039] S1.1 Data Characterization

[0040] In the first step of this invention, the database state is characterized. The database state X is represented by constructing histograms for each attribute, which describe the data distribution of each attribute. Specifically, for each attribute A of the database relation R, a histogram is constructed by dividing the value range of A into multiple buckets and calculating the frequency of the value of A in each bucket. These histograms are normalized and combined to form the feature vector set X of the database state.

[0041] S1.2 Query Characterization

[0042] like Figure 1 The diagram illustrates the query feature generation process, transforming the various components of an SQL query into numerical vectors to prepare for subsequent model input. Query feature generation includes join feature generation and filter feature generation. Join feature generation converts the join conditions in the query into binary vectors representing the relationship between the two joined attributes. Filter feature generation converts the filter conditions in the query into vectors representing the boundary points of the search hyperrectangle. These feature vectors are combined to form the feature vector q of the SQL query.

[0043] S2, Offline Training

[0044] S2.1 Dual Attention Model Training

[0045] During the offline training phase, a dual-attention model is used to learn the features of the data and query to obtain a preliminary cardinality estimation model. For example... Figure 2 As shown, the dual attention model includes a positional attention module (PAM) and a cross attention module (CAM). PAM captures the correlation between attributes in the database state through a self-attention mechanism, while CAM discovers the correlation between database data and queries through cross attention.

[0046] S2.2 Feature Vector Input

[0047] The data feature vector X and the query feature vector q are input into a dual-attention model. The model learns the relationship between these feature vectors and outputs a preliminary cardinality estimate. The goal of this step is to train a model that can understand the complex relationship between the data and the query.

[0048] S3, Application of Active Learning Strategies

[0049] A Bayesian neural network is used to quantify the uncertainty of the model output, and data sampling is performed through an active learning strategy. For example... Figure 3 As shown, the architecture of the ADAL framework is illustrated, including the construction process of data featureization, query featureization, PAM, CAM, Bayesian neural network, sampling strategy, and ensemble model.

[0050] S3.1 Uncertainty Sampling

[0051] An uncertainty-based sampling strategy is employed, selecting samples with high model prediction uncertainty for further training. This strategy quantifies the uncertainty of the model output to select samples most likely to improve model performance. The present invention proposes the following formula for the data sampling strategy:

[0052]

[0053] Among them, f B (q|W B ) is the prediction of sample q by a Bayesian neural network. It is the average of multiple predictions made by the network for the same sample.

[0054] S3.2 Sampling Combining Upper Confidence Bounds and Uncertainty

[0055] By combining the upper confidence bound and uncertainty, samples with high improvement potential are selected. This strategy selects samples that maximize model improvement by considering the uncertainty and bias of the model predictions. The present invention designs the following formula for the data sampling strategy:

[0056]

[0057] Where, q i It corresponds to f B (q|W B The q-error.

[0058] S3.3 Diversity Sampling

[0059] The data is clustered using the k-means clustering algorithm, and then samples with higher uncertainty or upper confidence bounds are selected from each cluster. This diversity sampling ensures the model learns different aspects of the data, improving its generalization ability. This strategy avoids overfitting to a particular class of data by selecting samples with different characteristics.

[0060] S4, Incremental Training and Model Fusion

[0061] S4.1 Incremental Training

[0062] Based on the data sampled using the active learning strategy, the model is incrementally trained. In this step, different data sampling strategies are employed, such as uncertainty-based sampling, sampling combining upper confidence bounds and uncertainty, and diversity sampling, to select the most valuable samples for model training.

[0063] S4.2 Model Fusion

[0064] By employing an ensemble learning strategy, multiple models trained using different sampling strategies are combined to form an integrated model. This ensemble model can improve the model's adaptability to different queries and data distributions, and enhance its generalization ability.

[0065] S5. Optimize the database cardinality estimate using the obtained ensemble model.

[0066] The end-to-end performance evaluation of the method of the present invention includes:

[0067] Accuracy assessment

[0068] The entire query optimization process is evaluated, starting with the accuracy of cardinality estimation. The model's predictive performance is assessed by comparing the cardinality predicted by the model with the actual cardinality.

[0069] Efficiency assessment

[0070] Secondly, evaluate the computational efficiency of the model, including the time spent on model training and testing. By optimizing the model structure and learning strategy, reduce the computational overhead of the query optimizer in actual query processing.

[0071] Experimental verification

[0072] Experiments on multiple real-world database workloads ensured the effectiveness and superiority of the framework in practical applications. Experimental results demonstrate that the ADAL framework exhibits superior performance across various query scenarios.

[0073] In summary, the present invention has the following characteristics:

[0074] First, a data-aware dual attention model (DDAM) is proposed, which integrates the positional attention module (PAM) and the cross attention module (CAM). This model can deeply explore the intrinsic relationship between query features and data features, thereby significantly improving the accuracy of cardinality estimation.

[0075] Secondly, this invention adopts a learning strategy based on Bayesian neural networks, and uses the Monte Carlo dropout method to quantify model uncertainty, which effectively improves the model's adaptability and generalization ability to diverse data distributions.

[0076] Finally, an active learning data sampling strategy was designed, including uncertainty-based sampling, sampling combining upper confidence bounds and uncertainty, and diversity sampling. These strategies can intelligently select the most critical data samples for model training, optimize the learning process, and improve prediction accuracy. Through an ensemble learning strategy, multiple models obtained through different data sampling strategies were combined to construct a powerful ensemble model, further enhancing the model's generalization ability and stability under various query workloads.

[0077] Therefore, this invention proposes an efficient and accurate cardinality estimation method, providing a new technical solution for the field of database query optimization.

Claims

1. A cardinality estimation optimization method based on attention mechanism and data awareness, characterized in that, Includes the following steps: S1. Characterize the database data and query statements to construct training data. Specifically, the database state is characterized. For each attribute A of the database relation R, a histogram is constructed by dividing the value range of A into multiple buckets and calculating the frequency of the value of A in each bucket. Then, the histogram is normalized and combined into a feature vector set X of the database state. Query characterization includes join characterization and filter characterization. Join characterization converts the join conditions in the query into binary vectors, representing the relationship between the two joined attributes. Filter characterization converts the filter conditions in the query into vectors representing the boundary points of the search hyperrectangle. Finally, these feature vectors are combined into the feature vector q of the SQL query. S2. Construct a cardinality estimation model based on a dual attention model. The dual attention model refers to a dual attention model that includes a positional attention module and a cross attention module. The feature vector of the database state is input into the positional attention module for processing. The positional attention module captures the correlation between attributes in the database state through a self-attention mechanism. The feature vector of the SQL query and the output of the positional attention module are simultaneously input into the cross attention module. The cross attention module discovers the correlation between the database data and the query through cross attention and obtains the cardinality estimate. The initial cardinality estimation model is obtained by offline training of the constructed cardinality estimation model using training data; S3. Quantify the uncertainty of the output of the preliminary cardinality estimation model using a Bayesian neural network, and perform data sampling through an active learning strategy, including uncertainty sampling, sampling combining upper confidence bound and uncertainty, and diversity sampling combining the two. The formula for uncertainty sampling is: Where, n B f is the number of times the Bayesian neural network makes predictions for the same sample point q. B (q|W B ) is the prediction of sample q by a Bayesian neural network. It is the average of multiple predictions made by the network for the same sample; The formula for combining the upper confidence bound with uncertainty sampling is as follows: Where, q i It corresponds to f B (q|W B The q-error; The diversity sampling is achieved by clustering the data using the k-means clustering algorithm, and then selecting samples with greater uncertainty or upper confidence thresholds from each cluster. S4. Incremental training is performed on the cardinality estimation model. Specifically, different models are trained using samples obtained through uncertainty sampling, upper confidence bound and uncertainty combined sampling, and diversity sampling combining the two. Then, multiple models are combined to form an ensemble model. S5. Optimize the database cardinality estimate using the obtained ensemble model.

Citation Information

Patent Citations

  • Cardinality cost estimation method based on attention mechanism

    CN114398395A

  • Deep learning-based method and apparatus for skyline query cardinality estimation

    WO2024021624A1