Learning type cardinal number estimation method and system based on cumulative distribution

Abstract

The invention discloses a learning type cardinal number estimation method and system based on cumulative distribution, and relates to the technical field of database query optimization. According to the method, the stability is ensured while high precision is ensured. The stability ensures the consistency of the generated execution plans, so that the continuous stability of the performance of the commercial database is facilitated. The cumulative distribution function can directly provide the cumulative probability of the random variable in any interval, which is very convenient for evaluating the probability that the variable falls in a specific range. Compared with the prior art, when the probability density function or the probability quality function is used for determining the interval probability, integration or summation needs to be carried out, which is not only more complex, but also may lead to larger errors. Meanwhile, the delay of reasoning acceleration of high-dimensional data is remarkably reduced, the performance is remarkably improved, and the method particularly has important value for large-scale data processing.

Description

Technical Field

[0001] The present invention belongs to the technical field of database query optimization, and particularly relates to a learning-based cardinality estimation method and system based on cumulative distribution. Background Art

[0002] Database cardinality estimation is a crucial part of the query optimizer. Its core task is to predict the size of the result set returned by a query operation, thereby helping the optimizer select an efficient query execution plan. This process directly affects the performance and resource utilization efficiency of the database system. Traditional cardinality estimation methods such as histograms and sampling can quickly provide estimated values by recording the distribution characteristics of the data or sampling a part of the data. However, these methods usually assume that the data is uniformly distributed or the columns are independent of each other, and it is difficult to capture the complex relationships in the real data, especially when facing high-dimensional data or non-linear distributions, they show limitations.

[0003] To overcome these problems, machine learning-based cardinality estimation methods have received extensive attention in recent years and have gradually become the mainstream research direction. Machine learning-based techniques have significantly improved the accuracy of cardinality estimation by modeling the complex distribution of the data or extracting patterns from historical queries. The main research methods can be divided into two categories: query-driven and data-driven. Query-driven methods focus on learning the mapping relationship between query patterns and their corresponding cardinalities, and often use methods such as tree regression models and multi-set convolutional networks. Such techniques can make full use of the existing query history information to generate more accurate cardinality estimates for similar queries. Data-driven methods directly capture the joint probability distribution from the original data and can support a wider range of query types and significantly improve the estimation accuracy of complex queries by modeling the dependencies between columns.

[0004] Traditional statistical-based cardinality estimation methods such as histograms and sampling usually have low accuracy because they make simplistic assumptions about data modeling, such as data uniformity and independence between columns in a table. Query-driven methods, by learning the mapping from a query to its cardinality, such as tree regression models and multi-set convolutional networks, require a large amount of training data. These methods may perform poorly when the workload changes. Therefore, query-driven methods are costly and lack generalization ability. Some studies have tried to make these methods have better generalization ability when the workload changes, but this can only alleviate their defects to a certain extent and cannot completely solve them. Data-driven methods directly capture the joint probability distribution from the data and can be used to estimate the cardinality of a query. Some methods learn the distribution of tuples, such as Naru and NeuroCard, which use deep autoregressive models to approximate the conditional probability distribution between attributes. However, these methods are slow in estimating range queries because they rely on sampling to calculate selectivity, and this problem becomes more significant when the data dimension is high. Summary of the Invention

[0005] Aiming at the deficiencies of the prior art, the present invention proposes a learning-based cardinality estimation method and system based on the cumulative distribution function (CDF). This method can calculate the cardinality of range queries without sampling or integration, ensuring the stability and accuracy of the estimation results. By deeply integrating this theory into the model inference framework, fast inference speed can be obtained, enabling the cardinality estimation method proposed by the present invention to be applicable to actual database application scenarios, so as to solve the problems that traditional cardinality estimation methods cannot capture the correlation between attributes well, and the existing machine learning methods cannot meet the requirements of obtaining accurate, fast, and stable cardinality estimation results simultaneously.

[0006] In the first aspect, the present invention provides a learning-based cardinality estimation method based on the cumulative distribution function, including the following steps:

[0007] Step 1: Convert the database cardinality estimation problem into a probability problem based on the cumulative distribution function, model the database cardinality estimation problem, and obtain a probability model for the cardinality estimation problem;

[0008] First, set the query statement:

[0009] select count(*)from t where l 1 ≤x 1 ≤r 1 and l 2 ≤x 2 ≤r 2 ...and l n ≤x n ≤r n (1)

[0010] where, select count(*) represents the number of tuples that meet the where condition, t represents the table to be queried, x i represents the value of the i-th attribute in the table to be queried, i is the attribute number, n is the number of attributes, l i and r i are respectively the lower and upper bounds of the value of the i-th attribute;

[0011] Then set the query region Ω = [l 1 ,r 1 ×…×[l n ,r n , and the multivariate cumulative distribution function of the random variable X = (X 1 ,X 2 ,…,X i ,…,X n ) is defined as:

[0012] F x (x) = Pr(X 1 ≤ x 1 , X 2 ≤ x 2 ,..., X n ≤ x n )(2)

[0013] Wherein, F X (x) represents the multivariate cumulative distribution function of the random variable X, and X i represents the i-th random variable, and the vector x = (x 1 , x 2 , …, x i , …, x n );

[0014] Set the variable s i ∈ {0, 1}, the vector s = (s 1 , s 2 ,..., s i ..., s n ), |s| = s 1 + s 2 +... + s n , define the variable v i ∈ {l i , r i}, the vector v = (v 1 , …, v i , …, v n ), and:

[0015]

[0016] The probability model of the cardinality estimation problem is:

[0017]

[0018] Wherein, F x (v) is the multivariate cumulative distribution function of the random variable X;

[0019] Step 2: Obtain the tabular data, and then perform inverse discretization on the tabular data to obtain the inverse discretized tabular data;

[0020] The tabular data includes N n-dimensional vectors x, expressed as x (k) represents the k-th vector in the dataset, k is the number of the vector x, and inverse discretization is achieved by adding uniform noise z ∈ [0, b] to the vector x. The inverse discretized vector is expressed as x' = x + z, and then the inverse discretized tabular data D' is obtained, where b represents the minimum difference between any two values of the same attribute in the tabular data;

[0021] Step 3: Standardize the inverse-discretized tabular data to obtain the standardized tabular data; the standardization uses z- sco r e standardization;

[0022] Step 4: Input the standardized tabular data into a deep neural network for training to fit the cumulative distribution function and obtain a continuous multi-variable cumulative distribution approximation function;

[0023] The continuous multi-variable cumulative distribution approximation function is:

[0024]

[0025] where, represents the multi-variable cumulative distribution function, represents the space of absolutely continuous n-dimensional multi-variable cumulative distribution functions, is a coefficient, ji represents the coefficient number, and m represents the parameters of the neural network; represents the tensor function, represents the absolutely continuous univariate cumulative distribution function;

[0026]

[0027] For each univariate cumulative distribution function the following constraint conditions are satisfied:

[0028]

[0029] where, represents the space of absolutely continuous 1-dimensional univariate cumulative distribution functions, represents the univariate cumulative distribution function corresponding density function;

[0030] The deep neural network includes multiple feedforward neural network units, and each feedforward neural network unit fits an univariate cumulative distribution function Each feedforward neural network unit includes u hidden layers, and each hidden layer has q neurons;

[0031] The univariate cumulative distribution function is:

[0032]

[0033] where σ is a non-affine, increasing, and continuously differentiable activation function, sigmoid represents a non-linear function, represents the composition of functions, that is, the nested relationship of functions, is an affine mapping defined as L e (x i ) = W e x i + g e , where e is the number of the neural network layer, and W e is a weight matrix of h e+1 × h e . All elements of the weight matrix are non - negative numbers. g e is a bias vector of h e+1 × 1. h e+1 and h e represent the dimensions of the weight matrix, and h e = q; h u+1 = h 0 = 1;

[0034] Step 5: Encode the query statement to be estimated to obtain a vector encoding, and add a continuity correction factor to the vector encoding for continuity correction to obtain a corrected vector encoding;

[0035] The continuity correction factor is defined as w = (w 1 , …, w n ), where w i is the median of the differences between adjacent values sorted along the i - th attribute;

[0036] Step 6: Standardize the corrected vector encoding according to the method in Step 3 to obtain a standardized vector encoding;

[0037] Step 7: Apply the continuous multivariate cumulative distribution approximation function fitted in Step 4 to the probability model of the cardinality estimation problem obtained in Step 1, input the standardized vector encoding into the probability model of the cardinality estimation problem, and finally obtain the cardinality estimation result.

[0038] In the second aspect, the present invention provides a learning - based cardinality estimation system based on cumulative distribution for implementing a learning - based cardinality estimation method based on cumulative distribution, including a modeling module, a data acquisition and pre - processing module, a fitting module, a query statement acquisition and pre - processing module, and a solving module;

[0039] The modeling module is used to model the database cardinality estimation problem to obtain a cardinality estimation problem probability model;

[0040] The data acquisition and pre - processing module is used to acquire tabular data and pre - process it to obtain pre - processed tabular data; the pre - processing includes inverse discretization and standardization;

[0041] The fitting module is used to input the preprocessed tabular data into a deep neural network for training, fit the cumulative distribution function, and obtain a continuous multivariate cumulative distribution approximation function;

[0042] The query statement acquisition and preprocessing module is used to obtain the query statement to be estimated, encode the query statement to be estimated to obtain a vector encoding, add a continuity correction factor to the vector encoding for continuity correction to obtain a corrected vector encoding; then standardize the corrected vector encoding to obtain a standardized vector encoding;

[0043] The solving module is used to apply the obtained continuous multivariate cumulative distribution approximation function to the probability model of the cardinality estimation problem, input the standardized vector encoding into the probability model of the cardinality estimation problem, and finally obtain the cardinality estimation result.

[0044] In a third aspect, the present invention provides an electronic device, including: a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device runs, the processor communicates with the memory through the bus. When the machine-readable instructions are executed by the processor, the steps of the learning-based cardinality estimation method based on cumulative distribution are executed;

[0045] In a fourth aspect, the present invention provides a computer-readable storage medium. A computer program is stored in the computer-readable storage medium. When the computer program is run by a processor, the steps of the learning-based cardinality estimation method based on cumulative distribution are executed.

[0046] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0047] The present invention proposes a learning-based cardinality estimation method and system based on cumulative distribution (CDF). This method ensures stability while guaranteeing high accuracy. This stability ensures the consistency of the generated execution plan, thus contributing to the continuous stability of the performance of commercial databases. The cumulative distribution function (CDF) can directly provide the cumulative probability of a random variable within any interval, which is very convenient for evaluating the probability that a variable falls within a specific range. In contrast, using the probability density function (PDF) or probability mass function (PMF) to determine the interval probability requires integration or summation, which is not only more complex but also may lead to greater errors. At the same time, the method significantly accelerates the inference of high-dimensional data, reduces latency, brings significant performance improvement, and is particularly valuable for large-scale data processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 It is a flowchart of a learning-based cardinality estimation method based on cumulative distribution (CDF) in an embodiment of the present invention;

[0049] Figure 2 This is the cardinality estimation result graph for running the same query multiple times in the embodiments of the present invention. Detailed implementation manners

[0050] The present invention will be described in detail below with reference to the accompanying drawings and embodiments.

[0051] A learning-based cardinality estimation method based on the cumulative distribution function (CDF), as Figure 1 shown, includes the following steps:

[0052] Step 1: Model the database cardinality estimation problem, convert the database cardinality estimation problem into a probability problem based on the cumulative distribution function, and obtain a cardinality estimation problem probability model;

[0053] First, given the following query (SQL) statement, it is required to estimate the result of the query statement;

[0054] select count(*)from t where l 1 ≤x 1 ≤r 1 and l 2 ≤x 2 ≤r 2 ...and l n ≤x n ≤r n (1)

[0055] where, select count(*) represents the number of tuples that meet the where condition, t represents the table to be queried, x i represents the value of the i-th attribute in the table to be queried, i is the attribute number, n is the number of attributes, l i and r i are respectively the lower bound and the upper bound of the value of the i-th attribute;

[0056] Model the database cardinality estimation problem into a probability problem, that is, sel≈Pr(l 1 ≤x 1 ≤r 1 ,..., l n ≤x n ≤r n ), where, sel represents the selectivity of the query statement, and Pr represents the probability;

[0057] Given the query region Ω = [l 1 , r 1 ×…×[l n , r n ;

[0058] Random variable X = (X1 , X 2 , …, X i , …, X n )'s multivariate cumulative distribution function (CDF) is defined as:

[0059] F X (x) = Pr(X 1 ≤ x 1 , X 2 ≤ x 2 ,...,, X n ≤ x n ) (2)

[0060] Wherein, F X (x) represents the multivariate cumulative distribution function of the random variable X, and X i represents the i-th random variable, and the vector x = (x 1 , x 2 , …, x i , …, x n );

[0061] Set the variable s i ∈ {0, 1}, the vector s = (s 1 , s 2 ,...,, s i ...,, s n ), |s| = s 1 + s 2 +...+ s n , define the variable v i ∈ {l i , r i}, the vector v = (v 1 , …, v i , …, v n ), and:

[0062]

[0063] Then, using the multivariate cumulative distribution function to calculate the probability that the vector x falls within the query region Ω, that is, the probability model of the cardinality estimation problem is:

[0064]

[0065] Wherein, F X (v) is the multivariate cumulative distribution function of the random variable X;

[0066] Step 2: Obtain the tabular data, and then perform dequantization on the tabular data to obtain the dequantized tabular data;

[0067] Modeling discrete data using a continuous distribution can lead to arbitrarily high density values, which form narrow high-density spikes on discrete values. However, a continuous model may not necessarily be able to approximate the probability mass function because the spaces of discrete and continuous variables are topologically different. Under the continuous model, the density value of a data point is zero. To address this issue, an appropriate amount of uniform noise can be added to the data. By adopting this method, the log-likelihood of the continuous model on the anti-discretized data is closely related to the log-likelihood of the discrete model on the discrete data, thereby improving the model performance.

[0068] The tabular data includes N n-dimensional vectors x, denoted as x (k) representing the k-th vector in the dataset, where k is the number of the vector x. Anti-discretization is achieved by adding uniform noise z ∈ [0, b] to the vector x, and the anti-discretized vector is denoted as x' = x + z. Then, the anti-discretized tabular data D' is obtained, where b represents the minimum difference between any two values of the same attribute in the tabular data;

[0069] Step 3: Standardize the anti-discretized tabular data to obtain the standardized tabular data; the standardization uses z-score standardization;

[0070] For data with different dimensions, attributes with larger scales have a greater impact on other attributes. Using z-score standardization to scale the input data to a unified scale addresses this phenomenon, which helps reduce the influence of outliers and improve the accuracy and generalization ability of the model. At the same time, the path of gradient descent after standardization will be smoother, thus accelerating convergence.

[0071] The specific standardization is as follows: for the anti-discretized vector x', the vector after z-score standardization is represented as (x' - μ) / δ, where μ represents the mean of the tabular data and δ represents the standard deviation of the tabular data;

[0072] Step 4: Input the standardized tabular data into a deep neural network for training to fit the cumulative distribution function and obtain a continuous multivariate cumulative distribution approximation function;

[0073] Given a random variable X = (X 1 , X 2 ,..., X n ), to estimate its cumulative distribution function as accurately as possible, consider combining univariate cumulative distribution functions into a multivariate cumulative distribution function, which is represented in the following form:

[0074]

[0075] where, represents the multivariate cumulative distribution function, Denotes the space of absolutely continuous n-variate cumulative distribution functions, is a coefficient, j i Denotes the number of the coefficient, m denotes the parameters of the neural network; Denotes a tensor function, Denotes an absolutely continuous univariate cumulative distribution function;

[0076]

[0077]

[0078] For each univariate cumulative distribution function It needs to satisfy the following constraints:

[0079]

[0080] Among them, Denotes the space of absolutely continuous 1-dimensional univariate cumulative distribution functions, Denotes the univariate cumulative distribution function The corresponding density function;

[0081] The deep neural network includes multiple feedforward neural network units, and each feedforward neural network unit fits a univariate cumulative distribution function The feedforward neural network unit includes u hidden layers, and each hidden layer has q neurons;

[0082] Univariate cumulative distribution function Is:

[0083]

[0084] Among them, σ is a non-affine, increasing and continuously differentiable activation function, sigmoid represents a non-linear function, Denotes the composition of functions, that is, the nested relationship of functions, Is an affine mapping, defined as L e (x i ) = W e x i + g e , e is the number of the neural network layer, W e Is a weight matrix of h e+1 × h e , the elements of the weight matrix are all non-negative numbers, g e Is a bias vector of h e+1 × 1, h e+1 And h e Denotes the dimension of the weight matrix, and h e= q; h u+1 = h 0 = 1;

[0085] The ultimate goal of the training process is to approximate the joint distribution of the data as closely as possible, that is, to minimize the distance between the true data distribution P(x|θ * ) and the data distribution P(x|θ) learned by the model. θ* represents the true data parameters, and this distance can be measured by the KL divergence. According to the law of large numbers, minimizing the KL divergence is equivalent to maximizing the log-likelihood. Among them, the entropy of the true data distribution P(x|θ * ) is independent of the estimated parameter θ. For an n-dimensional tabular data containing N vectors x The negative log-likelihood can be used as the loss function for unsupervised training to approximate the joint data distribution:

[0086]

[0087] Among them, is the loss function, θ is the estimated parameter, and P(x (k) ||θ) represents the probability of the k-th vector x (k) under the estimated parameter θ;

[0088] Among them, the data distribution P(x|θ) can be obtained through the training of a deep neural network:

[0089]

[0090] Among them, F(x|θ) is the multivariate cumulative distribution function fitted under the estimated parameter θ, represents the univariate cumulative distribution function fitted under the estimated parameter θ;

[0091] Step 5: Encode the query statement to be estimated to obtain a vector encoding, and add a continuity correction factor to the vector encoding for continuity correction to obtain the corrected vector encoding;

[0092] For a query region Ω = [l 1 , r 1 ×... × [l n , r n , a vector encoding with a length of n × 2 can be obtained;

[0093] When approximating a discrete distribution with a continuous distribution, adjustments are needed. Define the continuity correction factor as w = (w 1 , …, w n ), where w iis the median of the differences between adjacent values sorted along the i-th attribute. When converting different types of queries into interval queries, continuity correction needs to be performed at appropriate positions. For left-open or right-closed intervals, a correction factor needs to be added at the corresponding left or right endpoint, while other endpoints remain unchanged. After using continuity correction, the probability of a single vector x can be approximated as:

[0094]

[0095] where Pr(X = x) represents the probability that the random variable X equals x, and represents the multivariate cumulative distribution approximation function obtained by fitting the random variable X using a deep neural network;

[0096] Similarly,

[0097]

[0098] where Pr(X ≤ x) represents the probability that the random variable X is less than or equal to x;

[0099] Step 6: Standardize the corrected vector encoding according to the method in Step 3 to obtain the standardized vector encoding;

[0100] Step 7: Apply the continuous multivariate cumulative distribution approximation function obtained by fitting in Step 4 to the probability model of the cardinality estimation problem obtained in Step 1, input the standardized vector encoding into the probability model of the cardinality estimation problem, and finally obtain the cardinality estimation result;

[0101] Given the neural network characteristics of the multivariate cumulative distribution function fitted in the present invention, the inference speed can be fundamentally improved through mathematical derivation, without the need to calculate 2 n joint cumulative probabilities for answering any query, and only one calculation is required, which is specifically expressed as follows:

[0102]

[0103] The complexity of the original inference strategy is O(2 n *m n ), while the optimized complexity is O(m n ). Therefore, for a query encoded as an n-dimensional vector, a speedup ratio of 2 n can be theoretically obtained. The higher the data dimension, the more significant the acceleration effect, which greatly improves the applicability of the model to high-dimensional data.

[0104] Compared with existing methods, the method proposed by the present invention has the highest accuracy and performs excellently on both the BJAQ and POWER datasets, even exceeding Naru and FACE. In the two datasets, its 95% Q-error (1.07 and 1.10 respectively) is very close to 1; traditional methods are very fast in inference due to their simple implementation, but have low accuracy. After inference acceleration, the method proposed by the present invention controls the latency within 1 ms while maintaining high accuracy, while FACE and Naru are more than 10 times slower in comparison.

[0105] Table 1 Cardinality Estimation Result Diagram

[0106]

[0107] Currently, no method can satisfy both high accuracy and high stability. However, the cumulative distribution-based method proposed by the present invention not only ensures the highest accuracy but also maintains stability, as Figure 2 shown. In this regard, the present invention runs the same query multiple times on the POWER dataset for different methods to evaluate the stability of these cardinality estimators. Regression methods (MSCN, lw-nn), DeepDB, KDE, and traditional methods other than sampling maintain stability at low accuracy. Although Naru and FACE maintain relatively high accuracy, they lose stability. The progressive sampling technique of Naru and the Monte Carlo integration process of FACE introduce uncertainty into the inference, thus causing the stability to be damaged.

[0108] This embodiment provides a learning-based cardinality estimation system based on cumulative distribution for implementing a learning-based cardinality estimation method based on cumulative distribution, including a modeling module, a data acquisition and preprocessing module, a fitting module, a query statement acquisition and preprocessing module, and a solving module;

[0109] The modeling module is used to model the database cardinality estimation problem to obtain a cardinality estimation problem probability model;

[0110] The data acquisition and preprocessing module is used to acquire tabular data and preprocess it to obtain preprocessed tabular data; the preprocessing includes inverse discretization and standardization;

[0111] The fitting module is used to input the preprocessed tabular data into a deep neural network for training, fit the cumulative distribution function, and obtain a continuous multivariate cumulative distribution approximation function;

[0112] The query statement acquisition and preprocessing module is used to acquire the query statement to be estimated, encode the query statement to be estimated to obtain a vector encoding, add a continuity correction factor to the vector encoding for continuity correction to obtain a corrected vector encoding; then standardize the corrected vector encoding to obtain a standardized vector encoding;

[0113] The solving module is used to apply the fitted continuous multivariate cumulative distribution approximation function to the probability model of the cardinality estimation problem, input the standardized vector encoding into the probability model of the cardinality estimation problem, and finally obtain the cardinality estimation result.

[0114] This embodiment provides an electronic device, including: a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device runs, the processor communicates with the memory through the bus. When the machine-readable instructions are executed by the processor, the steps of the learning-based cardinality estimation method based on cumulative distribution are executed;

[0115] This embodiment provides a computer-readable storage medium. A computer program is stored in the computer-readable storage medium. When the computer program is run by a processor, the steps of the learning-based cardinality estimation method based on cumulative distribution as described are executed.

Claims

1. A learning-based cardinality estimation method based on cumulative distribution, characterized in that: The following steps are involved: Step 1: Convert the database cardinality estimation problem into a probability problem based on the cumulative distribution function, model the database cardinality estimation problem, and obtain a probability model for the cardinality estimation problem; Step 2: Obtain the table data, and then de-discretize the table data to obtain the de-discretized table data; Step 3: Standardize the de-discretized tabular data to obtain standardized tabular data; Step 4: Input the standardized table data into the deep neural network for training, fit the cumulative distribution function, and obtain the continuous multivariate cumulative distribution approximation function; Step 5: Encode the query statement to be estimated to obtain a vector code, add a continuity correction factor to the vector code to perform continuity correction, and obtain a corrected vector code; Step 6: Standardize the corrected vector code according to the method in step 3 to obtain a standardized vector code; Step 7: Apply the continuous multivariate cumulative distribution approximation function fitted in step 4 to the probability model of the cardinality estimation problem obtained in step 1, input the standardized vector encoding into the probability model of the cardinality estimation problem, and finally obtain the cardinality estimation result.

2. A learning-based cardinality estimation method based on cumulative distribution according to claim 1, characterized in that: The step 1 is specifically as follows: First, set the query: select count(*)from t where l1≤x1≤r1 and l2≤x2≤r2...and l n ≤x n ≤r n (1) Among them, select count(*) indicates the number of tuples that meet the where condition, t indicates the table to be queried, and x i Indicates the value of the i-th attribute in the table to be queried, i is the attribute number, n is the number of attributes, l i and r i are the lower and upper bounds of the value of the i-th attribute respectively; Then set the query area Ω = [l1, r1] × ... × [l n , r n ], random variable X = (X1, X2, ..., X i , …, X n ) is defined as: F X (x)=Pr(X1≤x1,X2≤x2,...,X n ≤x n ) (2) Among them, F X (x) represents the multivariate cumulative distribution function of the random variable X, X i represents the i-th random variable, vector x=(x1,x2,…,x i , …, x n ); Set variable s i ∈{0,1}, vector s=(s1,s2,...,s i ...,s n ), |s|=s1+s2+...+s n , define the variable v i ∈{l i , ri}, vector v = (v1, ..., v i , …, v n ),and: The probability model for the cardinality estimation problem is: Among them, F X (v) is the multivariate cumulative distribution function of the random variable X.

3. A learning-based cardinality estimation method based on cumulative distribution according to claim 2, characterized in that: The table data in step 2 includes N n-dimensional vectors x, represented as x (k) represents the kth vector in the data set, k is the number of vector x, and de-discretization is achieved by adding uniform noise z∈[0,b] to vector x. The de-discretized vector is expressed as x′=x+z, and then the de-discretized tabular data D′ is obtained, where b represents the minimum difference between any two values ​​of the same attribute in the tabular data.

4. The learning-based cardinality estimation method based on cumulative distribution according to claim 1, characterized in that: The standardization described in step 3 adopts z-score standardization.

5. The learning-based cardinality estimation method based on cumulative distribution according to claim 3, characterized in that: The continuous multivariate cumulative distribution approximation function described in step 4 is: in, represents the multivariate cumulative distribution function, represents the space of absolutely continuous n-dimensional multivariate cumulative distribution functions, is the coefficient, ji represents the number of the coefficient, and m represents the parameter of the neural network; Φ(x): represents a tensor function, Represents an absolutely continuous univariate cumulative distribution function; For each univariate cumulative distribution function The following constraints are met: in, represents the space of absolutely continuous 1-dimensional univariate cumulative distribution functions, Represents the univariate cumulative distribution function The corresponding density function; The deep neural network includes multiple feedforward neural network units, each of which fits a univariate cumulative distribution function The feedforward neural network unit includes u hidden layers, each hidden layer has q neurons; Univariate cumulative distribution function for: Among them, σ is a non-affine, increasing and continuously differentiable activation function, sigmoid represents a nonlinear function, It represents the composition of functions, that is, the nested relationship of functions. is an affine mapping, defined as L e (x i )=W e x i +g e , e is the number of the neural network layer, W e It is a h e+1 ×h e The weight matrix, the weight matrix elements are all non-negative numbers, g e It is a h e+1 ×1 bias vector, h e+1 and h e represents the dimension of the weight matrix, and h e =q;h u+1 =h0=1.

6. A learning-based cardinality estimation method based on cumulative distribution according to claim 1, characterized in that: The continuity correction factor in step 5 is defined as w=(w1, ..., w n ), where w i is the median of the differences between adjacent values ​​along the order of the i-th attribute.

7. A learning cardinality estimation system based on cumulative distribution, used to implement a learning cardinality estimation method based on cumulative distribution as claimed in any one of claims 1 to 6, characterized in that: It includes a modeling module, a data acquisition and preprocessing module, a fitting module, a query statement acquisition and preprocessing module and a solution module; The modeling module is used to model the database cardinality estimation problem to obtain a probability model of the cardinality estimation problem; The data acquisition and preprocessing module is used to acquire the table data and preprocess it to obtain the preprocessed table data; The preprocessing includes de-discretization and standardization; The fitting module is used to input the preprocessed table data into the deep neural network for training, fit the cumulative distribution function, and obtain a continuous multivariate cumulative distribution approximation function; The query statement acquisition and preprocessing module is used to acquire the query statement to be estimated, encode the query statement to be estimated to obtain a vector code, add a continuity correction factor to the vector code to perform continuity correction, and obtain a corrected vector code; then standardize the corrected vector code to obtain a standardized vector code; The solution module is used to apply the fitted continuous multivariate cumulative distribution approximation function to the probability model of the cardinality estimation problem, input the standardized vector encoding into the probability model of the cardinality estimation problem, and finally obtain the cardinality estimation result.

8. An electronic device, characterized in that: include: A processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate via the bus, and when the machine-readable instructions are executed by the processor, the steps of a learning cardinality estimation method based on cumulative distribution are performed.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the learning cardinality estimation method based on cumulative distribution as claimed in any one of claims 1 to 6 are executed.

Images