A cumulative distribution based learning cardinality estimation method and system

Abstract

The application discloses a learning-based radix estimation method and system based on a cumulative distribution function, and relates to the technical field of database query optimization. The method ensures stability while guaranteeing high precision. This stability ensures the consistency of the generated execution plan, thereby helping to continuously and stably improve the performance of a commercial database. The cumulative distribution function can directly provide the cumulative probability of a random variable in any interval, which is very convenient for evaluating the probability of the variable falling within a specific range. In contrast, using the probability density function or the probability mass function to determine the interval probability requires integration or summation, which is not only more complex, but also may cause greater errors. At the same time, the reasoning acceleration of the method for high-dimensional data significantly reduces the delay and brings significant performance improvement, which is particularly valuable for large-scale data processing.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of database query optimization, and particularly relates to a learning-based cardinality estimation method and system based on cumulative distribution. BACKGROUND

[0002] Database cardinality estimation is a crucial step in query optimizers, whose core task is to predict the size of the result set returned by a query operation, thereby helping the optimizer choose an efficient query execution plan. This process directly affects the performance and resource utilization efficiency of the database system. Traditional cardinality estimation methods such as histograms and sampling can quickly provide estimation values by recording the distribution characteristics of data or sampling part of the data. However, these methods usually assume that data is uniformly distributed or that columns are independent of each other, making it difficult to capture complex relationships in real data, especially when facing high-dimensional data or non-linear distribution.

[0003] To overcome these problems, machine learning-based cardinality estimation methods have received widespread attention in recent years and have gradually become the mainstream research direction. Machine learning-based techniques significantly improve the accuracy of cardinality estimation by modeling the complex distribution of data or extracting patterns from historical queries. The main research methods can be divided into two categories: query-driven and data-driven. Query-driven methods focus on learning the mapping relationship between query patterns and their corresponding cardinality, often using tree-based regression models, multi-set convolutional networks, and other methods. This type of technology can fully utilize existing query history information to generate more accurate cardinality estimation values for similar queries. Data-driven methods directly capture joint probability distributions from raw data by modeling the dependency between columns, which can support a wider range of query types and significantly improve the estimation accuracy of complex queries.

[0004] Traditional cardinality estimation methods based on statistics such as histograms and sampling usually have low accuracy because they make simplified assumptions about data modeling, such as data uniformity and independence between columns in the table. Query-driven methods learn the mapping from queries to their cardinality, such as tree-based regression models and multi-set convolutional networks, which require a large amount of training data. These methods may perform poorly when the workload changes. Therefore, query-driven methods are costly and lack generalization ability. Some research attempts to make these methods have better generalization ability when the workload changes, but this can only alleviate their defects to a certain extent and cannot be completely solved. Data-driven methods directly capture joint probability distributions from data and can be used to estimate the cardinality of queries. Some methods learn the distribution of tuples, such as Naru and NeuroCard, which use deep autoregressive models to approximate the conditional probability distribution between attributes. However, these methods are slow when estimating range queries because they rely on sampling to calculate selectivity, and this problem becomes more pronounced when the data dimension is high. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention proposes a learning-based cardinality estimation method and system based on cumulative distribution (CDF). This method can calculate the cardinality of range queries without sampling or integration, ensuring the stability and accuracy of the estimation results. By deeply integrating this theory into the model inference framework, a fast inference speed can be achieved, making the cardinality estimation method proposed in this invention applicable to real-world database application scenarios. This solves the problems that traditional cardinality estimation methods cannot capture the correlation between attributes well, and that existing machine learning methods cannot simultaneously obtain cardinality estimation results accurately, quickly, and stably.

[0006] In a first aspect, the present invention provides a learning-based cardinality estimation method based on cumulative distribution, comprising the following steps:

[0007] Step 1: Transform the database cardinality estimation problem into a probability problem based on the cumulative distribution function, model the database cardinality estimation problem, and obtain the probabilistic model of the cardinality estimation problem;

[0008] First, define the query statement:

[0009] select count(*)from t where l1≤x1≤r1 and l2≤x2≤r2...and l n ≤x n ≤r n (1)

[0010] In this context, `select count(*)` represents the number of tuples that meet the WHERE condition, `t` represents the table to be queried, and `x` represents the number of tuples that meet the WHERE condition. i This represents the value of the i-th attribute in the table to be queried, where i is the attribute number, n is the number of attributes, and l i and r i These are the lower and upper bounds of the value of the i-th attribute, respectively;

[0011] Then set the query range Ω = [l1, r1] × ... × [l n r n ], random variable X = (X1, X2, ..., X i , ..., X n The multivariate cumulative distribution function is defined as follows:

[0012] F x (x)=Pr(X1≤x1,X2≤x2,...,X n ≤x n (2)

[0013] Among them, F X(x) represents a multivariate cumulative distribution function of a random variable X, X i represents the i-th random variable, and the vector x = (x1, x2, …, x i , …, x n );

[0014] Set variable s i ∈{0, 1}, vector s = (s1, s2, …, s i ..., s n ), |s| = s1 + s2 + … + s n , define variable v i ∈{l i , r i}, vector v = (v1, …, v i , …, v n ), and:

[0015]

[0016] The probability model of the cardinality estimation problem is:

[0017]

[0018] where F x (v) is a multivariate cumulative distribution function of a random variable X;

[0019] Step 2: Obtain table data, and then perform inverse discretization on the table data to obtain inverse-discretized table data;

[0020] The table data includes N n-dimensional vectors x, denoted as x (k) represents the k-th vector in the data set, k is the number of the vector x, and inverse discretization is achieved by adding uniform noise z ∈ [0, b] to the vector x, and the inverse-discretized vector is denoted as x′ = x+z, and further inverse-discretized table data D′ is obtained, wherein b represents the minimum difference between any two values of the same attribute in the table data;

[0021] Step 3: Standardize the inverse-discretized table data to obtain standardized table data; the standardization adopts z- sco r e standardization;

[0022] Step 4: input the standardized table data into a deep neural network for training to fit a cumulative distribution function, and obtain a continuous multivariate cumulative distribution approximation function;

[0023] The continuous multivariate cumulative distribution approximation function is:

[0024]

[0025] wherein, denotes a multivariate cumulative distribution function, denotes a space of absolutely continuous n-dimensional multivariate cumulative distribution functions, are coefficients, j denotes the index of the coefficients, and m denotes the parameters of the neural network; denotes a tensor function, denotes an absolutely continuous univariate cumulative distribution function;

[0026]

[0027] for each univariate cumulative distribution function satisfies the following constraints:

[0028]

[0029] wherein, denotes a space of absolutely continuous 1-dimensional univariate cumulative distribution functions, denotes a univariate cumulative distribution function corresponding density function;

[0030] the deep neural network comprises a plurality of feedforward neural network units, each feedforward neural network unit fitting a univariate cumulative distribution function the feedforward neural network unit comprises u hidden layers, each hidden layer having q neurons;

[0031] a univariate cumulative distribution function is:

[0032]

[0033] wherein σ is a non-affine, increasing and continuously differentiable activation function, sigmoid denotes a non-linear function, denotes a composition of functions, i.e. a nesting of functions, is an affine mapping defined as L e (x i ) = W e x i + g e , e is the index of the neural network layer, W e is an h e+1 x h e weight matrix, the elements of the weight matrix are non-negative, g e is an h e+1 x 1 bias vector, h e+1 and h e denote the dimensions of the weight matrix, and he = q; h u+1 = h0= 1;

[0034] Step 5: encode the query statement to be estimated to obtain a vector code, add a continuity correction factor to the vector code to correct the continuity, and obtain a corrected vector code;

[0035] The continuity correction factor is defined as w = (w1, …, w n ), wherein w i is the median of the difference between adjacent values after sorting along the i-th attribute;

[0036] Step 6: standardize the corrected vector code according to the method in step 3 to obtain a standardized vector code;

[0037] Step 7: apply the continuous multivariate cumulative distribution approximation function fitted in step 4 to the probability model of the cardinality estimation problem obtained in step 1, input the standardized vector code into the probability model of the cardinality estimation problem, and finally obtain the cardinality estimation result.

[0038] In a second aspect, the present application provides a learning-based cardinality estimation system based on cumulative distribution, which is used to implement a learning-based cardinality estimation method based on cumulative distribution, and includes a modeling module, a data acquisition and preprocessing module, a fitting module, a query statement acquisition and preprocessing module, and a solving module.

[0039] The modeling module is used to model the database cardinality estimation problem to obtain a probability model of the cardinality estimation problem.

[0040] The data acquisition and preprocessing module is used to acquire table data and preprocess it to obtain preprocessed table data; the preprocessing includes anti-discretization and standardization.

[0041] The fitting module is used to input the preprocessed table data into a deep neural network for training, fit a cumulative distribution function, and obtain a continuous multivariate cumulative distribution approximation function.

[0042] The query statement acquisition and preprocessing module is used to acquire a query statement to be estimated, encode the query statement to be estimated to obtain a vector code, add a continuity correction factor to the vector code to correct the continuity, and obtain a corrected vector code; and then standardize the corrected vector code to obtain a standardized vector code.

[0043] The solving module is used to apply the continuous multivariate cumulative distribution approximation function fitted to the probability model of the cardinality estimation problem, input the standardized vector code into the probability model of the cardinality estimation problem, and finally obtain the cardinality estimation result.

[0044] In a third aspect, the present application provides an electronic device, comprising a processor, a memory and a bus, the memory storing machine readable instructions executable by the processor, the processor and the memory communicating through the bus when the electronic device is running, the machine readable instructions being executed by the processor to perform the steps of the cumulative distribution based learning cardinality estimation method as described above.

[0045] In a fourth aspect, the present application provides a computer readable storage medium, the computer readable storage medium storing a computer program, the computer program being executed by a processor to perform the steps of the cumulative distribution based learning cardinality estimation method as described above.

[0046] Compared with the prior art, the present application has the following advantages:

[0047] The present application provides a cumulative distribution based (CDF) learning cardinality estimation method and system, which ensures high precision and stability. This stability ensures the consistency of the generated execution plan, thereby helping to maintain the stability of the performance of commercial databases. The cumulative distribution function (CDF) can directly provide the cumulative probability of a random variable within any interval, which is very convenient for evaluating the probability of a variable falling within a certain range. In contrast, using the probability density function (PDF) or the probability mass function (PMF) to determine the interval probability requires integration or summation, which is not only more complex but also can lead to greater errors. At the same time, the reasoning acceleration of high-dimensional data of this method significantly reduces the delay and brings significant performance improvement, which is particularly valuable for large-scale data processing. BRIEF DESCRIPTION OF DRAWINGS

[0048] Figure 1 A flowchart of a cumulative distribution based (CDF) learning cardinality estimation method in an embodiment of the present application;

[0049] Figure 2 A graph of cardinality estimation results for the same query running multiple times in an embodiment of the present application. DETAILED DESCRIPTION

[0050] The present application will be described in detail below with reference to the accompanying drawings and embodiments.

[0051] A cumulative distribution based (CDF) learning cardinality estimation method, as shown in Figure 1 includes the following steps:

[0052] Step 1: Model the database cardinality estimation problem and convert the database cardinality estimation problem into a cumulative distribution function based probability problem to obtain a cardinality estimation problem probability model;

[0053] First, given the following query (SQL) statement, estimate the result of the query statement;

[0054] select count(*)from t where l1≤x1≤r1 and l2≤x2≤r2...and l n ≤x n ≤r n (1)

[0055] where, select count(*) represents the number of tuples that meet the where condition, t represents the table to be queried, x i represents the value of the i-th attribute in the table to be queried, i is the attribute number, n is the number of attributes, l i and r i are the lower and upper bounds of the i-th attribute, respectively;

[0056] Model the database cardinality estimation problem as a probability problem, i.e., sel≈Pr(l1≤x1≤r1,...,l n ≤x n ≤r n ), where sel represents the selection rate of the query statement, and Pr represents the probability;

[0057] Given the query region Ω=[l1,r1]×…×[l n ,r n ];

[0058] The multivariate cumulative distribution function (CDF) of the random variable X=(X1,X2,...,X i ,...,X n ) is defined as:

[0059] F X (x)=Pr(X1≤x1,X2≤x2,...,X n ≤x n ) (2)

[0060] where F X (x) represents the multivariate cumulative distribution function of the random variable X, X i represents the i-th random variable, and the vector x=(x1,x2,...,x i ,...,x n );

[0061] Let s i ∈{0,1} be a variable, and the vector s=(s1,s2,...,s i ...,s n ), |s|=s1+s2+...+s n , define the variable vi ∈{l i r i}, vector v = (v1, ..., v i , ..., v n ),and:

[0062]

[0063] Therefore, the probability model for calculating the probability that vector x falls within the query region Ω using the multivariate cumulative distribution function, i.e., the cardinality estimation problem, is as follows:

[0064]

[0065] Among them, F X (v) is the multivariate cumulative distribution function of the random variable X;

[0066] Step 2: Obtain the tabular data, and then perform dequantization on the tabular data to obtain the dequantized tabular data;

[0067] Modeling discrete data using continuous distributions can result in arbitrarily high density values, forming narrow, high-density spikes on the discrete data. However, continuous models do not necessarily approximate the probability mass function because the spaces of discrete and continuous variables are topologically different. Under a continuous model, the density of data points is zero. To address this issue, a suitable amount of uniform noise can be added to the data. By employing this method, the log-likelihood of the continuous model on dediscretized data becomes closely correlated with the log-likelihood of the discrete model on discrete data, thereby improving model performance.

[0068] The table data includes N n-dimensional vectors x, represented as follows: x (k) Let x be the k-th vector in the dataset, where k is the number of vector x. Undiscretization is achieved by adding uniform noise z∈[0, b] to vector x. The vector after undiscretization is represented as x′=x+z, and thus the tabular data D′ after undiscretization is obtained, where b represents the minimum difference between any two values ​​of the same attribute in the tabular data.

[0069] Step 3: Standardize the dediscretized tabular data to obtain standardized tabular data; the standardization adopts z-score standardization.

[0070] In data with different dimensions, attributes with larger scales have a greater impact on other attributes. To address this phenomenon, z-score normalization scales the input data to a uniform scale, which helps reduce the influence of outliers and improves the model's accuracy and generalization ability. Furthermore, normalization makes the gradient descent path smoother, thus accelerating convergence.

[0071] The standardization is specifically that, for the inverse-discretized vector x', a z-score standardized vector is represented as (x'-μ) / δ, where μ represents the mean of the table data, and δ represents the standard deviation of the table data;

[0072] Step 4: input the standardized table data into a deep neural network for training, fit the cumulative distribution function, and obtain a continuous multivariate cumulative distribution approximation function;

[0073] Given a random variable X=(X1, X2,..., Xn), to estimate its cumulative distribution function as accurately as possible, consider combining single-variable cumulative distribution functions into a multivariate cumulative distribution function, which is represented as follows: n

[0074]

[0075] wherein, represents a multivariate cumulative distribution function, represents a space of absolutely continuous n-dimensional multivariate cumulative distribution functions, is a coefficient, j i represents the number of coefficients, and m represents the parameters of the neural network; represents a tensor function, represents an absolutely continuous single-variable cumulative distribution function;

[0076]

[0077]

[0078] For each single-variable cumulative distribution function It needs to satisfy the following constraint conditions:

[0079]

[0080] wherein, represents a space of absolutely continuous 1-dimensional single-variable cumulative distribution functions, represents a single-variable cumulative distribution function corresponding density function;

[0081] The deep neural network includes multiple feedforward neural network units, each of which fits a single-variable cumulative distribution function The feedforward neural network unit includes u hidden layers, each of which has q neurons;

[0082] The single-variable cumulative distribution function is: ​

[0083]

[0084] where σ is a non-affine, increasing and continuously differentiable activation function, sigmoid represents a nonlinear function, represents the composition of functions, i.e. the nesting of functions, is an affine mapping defined as L e (x i ) = W e x i + g e , e is the number of neural network layer, W e is an h e+1 x h e weight matrix, the elements of the weight matrix are all non-negative, g e is an h e+1 x 1 bias vector, h e+1 and h e represent the dimensions of the weight matrix, and h e = q; h u+1 = h0= 1;

[0085] The ultimate goal of the training process is to fit the joint distribution of data as much as possible, i.e. to minimize the distance between the real data distribution P(x|θ * ) and the data distribution P(x|θ) learned by the model, θ* represents the real data parameter, and this distance can be measured by the KL divergence. According to the law of large numbers, minimizing the KL divergence is equivalent to maximizing the log-likelihood. Where the entropy of the real data distribution P(x|θ * ) is independent of the estimated parameter θ. For an n-dimensional table data containing N vectors x The negative log-likelihood can be used as the loss function of unsupervised training to approximate the joint data distribution:

[0086]

[0087] where, is the loss function, θ is the estimated parameter, and P(x (k) ||θ) represents the probability of the kth vector x (k) under the estimated parameter θ;

[0088] where the data distribution P(x|θ) can be obtained by deep neural network training:

[0089]

[0090] where F(x|θ) is the multivariate cumulative distribution function fitted under the estimated parameter θ, is the univariate cumulative distribution function fitted under the estimated parameter θ;

[0091] Step 5: Encode the query statement to be estimated to obtain a vector code. Add a continuity correction factor to the vector code to perform continuity correction, and obtain the corrected vector code.

[0092] For a query range Ω = [l1, r1] × ... × [l1, r1] n r n This yields a vector code of length n×2;

[0093] When approximating a discrete distribution with a continuous distribution, adjustments are needed. The continuity correction factor is defined as w = (w1, ..., w...). n ), where w i It is the median of the differences between adjacent values ​​after sorting along the i-th attribute. When converting different types of queries into range queries, continuity correction is needed at appropriate points. For left-open or right-closed ranges, a correction factor needs to be added to the corresponding left or right endpoints, while other endpoints remain unchanged. After using continuity correction, the probability of a single vector x can be approximated as:

[0094]

[0095] Where Pr(X=x) represents the probability that the random variable X equals x. and This indicates that the random variable X is approximated by a multivariate cumulative distribution function fitted using a deep neural network;

[0096] Similarly,

[0097]

[0098] Where Pr(X≤x) represents the probability that the random variable X is less than or equal to x;

[0099] Step 6: Standardize the corrected vector code according to the method in Step 3 to obtain the standardized vector code;

[0100] Step 7: Apply the continuous multivariate cumulative distribution approximation function obtained in Step 4 to the probabilistic model of the cardinality estimation problem obtained in Step 1. Input the standardized vector code into the probabilistic model of the cardinality estimation problem to finally obtain the cardinality estimation result.

[0101] Given the characteristics of the neural network that fits the multivariate cumulative distribution function in this invention, mathematical derivation can fundamentally improve inference speed, eliminating the need to calculate 2 for answering any query. n The joint cumulative probability can be calculated in only one step, as shown below:

[0102]

[0103] The complexity of the original inference strategy is O(2 n *m n ), while the complexity of the optimized one is O(m n ). Therefore, for a query encoded as an n-dimensional vector, the theoretical speedup ratio can be 2 n . The higher the dimension of the data, the more significant the speedup effect, greatly improving the applicability of the model on high-dimensional data.

[0104] Compared with existing methods, the method proposed by the application has the highest accuracy and performs well on both BJAQ and POWER datasets, even exceeding Naru and FACE. In the two datasets, its 95% Q-error (1.07 and 1.10, respectively) is very close to 1; traditional methods are very fast in inference due to their simplicity, but have lower accuracy. After inference acceleration, the method proposed by the application maintains high accuracy while controlling the delay within 1ms, while the speed of FACE and Naru is more than 10 times slower.

[0105] Table 1 Base estimation results

[0106]

[0107] Currently, no method can meet both high accuracy and high stability. However, the method proposed by the application based on cumulative distribution not only guarantees the highest accuracy, but also maintains stability, as shown in Figure 2 . For this purpose, the application runs different methods on the POWER dataset multiple times for the same query to evaluate the stability of these base estimators. Regression methods (MSCN, lw-nn), DeepDB, KDE, and traditional methods except sampling maintain stability at low accuracy. Naru and FACE, although maintaining high accuracy, lose stability. The progressive sampling technique of Naru and the Monte Carlo integration process of FACE introduce uncertainty into inference, resulting in the destruction of stability.

[0108] The embodiment provides a learning-based base estimation system based on cumulative distribution, which is used to implement a learning-based base estimation method based on cumulative distribution, and includes a modeling module, a data acquisition and preprocessing module, a fitting module, a query statement acquisition and preprocessing module, and a solving module.

[0109] The modeling module is used to model the database base estimation problem to obtain a base estimation problem probability model.

[0110] The data acquisition and preprocessing module is used to acquire table data and preprocess the table data to obtain preprocessed table data; the preprocessing includes reverse discretization and standardization.

[0111] The fitting module is configured to input the preprocessed table data into a deep neural network for training, fit a cumulative distribution function, and obtain a continuous multivariate cumulative distribution approximation function.

[0112] The query statement acquisition and preprocessing module is configured to acquire a query statement to be estimated, encode the query statement to be estimated to obtain a vector code, add a continuity correction factor to the vector code to correct the continuity, and obtain a corrected vector code; and then normalize the corrected vector code to obtain a normalized vector code.

[0113] The solving module is configured to apply the fitted continuous multivariate cumulative distribution approximation function to a probability model of a cardinality estimation problem, input the normalized vector code into the probability model of the cardinality estimation problem, and finally obtain a cardinality estimation result.

[0114] The embodiment provides an electronic device, comprising a processor, a memory and a bus, the memory stores machine readable instructions executable by the processor, when the electronic device is running, the processor and the memory communicate through the bus, and the machine readable instructions are executed by the processor to perform the steps of the learning-based cardinality estimation method based on cumulative distribution.

[0115] The embodiment provides a computer readable storage medium, the computer readable storage medium stores a computer program, and the computer program is executed by a processor to perform the steps of the learning-based cardinality estimation method based on cumulative distribution.

Claims

1. A learning-based radix estimation method based on cumulative distribution, characterized by, The method comprises the following steps: Step 1: converting the database cardinality estimation problem into a probability problem based on a cumulative distribution function, modeling the database cardinality estimation problem to obtain a cardinality estimation problem probability model; Step 2: obtaining table data, and then performing inverse discretization on the table data to obtain inverse-discretized table data; Step 3: standardizing the inverse-discretized table data to obtain standardized table data; Step 4: inputting the standardized table data into a deep neural network for training to fit a cumulative distribution function, and obtaining a continuous multivariate cumulative distribution approximation function; Step 5: encoding the query statement to be estimated to obtain a vector code, adding a continuity correction factor to the vector code for continuity correction to obtain a corrected vector code; Step 6: standardizing the corrected vector code according to the method in step 3 to obtain standardized vector code; Step 7: applying the continuous multivariate cumulative distribution approximation function fitted in step 4 to the probability model of the cardinality estimation problem obtained in step 1, inputting the standardized vector code into the probability model of the cardinality estimation problem, and finally obtaining a cardinality estimation result; The step 1 is specifically: First, set a query statement: select count(*)from t wherel1≤x1≤r1 andl2≤x2≤r2…andl n ≤x n ≤r n (1) where condition, t represents the table to be queried, x i represents the value of the i-th attribute in the table to be queried, i is the attribute number, n is the attribute number, l i and r i are the lower and upper bounds of the value of the i-th attribute, respectively; Then set the query area Ω = [l1, r1] x... x [l n ,r n ], the multivariate cumulative distribution function of random variable X = (X1, X2,..., X i ,...,X n ) is defined as: F X (x) = Pr(X1≤x1,X2≤x2,…,X n ≤x n ) (2) where F X (x) denotes the multivariate cumulative distribution function of the random variable X, X i represents the i-th random variable, the vector x = (x1, x2,..., x i ,..., x n ); Setting variable s i ∈ {0,1}, vector s = (s1, s2,..., s i ..., s n ), |s| = s1 + s2 +... + s n , defining variable v i ∈ {1 i , r i}, vector v = (v1,..., v i ,..., v n ), and: The probability model of the cardinality estimation problem is: where F X (v) is the multivariate cumulative distribution function of the random variable X.

2. The learning-based radix estimation method based on cumulative distribution according to claim 1, characterized in that, The table data in step 2 includes N n-dimensional vectors x, denoted as x (k) denotes the kth vector in the data set, k is the number of the vector x, the inverse discretization is realized by adding uniform noise z ∈ [0, b] to the vector x, the vector after inverse discretization is denoted as x' = x + z, and then the table data D after inverse discretization is obtained ′ wherein b represents the minimum difference between any two values of the same attribute in the table data.

3. The learning-based radix estimation method based on cumulative distribution according to claim 1, characterized in that, The standardization in step 3 adopts z-score standardization.

4. The learning-based radix estimation method based on cumulative distribution according to claim 2, characterized in that, The continuous multivariate cumulative distribution approximation function in step 4 is: wherein, denotes a multivariate cumulative distribution function, denotes a space of absolutely continuous n-dimensional multivariate cumulative distribution functions, are coefficients, j i denotes the index of the coefficients, m denotes the parameters of the neural network; denotes a tensor function, denotes an absolutely continuous univariate cumulative distribution function; For each univariate cumulative distribution function satisfies the following constraints: wherein, denotes the space of absolutely continuous 1 -dimensional univariate cumulative distribution functions, denotes a univariate cumulative distribution function the corresponding density function; The deep neural network comprises a plurality of feedforward neural network units, each feedforward neural network unit fitting a univariate cumulative distribution function The feedforward neural network units comprise u hidden layers, each hidden layer having q neurons; Monovariate cumulative distribution function is: where σ is a non-affine, increasing, and continuously differentiable activation function, sigmoid represents a nonlinear function, denotes the composition of functions, i.e., the nesting of functions, is an affine mapping defined as L e (x i ) = W e x i + g e , e is the number of neural network layers, W e is an h e+1 x h e weight matrix, the elements of which are non-negative, g e is an h e+1 x 1 bias vector, h e+1 and h e denote the dimensions of the weight matrix, and h e = q; h u+1 = h0= 1.

5. The learning-based radix estimation method based on cumulative distribution according to claim 1, characterized in that, The continuity correction factor in step 5 is defined as w = (w1,..., w n ), where w i is the median of the differences between consecutive values after sorting along the i-th attribute.

6. A cumulative distribution based learning cardinality estimation system for implementing a cumulative distribution based learning cardinality estimation method according to any one of claims 1-5, characterized by It comprises a modeling module, a data acquisition and preprocessing module, a fitting module, a query statement acquisition and preprocessing module, and a solving module; The modeling module is used for modeling the database cardinality estimation problem to obtain a cardinality estimation problem probability model; The data acquisition and preprocessing module is used for acquiring table data and preprocessing the table data to obtain preprocessed table data; The preprocessing includes inverse discretization and standardization; The fitting module is used for inputting the preprocessed table data into a deep neural network for training to fit a cumulative distribution function, and obtaining a continuous multivariate cumulative distribution approximation function; The query statement acquisition and preprocessing module is used for acquiring a query statement to be estimated, encoding the query statement to be estimated to obtain a vector code, adding a continuity correction factor to the vector code for continuity correction to obtain a corrected vector code, and then standardizing the corrected vector code to obtain standardized vector code; The solving module is used for applying the continuous multivariate cumulative distribution approximation function fitted to the probability model of the cardinality estimation problem, inputting the standardized vector code into the probability model of the cardinality estimation problem, and finally obtaining a cardinality estimation result.

7. An electronic device, comprising: It comprises: A processor, a memory, and a bus, the memory stores machine readable instructions executable by the processor, when the electronic device is running, the processor and the memory communicate through the bus, the machine readable instructions are executed by the processor to execute the steps of the cumulative distribution based learning type cardinality estimation method in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program is run by the processor to execute the steps of the cumulative distribution-based learning-based radix estimation method according to any one of claims 1 to 5.

Images