An online approximate query method based on machine learning model sample generation

Abstract

The present invention discloses an online approximate query method based on machine learning model sample generation. First, the tuples in the data table are marked and assigned labels. Then, the tuples and labels are encoded into vector form. On this basis, the data table tuple vectors are used to train the machine learning model to learn the underlying distribution information of the data, so that data table sample tuples can be generated. Next, the query statement is parsed to obtain the key query information, and the required number of sample tuples of each type is calculated according to the sample allocation algorithm. Finally, according to the sample allocation result, the trained machine learning model is used to generate a corresponding number of sample tuples to form a sample table, and the approximate query result is estimated based on the sample table. The present invention can reduce the data reading overhead in approximate query, improve the efficiency of approximate query while ensuring the accuracy of the approximate query result, and solve the problems of low sampling efficiency and large result error of the existing approximate query method.

Description

Technical Field

[0001] The present invention relates to the fields of database query and machine learning, and in particular to an online approximate query method based on machine learning model sample generation. Background Art

[0002] In recent years, with the rapid growth of data volumes across various fields, big data technology has been widely adopted across numerous industries. Big data query is a typical big data application, capable of analyzing and mining the rich information and value contained in data. It plays a vital role in many big data scenarios, including government services, business decision-making, and personal life. Aggregate query is a core data analysis method in big data query. Because it involves aggregate calculations on massive amounts of data, executing aggregate queries on large-scale data is extremely time-consuming.

[0003] Because traditional precise aggregate queries struggle to meet the real-time requirements of many big data query scenarios, approximate query processing is becoming a hot topic in aggregate query research. Approximate query processing reduces query processing overhead by efficiently computing approximate query results. It is widely used in real-time big data query scenarios that can tolerate a certain degree of query error, such as trend analysis and data visualization.

[0004] The goal of approximate queries is to efficiently compute the most accurate query results possible. Achieving this goal faces three major challenges. The first is selecting high-quality data samples. Because approximate query results are primarily estimated based on samples, sample quality directly impacts query accuracy. The second challenge is reducing data sampling time. The core concept of approximate queries is to introduce additional sampling operations to reduce the amount of data involved in the aggregation calculation. Therefore, sampling overhead must be minimized to fully leverage the acceleration benefits of approximate queries.

[0005] The third area is support for join and aggregate queries. Since join and aggregate queries typically involve two or more tables, sampling data tables requires consideration of joins between the sample tables, complicating approximate queries. However, join and aggregate queries are a common type of aggregate query in data analysis, and supporting them effectively expands the application scenarios of approximate queries.

[0006] Existing approximate query processing still has certain shortcomings in terms of query result error, query response time, and query statement support, making it difficult to meet the actual application needs of approximate queries. First, some approximate query methods do not specifically consider the situation of skewed data distribution, which has a significant impact on the query result value. As a result, these methods have large errors when querying datasets with skewed distributions. Second, mainstream approximate query methods are based on sampling of data tables. Sampling requires traversing the data table, which inevitably introduces large read and write overhead, making the approximate query response time too long. In addition, some approximate query methods do not provide good support for join and aggregation queries, making it difficult to guarantee the accuracy of approximate query results when processing join and aggregation queries. Summary of the Invention

[0007] Purpose of the invention: In view of the problems and shortcomings of the above-mentioned prior art, the purpose of the present invention is to propose an online approximate query method based on machine learning model sample generation to solve the problems of high overhead in sampling original data, large query errors on skewed data, and difficulty in ensuring the accuracy of results in connection queries in existing approximate query methods.

[0008] Technical solution: To achieve the above-mentioned purpose, the technical solution adopted by the present invention is an online approximate query method based on machine learning model sample generation, comprising the following steps:

[0009] (1) Label each tuple x in the data table, select the value of the tuple on some attributes as the label y of the tuple, thereby marking the characteristics of the tuple; at the same time, use the skew data perception optimization method to calculate the statistical information of the data table and separate the outlier tuples in the data table;

[0010] (2) using an input encoder to encode the tuple x and the label of the tuple x into a numerical vector x′, so that the machine learning model can be trained based on the numerical vector x′; dividing the attributes of the tuple x into categorical attributes and continuous attributes, and using different encoding methods to encode the values ​​of the two types of attributes respectively;

[0011] (3) configuring relevant parameters of the machine learning model and using the numerical vector as training data for training the machine learning model, so that the machine learning model learns the underlying data distribution in the data table; after completing the training, the machine learning model is able to generate sample numerical vectors;

[0012] (4) parsing the aggregate query statement to obtain query semantic information, wherein the query semantic information includes data table information, query attribute information, and aggregate function information involved in the aggregate query statement;

[0013] (5) using a sample allocation algorithm to calculate and allocate the number of each type of label samples to be generated based on the query semantic information and the sampling rate; and using the machine learning model to generate a specified number of sample numerical vectors based on the sample quantity allocation result;

[0014] (6) The output decoder decodes the sample numerical vector into sample tuples and forms a sample table; then merges the outlier tuples into the sample table;

[0015] (7) Replace the data table involved in the aggregate query statement so that the aggregate query statement is executed on the sample table; modify the aggregate function part in the aggregate query statement and scale the aggregate function result value; use a parallel multi-sampling optimization method to improve the accuracy of the final approximate query result.

[0016] Furthermore, in step (1), the skewed data-aware optimization method is used to calculate the mean, variance, and quantile statistics of the continuous attributes in the data table, and then the outlier boundary conditions of the continuous attributes are calculated, and the outlier tuples are separated according to the outlier boundary conditions.

[0017] Furthermore, in step (2), the numerical vector x′=(e1, e2,…, e d ), where e j The value range of (j=1, 2, ..., d) is between 0 and 1; the categorical attributes are encoded using a binary encoding method, and the continuous attributes are encoded using a Gaussian mixture encoding method.

[0018] Furthermore, in step (3), the machine learning model is a conditional variational autoencoder, which learns the conditional probability distribution conditioned on the label y; after completing the training, the conditional variational autoencoder performs conditional sample generation based on the input label to generate a sample numerical vector that meets the specified label.

[0019] Furthermore, in step (5), the number of samples to be generated for each type of label sample is calculated using a connection attribute-oriented sample allocation method based on the query semantic information and the sampling rate; a corresponding number of sample labels are generated according to the sample number allocation result, and the sample labels are input into the conditional variational autoencoder that has completed training to generate a specified number of sample numerical vectors.

[0020] Furthermore, in step (6), the output decoder decodes the sample numerical vector according to the coding information recorded in the input coding process, forms a sample table, and adds the outlier tuple to the sample table.

[0021] Furthermore, in step (7), the data table name and aggregation function in the aggregation query statement are modified by using regular expression replacement; multiple sample tables are generated by using parallel multi-sampling optimization to obtain multiple groups of query results, and the average approximate query results are calculated to reduce the error of the final query result.

[0022] Beneficial effects: The present invention can use machine learning models to learn data distribution and generate database tuples, effectively solving the problems of high overhead in sampling original data, large query errors on skewed data, and difficulty in ensuring the accuracy of results on join queries in approximate queries. First, the present invention uses machine learning models to generate samples, avoids accessing and sampling databases, and significantly reduces the sampling overhead in the approximate query process. Second, the present invention uses skewed data perception optimization methods to process skewed data, reducing the error of query results on skewed data. Third, the present invention uses conditional sample generation and sample allocation methods for join queries to generate sample tables that can be effectively joined, thereby improving the query accuracy on join queries. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 is an overall flow chart of the method of the present invention;

[0024] Figure 2 (a) and (b) are schematic diagrams of the data table annotation process in the case of a single tag attribute and multiple tag attributes in the method of the present invention, respectively;

[0025] Figure 3 Schematic diagram of the data table encoding process in the method of the present invention;

[0026] Figure 4 Schematic diagram of the model structure and model training generation process in the method of the present invention;

[0027] Figure 5 This is a schematic diagram of parallel multiple sampling optimization in the method of the present invention;

[0028] Figure 6 (a) and (b) are experimental comparison diagrams of query errors of the method of the present invention and the comparative method when using different sampling rates on the TPC-DS dataset and the Movielen dataset, respectively;

[0029] Figure 7 (a) and (b) are respectively the query time experimental comparison diagrams of the method of the present invention and the comparative method when using different sampling rates on the TPC-DS dataset and the Movielen dataset. DETAILED DESCRIPTION

[0030] The present invention is further illustrated below with reference to the accompanying drawings and specific embodiments. It should be understood that these embodiments are only used to illustrate the present invention and are not used to limit the scope of the present invention. After reading the present invention, modifications of various equivalent forms of the present invention made by those skilled in the art all fall within the scope defined by the claims attached to this application.

[0031] This paper proposes an online approximate query method based on machine learning model sample generation to solve the problems of existing approximate query methods such as high raw data sampling overhead, large query errors on skewed data, and difficulty in ensuring the accuracy of results on connection queries. Figure 1 As shown, the complete process of the present invention includes six parts: labeling stage, encoding stage, learning stage, sampling stage, decoding stage, and execution stage. The specific implementation method is described as follows:

[0032] The marking stage corresponds to step (1) of the technical solution. The specific implementation method is: for each tuple x=(c1, c2, ..., c n ), where n represents the number of attributes in the tuple, c k (k = 1, 2, …, n) represents the values ​​of the tuple on different attributes. The goal of the labeling phase is to assign a corresponding label y to each tuple x in the data table, enabling the model to learn the conditional data distribution conditioned on the label and perform conditional sample generation. The label value y is derived from the values ​​of the tuple on some attributes, which are called label attributes.

[0033] For example, in a data table describing basic personal information, assuming that the label attribute is the "occupation" attribute, the label value of each tuple is the value y=c of the tuple on the "occupation" attribute. 职业 Using labeled data to train the model allows it to learn a conditional data distribution conditioned on the value of the "occupation" attribute. After training, given a value for the "occupation" attribute (such as "doctor"), the trained conditional generative model can generate sample tuples containing the corresponding value. This is called conditional sample generation.

[0034] At the same time, the labeling stage supports multiple label attributes and can combine the values ​​of multiple label attributes to form a label. Generally speaking, label attributes are composed of the connection attributes, grouping attributes and attributes involved in the query predicate in the query statement. For example, for a data table describing basic personal information, assuming that the label attributes include "gender" attribute and "occupation" attribute, the label value of each tuple in the data table is composed of its values ​​on these two label attributes. Specifically, a tuple whose values ​​on the "gender" attribute and "occupation" attribute are "male" and "doctor" respectively has a label value of "male-doctor". The specific process of label labeling is as follows: Figure 2 As shown in (a) and (b).

[0035] In addition, the annotation phase performs skew-data-aware optimization, aiming to minimize the omission of outlier tuples. The primary step in skew-data-aware optimization is identifying and isolating outlier tuples. Outlier tuples are tuples whose values ​​on continuous attributes differ significantly from those of other tuples and have a greater impact on aggregate query results than the remaining tuples. Outlier tuples typically represent a very small percentage of a data table, making them easily missed during sampling, leading to significant query errors.

[0036] Identifying and separating outliers is achieved by using outlier tuple discrimination conditions. When a data table contains outliers, the variance of the continuous attribute of the data table containing outliers will be larger. Therefore, the mean and variance of the continuous attribute of the data table are used to determine whether the data table contains outliers. Assume that table T has a continuous attribute A. i (i=1,2,…,n a ) is μ i , with variance σ i , where n a Express the total number of continuous attributes. If the variance exceeds α times the mean (i.e. σ i >α·μ i , α is set to 10 by default), it means that there is a continuous attribute A in table T i The outlier tuple whose upper value is an outlier.

[0037] When it is determined that there is an outlier tuple in the data table, the continuous attribute A containing the outlier value i η-quantile Q on η (A i ) times as the corresponding outlier boundary condition. In other words, for a tuple x, if it has a continuous attribute A i The value A on i (x) is greater than β·Q η (A i ), then it has continuous attribute A i The value of is an outlier. As long as the value of any continuous attribute of a tuple is an outlier, the tuple is judged to be an outlier. Therefore, considering all the continuous attributes A containing outliers, i The outlier boundary on the tuple can be used to determine the outlier tuple. Where ∨ represents the disjunction symbol, which expresses the logical OR operation. Using the outlier tuple discrimination condition to filter can separate outlier tuples from the data table.

[0038] The encoding stage corresponds to step (2) of the technical solution. The specific implementation method is: convert the data table tuple x=(c1, c2, ..., c n ) and its label y are encoded into a numerical vector form x′=(e1,e2,…,ed A tuple in a data table is composed of values ​​from multiple attributes, and queries are usually based on categorical and numerical attributes. The values ​​of categorical attributes are discrete (for example, the values ​​of the "occupation" attribute include "doctor" and "teacher"), while the values ​​of numerical attributes are continuous (for example, the value of the "salary" attribute is a specific salary value).

[0039] The encoding phase is mainly responsible for encoding the values ​​of each attribute and label of the tuple into a regularized numerical vector representation so that the model can be trained on it. Due to the large differences in the encoding methods of different types of data, different encoding methods are used for categorical attributes and numerical attributes, such as Figure 3 shown.

[0040] For discrete data, categorical attributes are encoded using binary encoding. Binary encoding first maps the discrete values ​​of the categorical attribute to integers, and then encodes the corresponding integers into binary form. For example, the value "doctor" in the "occupation" attribute is mapped to the integer "3" and then represented as [1,1,0,…,0]. The mapping between the specific values ​​of the categorical attribute and the integers is called the category mapping, which is recorded and saved for subsequent decoding.

[0041] For continuous data, a Gaussian mixture encoder is used for encoding. The Gaussian mixture encoder first uses a Gaussian mixture model to fit the data distribution on each numerical attribute. A Gaussian mixture model consists of multiple Gaussian distributions, each of which is represented by its distribution parameters (mean μ and variance σ). 2 )Decide.

[0042] During the fitting process, each value v on the numerical attribute will be assigned to a Gaussian distribution, so that the normalized value v on the distribution can be calculated based on the Gaussian distribution parameters. norm =(v - μ) / σ. A value on a numerical attribute corresponds to only one Gaussian distribution, while a Gaussian distribution may correspond to multiple values. Finally, the encoding of each value on a numerical attribute consists of the number of the Gaussian distribution to which it belongs and the normalization value on that Gaussian distribution. The Gaussian distribution number is encoded using a one-hot encoding method. Finally, the Gaussian mixture encoder converts each value of the numerical attribute into the one-hot encoding of the Gaussian distribution number and the normalization value.

[0043] The learning phase corresponds to step (3) of the technical solution. The specific implementation method is: use the Conditional Variational Autoencoder (CVAE) model to train on the encoded database tuples and their labels to learn the conditional data distribution and form a Table-CVAE model.

[0044] Table-CVAE defines a latent space z and learns the transformation from the latent space distribution to the original data distribution through model training, so that samples in the latent space can be converted into original data samples. The neural network model in Table-CVAE consists of two parts: the encoder network (Encoder) and the decoder network (Decoder). Table-CVAE uses Encoder and Decoder to implement the distribution mapping between the data table and the latent space. The Encoder is responsible for modeling the distribution probability function P(z|X,y), mapping the tuple x and its label y to the latent space. Symmetrically, the Decoder is responsible for learning the distribution probability function P(X|z,y), mapping the latent variable z and label y back to the tuple x.

[0045] The model training and generation process of Table-CVAE is as follows Figure 4 As shown in the figure, during training, the encoder maps the encoded tuple data into latent variables, while the decoder reconstructs the latent variables into the encoded tuple data. The model then calculates the loss between the reconstructed data and the original data and adjusts the neural network parameters in the encoder and decoder based on the loss, allowing the model to continuously optimize the parameters and learn the distribution mapping. Once model training is complete, Table-CVAE can use the learned data conditional probability distribution for conditional sample generation.

[0046] The sampling phase corresponds to steps (4) and (5) of the technical solution. The specific implementation method is: using a query parser to extract query key information from the query statement. Then, a connection attribute-oriented sample allocation algorithm performs reasonable sample allocation at a given sampling rate based on the query parsing results, and determines the number of sample tuples to be generated for each type of label to improve the accuracy of the query results.

[0047] Consider a join aggregation query q where a table T containing n tuples is joined based on the join attribute A. J Join with another table T', and the query goal is to obtain the join result according to attribute A J After grouping, in attribute A a Define the connection attribute A J The value range is attribute D, then for each value v∈D, G v Indicates that the data table attribute T is in the connection attribute AJ The set of tuples whose value is v, and whose group mean is μ v , the group variance is

[0048] Given the total sample size M, the goal of the sample allocation algorithm is to determine its group G for each connection attribute value v∈D v The corresponding sample size m v , and minimize the error of the approximate query results as much as possible. At the same time, the sample allocation needs to satisfy the constraint that the total sample M = ∑v ∈D m v , that is, the sum of the sample sizes allocated to each group is equal to the total sample size.

[0049] Treat sample allocation as an optimization problem, and minimize the expected error metric by reasonably allocating the number of samples. The smaller the expected error metric corresponding to the approximate query result, the smaller its error. For each group G v , assuming that the approximate mean based on sample estimation is Its grouping error err v It is calculated according to the following formula:

[0050]

[0051] Obviously, for any connection attribute value v∈D, the error metric err v If both are in the range [0,1], then the expectation of the error measure is for:

[0052]

[0053] In the above formula, δ is the integral variable in the range [0,1], Pr[err v >δ] represents the error err v The probability of being greater than δ, is the mathematical expectation symbol, ∫ is the integral symbol, and ln is the natural logarithm function. v Contains the inverse exponential function, directly minimizing err v Therefore, the error metric err is established. v The expected upper bound of , and then minimize its upper bound as the optimization target. Assume that the mean and variance of the data distribution learned by the model are the same as the original data distribution, which are μ v and Therefore, the sample mean The mean of the Gaussian distribution is equal to μ v , the variance is equal to and μ v Expected squared difference Right now Therefore, according to Markov's Inequality, for any δ∈[0,1], we have the following inequality:

[0054]

[0055] In the above formula, Pr represents the probability symbol, which indicates the probability of the event in the square brackets. For each value v∈D, combining the first two formulas, we can get the expected error The upper bound of :

[0056]

[0057] Then, for the overall query result, the query overall error metric err(q) is the error err of each group result v The weighted average of:

[0058]

[0059] Therefore, the expected upper bound of the query overall error metric is for:

[0060]

[0061] where w i Indicates group G v The weight can be defined by the user, and δ represents an integral variable in the range [0,1]. The goal of the sample allocation algorithm is to determine the number of samples m for each group. v The value of , so that the upper bound of the overall query error expectation Minimize. In the above formula is a constant term, so the minimization objective is Since there is an equality constraint M = ∑v ∈D m v , using the Lagrange multiplier method to solve, we can get the number of samples in each group m v The value of:

[0062]

[0063] After determining the number of samples to be generated, latent variable samples are sampled from the latent space and input into the Table-CVAE model to be converted into tuple sample vectors. Assuming that the dimension of the latent space is n, the data points in the latent space correspond to an n-dimensional vector z, also called a latent variable. The values ​​of the latent variable z in each dimension come from n mixed Gaussian distributions {N(μ1,σ1),N(μ2,σ2),…,N(μ n ,σ n )}, denoted as N(μ,σ), where the distribution parameters μ={μ1,μ2,…,μn},σ={σ1,σ2,…,σ n The distribution parameters of the Gaussian distribution are mainly learned during the learning phase and contain the underlying distribution information of the data table.

[0064] Assume that the latent variable samples obtained from the latent space are {z1,z2,…,z m}. Each latent variable sample z r (r=1,2,…,m) is assigned a label y according to the sample distribution result r , and the labeled latent variable samples (z r ,y r ) After transformation, we can get the label y r The data input to the model is a set of labeled latent variable samples {(z1,c1),(z2,c2),…,(z m ,c m )}, and the Decoder in the model converts the latent variable samples into a set of sample tuple vectors {x′1,x′2,…,x′ m}.

[0065] The decoding phase corresponds to step (6) of the technical solution. The specific implementation is: convert the sample tuple vector generated by the decoder into a tuple in the data table. The decoding process is the inverse of the encoding process. Categorical attribute data and numerical attribute data are encoded separately, so the two types of data are decoded separately.

[0066] For categorical attribute data, the decoding process first converts the binary-encoded data into integers. Then, based on the category mapping recorded during the encoding process, the integers are converted back to the original categorical attribute values. For example, suppose a sample tuple vector has a binary encoding vector for the "occupation" attribute of [1, 1, 0, ..., 0]. First, the encoding vector is converted to the integer "3." Then, based on the category mapping, the integer "3" represents the value "doctor," and this value is used to replace the encoding vector in the sample tuple.

[0067] For numerical attribute data, its Gaussian mixture encoding representation consists of two parts: the one-hot encoding of the Gaussian distribution number and the regularization value. First, the one-hot encoding of the Gaussian distribution number is inversely transformed to obtain the specific Gaussian distribution number. Then, the mean μ and variance σ of the Gaussian distribution can be obtained based on the number. 2 Finally, after inverse regularization, the original numerical attribute value can be obtained. Assume that the regularization value is v norm , then the original numerical attribute value can be obtained by reverse regularization v = c norm ·σ+μ.

[0068] After decoding the data in a sample tuple vector, multiple attribute values ​​corresponding to it can be obtained, and by concatenating all the values, a specific sample tuple can be obtained. After the decoding phase is completed, all sample tuples are aggregated into a sample table, which is used to estimate the subsequent approximate query results. In addition, skewed data-aware optimization merges the outlier tuples separated from the original data in step (1) into the sample table to avoid the loss of outlier tuples.

[0069] The execution phase corresponds to step (7) of the technical solution. The specific implementation method is: rewrite the aggregate query and execute it on the generated sample table to obtain approximate query results. During the rewriting process, the original data table in the original aggregate query statement is replaced with the corresponding sample table, so that the query statement is executed on the sample table instead of the original data table.

[0070] At the same time, the aggregation function in the original aggregation query statement will be rewritten to include the scaling factor, thereby achieving the scaling of the aggregation function results on the sample table. i SUM(A i ) will be modified to SUM(A i / ρ). Due to the difference in sampling rate between outlier tuples and non-outlier tuples, attribute A in the query statement i Upper mean aggregation function AVG(A i ) will be rewritten as SUM(A i / ρ) / SUM(1 / ρ).

[0071] In addition, since the overhead of conditional sample generation using the model is relatively low, parallel multi-sampling aggregation optimization can perform multiple sample generation in parallel in a relatively short time to obtain multiple groups of approximate query results, and then combine the multiple groups of query results, and finally combine the average approximate query results to obtain the final approximate query result with smaller error, such as Figure 5 shown.

[0072] The present invention proposes an online approximate query method based on a machine learning model, and implements an online approximate query prototype system based on existing open source software, providing a good programming interface and SQL command line interaction.

[0073] To test the performance of the approximate query method of the present invention, complex join aggregation queries were conducted on the TPC-DS and Movielen datasets in the same Spark distributed data cluster environment. The experiment primarily evaluated the performance of the approximate query method, with the metrics involved including query error and query time. The lower the query error and the shorter the query time, the better the performance of the approximate query method. For comparison, the performance of the approximate query methods Uniform and Senate based on traditional sampling, the approximate query methods Wander Join and Universe Sampler for join attribute sampling, and the approximate query methods DeepGen and DBEst++ based on machine learning models were also evaluated.

[0074] Figure 6 (a) and (b) show the query errors of the proposed method and the comparative method on the TPC-DS and Movielen datasets, respectively, using different sampling rates. Compared to the comparative method, the average error of the approximate query results of the proposed method is reduced by 63.7% and 62.6%, respectively. Figure 7 (a) and (b) show the query times of the proposed method and the comparative method on the TPC-DS and Movielen datasets, respectively, using different sampling rates. The dashed lines correspond to the exact query times. The proposed approximate query times achieve an average speedup of 1.78x and 3.82x, respectively, compared to existing sampling-based approximate query methods, and 3.88x and 8.15x, respectively, compared to exact queries.

Claims

1. An online approximate query method based on machine learning model sample generation, comprising the following steps: (1) Label each tuple x in the data table, select the value of the tuple on some attributes as the label y of the tuple, thereby marking the characteristics of the tuple; at the same time, use the skew data perception optimization method to calculate the statistical information of the data table and separate the outlier tuples in the data table; (2) using an input encoder to encode the tuple x and the label of the tuple x into a numerical vector x′, so that the machine learning model can be trained based on the numerical vector x′; dividing the attributes of the tuple x into categorical attributes and continuous attributes, and using different encoding methods to encode the values ​​of the two types of attributes respectively; (3) configuring relevant parameters of the machine learning model and using the numerical vector as training data for training the machine learning model, so that the machine learning model learns the underlying data distribution in the data table; after completing the training, the machine learning model is able to generate sample numerical vectors; (4) parsing the aggregate query statement to obtain query semantic information, wherein the query semantic information includes data table information, query attribute information, and aggregate function information involved in the aggregate query statement; (5) using a sample allocation algorithm to calculate and allocate the number of each type of label samples to be generated based on the query semantic information and the sampling rate; and using the machine learning model to generate a specified number of sample numerical vectors based on the sample quantity allocation result; (6) The output decoder decodes the sample numerical vector into sample tuples and forms a sample table; then merges the outlier tuples into the sample table; (7) replacing the data table involved in the aggregate query statement so that the aggregate query statement is executed on the sample table; Modify the aggregation function part in the aggregation query statement and scale the aggregation function result value; use a parallel multi-sampling optimization method to improve the accuracy of the final approximate query result; In step (3), training is performed on the encoded database tuples and their labels to learn the conditional data distribution and form a Table-CVAE model.

2. The online approximate query method based on machine learning model sample generation according to claim 1, characterized in that: In the step (1), the skewed data-aware optimization method is used to calculate the mean, variance, and quantile statistics of the continuous attributes in the data table, and then the outlier boundary conditions of the continuous attributes are calculated, and the outlier tuples are separated according to the outlier boundary conditions.

3. The online approximate query method based on machine learning model sample generation according to claim 1, characterized in that: In step (2), the numerical vector x ′ =(e1,e2,…,e d ), where e j The value range of (j=1, 2, ..., d) is between 0 and 1; the categorical attributes are encoded using a binary encoding method, and the continuous attributes are encoded using a Gaussian mixture encoding method.

4. The online approximate query method based on machine learning model sample generation according to claim 1, characterized in that: In step (3), the machine learning model is a conditional variational autoencoder, which learns the conditional probability distribution conditioned on the label y; after completing the training, the conditional variational autoencoder performs conditional sample generation based on the input label to generate a sample numerical vector that meets the specified label.

5. The online approximate query method based on machine learning model sample generation according to claim 1, characterized in that: In the step (5), the number of samples that should be generated for each type of label sample is calculated using a connection attribute-oriented sample allocation method based on the query semantic information and the sampling rate; a corresponding number of sample labels are generated according to the sample number allocation result, and the sample labels are input into the trained conditional variational autoencoder to generate a specified number of sample numerical vectors.

6. The online approximate query method based on machine learning model sample generation according to claim 1, characterized in that: In the step (6), the output decoder decodes the sample numerical vector according to the coding information recorded in the input coding process, forms a sample table, and adds the outlier tuple to the sample table.

7. The online approximate query method based on machine learning model sample generation according to claim 1, characterized in that: In the step (7), the data table name and the aggregation function in the aggregation query statement are modified by using a regular expression replacement method; Use parallel multi-sampling optimization to generate multiple sample tables to obtain multiple sets of query results, calculate the average approximate query results, and reduce the error of the final query results.

Images