A product-sum network and residual neural network hybrid system for approximate query processing
By using a hybrid system of sum-product networks and residual neural networks, the problem of excessive latency in interactive queries on large-scale datasets is solved, enabling fast and accurate approximate query processing, thereby improving user experience and decision effectiveness.
Patent Information
- Application Number
- CN202211379710.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-11-04
- Publication Date
- 2025-11-07
- Estimated Expiration
- 2042-11-04
AI Technical Summary
Existing technologies suffer from excessive latency when performing interactive exact queries on large-scale datasets. Sampling-based approximate query methods require more samples as the data volume increases, leading to increased query latency and storage space consumption, and are unable to cope with the ever-growing data scale requirements.
A hybrid system of sum-product networks and residual neural networks for approximate query processing is adopted. A pre-trained machine learning model is used to replace data samples for approximate queries. It includes an unsupervised learning module based on sum-product networks and a supervised learning module based on residual neural networks. The query results are corrected by constructing multiple density estimators and deep neural networks for the dataset.
It achieves fast and accurate approximate query processing on large-scale datasets, with query response time down to the sub-second level, low error level, reduced storage space requirements, and good scalability.
Smart Images
Figure CN115794880B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The application belongs to the technical field of data analysis, and particularly relates to a query optimization processing system in data analysis. BACKGROUND
[0002] When a data analyst performs interactive exploratory analysis on a large-scale dataset, the time delay of executing an exact query in a database is long. For example, it takes about 2 minutes and 32 seconds to execute a simple COUNT query on a flight delay data table containing 500 million rows of data records (about 43 GB), which is unacceptable for users in an interactive scenario. The long query waiting time exceeds the patience of users, seriously affecting the effectiveness of analysis and behavioral decision-making.
[0003] In order to reduce such query delay, a series of approximate query processing methods based on sampling are proposed. BlinkDB proposes a method of constructing hierarchical samples from original data, which dynamically selects a sample construction strategy based on the accuracy of the query and the response time requirement; Jinglin Peng et al. propose AQP++, which combines the sampling method with data cube pre-computation technology to improve the accuracy of approximate queries; Microsoft proposed Sample+Seek in 2016 to build auxiliary indexes for data with low selectivity to improve accuracy based on sampling. However, these approximate methods based on sampling do not have strong scalability, that is, when the data volume increases, more data samples are needed to ensure the accuracy of the sampling method, thereby causing longer query delay and larger storage space occupation, which cannot meet the growing demand for data size. Therefore, how to provide a faster, lighter and more accurate approximate query processing method for large data for users is an important problem to be solved. SUMMARY
[0004] In order to solve the above problems, the purpose of the present application is to provide an approximate query processing-oriented and product network and residual neural network hybrid system for large-scale datasets, which has faster query speed, higher accuracy and smaller storage space occupation, uses a trained machine learning model to replace data samples to execute approximate queries.
[0005] The approximate query processing-oriented and product network and residual neural network hybrid system provided by the present application mainly includes two modules: an unsupervised learning module based on a product network and a supervised learning module based on a residual neural network, wherein:
[0006] (i) An unsupervised learning module based on sum-product networks, which uses sum-product network model (H. Poon and P. Domingos, "Sum-product networks: A new deep architecture," in 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops), Barcelona, Spain, Nov. 2011, pp. 689-690.) to build multiple density estimators of the data set, so as to calculate approximate results for common SPJA queries (SPJA query refers to a Select-Project-Join (SPJ) query containing an aggregation operator in the SELECT clause). In this module:
[0007] In the offline learning phase, first, multiple data samples are randomly generated using a user-specified sampling rate for a given data set; then, a hybrid sum-product network is trained for each data sample, the hybrid sum-product network is a probability model for mixed type attributes, and each sum-product network learns the distribution of the data set according to the corresponding data sample, and finally multiple base density estimators are obtained;
[0008] In the online prediction phase, when the user query is input, the SPJA query of the user is parsed and converted into a probability expression, and the trained sum-product network model calculates the result of the expression through model inference to obtain the prediction value of the query;
[0009] The module uses a machine learning model to replace the data samples in the foregoing other methods for approximate calculation, and the model-based method performs calculation through model inference, which is faster than querying the samples; only the model parameters need to be stored, and the required storage space is greatly reduced; the model inference speed and storage space occupation present a sub-linear relationship with the data size, and have good scalability.
[0010] (ii) A supervised learning module based on residual neural network, which is a deep neural network with residual structure, integrates the foregoing multiple sum-product network models, and fuses the learned data distribution information to correct the prediction results of the sum-product network; in this module:
[0011] In the offline learning phase, a large number of queries (i.e. training data) are generated according to the SPJA query template, for each query, the foregoing multiple sum-product network models will calculate multiple prediction values, the query results predicted in the trained multiple sum-product networks and the feature vectors in the sum-product network prediction process are used as inputs, and the true results of the query are used as labels to train the deep neural network;
[0012] In the online prediction phase, the query input by the user is sequentially input into the aforementioned sum-product network model and the aforementioned trained neural network model, and finally a query prediction result with higher corrected accuracy is output;
[0013] The pre-experiment shows that the error of the single base density estimator based on the sum-product network mainly comes from the low selectivity of the query, and the number of tuples corresponding to the query is small and is easily ignored in the data sample, and the module fuses the information learned by the multiple base density estimators through the residual neural network, so that the network can capture more comprehensive original data distribution information, so as to improve the prediction accuracy of the query.
[0014] In the application, the data sample used in the training is selected by a specific sampling method according to the original data distribution characteristics, and the appropriate sampling rate and sample quantity are selected according to the spatial and temporal budget of the user; the base density estimator constructed is based on a mixed sum-product network model, and the sum-product network model is executed from bottom to top to calculate the probability distribution of the data; at the same time, the trained residual neural network model is composed of an encoder layer, a fusion layer and a decoder layer.
[0015] The generation of the data sample used for constructing multiple sum-product networks: according to the overall distribution characteristics of the original data set, a specific sampling method is selected, and the data with uniform distribution is randomly and uniformly sampled, and the data with skewed distribution is stratified sampled or weighted biased sampled. At the same time, according to the spatial budget of the user, the appropriate sample quantity is selected to improve the resource utilization benefit. Since the data sample does not need to be persistently stored after the model is trained, only the space resource consumed by the generated model needs to be considered; it is concluded that in the experimental setting of the application, the storage space size S occupied by a single model and the attribute number m of the data set are approximately linearly related, that is, S=Cm, wherein C is a constant that can be measured by experiment, and given the user's spatial overhead B, the maximum number of models that can be accommodated is n, that is,
[0016] When the sum-product network model is constructed from the data sample, the complete data set is recursively decomposed by rows and columns to speed up the model inference speed. In each round of recursive decomposition operation, there are three cases:
[0017] (1) If the data in the current node contains multiple column attributes, first calculate the correlation between columns according to the RDC correlation coefficient, group according to the correlation degree, decompose the column attributes with low correlation degree, and connect the decomposed data cluster using the product node;
[0018] (2) If the number of data records of the current node exceeds a specified threshold, the K-Means algorithm is used to cluster the data records by rows, and each data cluster is connected using a summation node; wherein the threshold size is generally taken as n s / 100, n s is the data sample size used for training a single model;
[0019] (3) If the data of the current node only involves one column attribute, and the number of data record entries is less than a specified decomposition threshold, a single variable distribution on the column attribute is returned to constitute a leaf node.
[0020] In the present application, the prediction of the query by the product-sum network model is converted into a bottom-up probability and expectation calculation, and for a relational data table R (i.e., a two-dimensional table stored by rows and columns), the conversion process is as follows:
[0021] (1) For a query in the form of SELECT COUNT(*) FROM R WHERE filter, the result can be expressed as where N is the total number of data records of the data set, and the expectation can be obtained by bottom-up calculation of the trained product-sum network.
[0022] (2) For a query in the form of SELECT AVG(A) FROM R WHERE filter, wherein A is an attribute on which an aggregation query is to be performed, the query can be expressed in the form of , and the calculation formula of the conditional expectation is wherein the expectation and the probability P(filter) can be obtained by bottom-up calculation of the product-sum network.
[0023] (3) For a SUM query, it can be expressed as COUNT*AVG, and the result can be obtained by performing two model inference calculations by the methods (1) and (2) described above.
[0024] In the present application, the product-sum network model is learned based on the data itself, and compared with other models based on query learning, it is more adaptable to changes in workloads, and the tree structure also makes data updating more convenient and robust.
[0025] In the present application, the residual neural network model is composed of an encoder layer, a fusion layer and a decoder layer, and the specific structure is as shown in Figure 1 , wherein:
[0026] (1) Encoder layer:
[0027] The encoder layer is composed of six fully connected neural networks, and the task of the encoder layer is to individually encode and represent the outputs of the multiple product network models; the input of the residual neural network includes the predicted values of the multiple product network models and the feature vectors extracted in the inference process of the product network model, the feature vectors can represent a specific state of a given model under a given query, the state contains both the data distribution information learned by the model itself and the feature information corresponding to the query, and the residual neural network can provide more learnable information; in the embodiment, the default number of product networks is 3, and the feature vectors (the dimensions of which are the same as the number of network nodes) of each product network are mapped to a vector space (512 dimensions) of the same dimension after being input into the corresponding encoder layer and being subjected to feature extraction and compression;
[0028] (2) fusion layer:
[0029] The fusion layer is composed of two linear layers (processing input and output) and four residual blocks, and the fusion layer inputs the outputs of the encoder layer into a fully connected neural network with a residual structure to learn more comprehensive data distribution features; the fully connected neural network has good vector representation capability, and the model is light and fast in inference; the residual blocks are added in the model training process, which can protect the integrity of the information, and reduce the loss of information in the propagation process by directly transmitting the input to the output; as shown in Figure 1 a residual block includes a 512-dimensional linear layer and a ReLU activation function;
[0030] (3) decoder layer:
[0031] The decoder layer is composed of six fully connected neural networks, and the task of the decoder layer is to input the feature vectors output by the previous two layers as input, and fit the relationship between the predicted results of the product network, the feature vectors and the real query results, so as to be able to predict the real query result for a given query input; a residual structure is added at the end of the network to limit the large fluctuation of the error, and the mean value of the preliminary results predicted by the multiple product networks is input into the decoder layer as a guide, so that the decoder layer learns a residual distribution rather than a complete output, the learning goal of the network is simplified and easy to train, so as to obtain a more accurate prediction result; the decoder layer performs nonlinear transformation on the output (512 dimensions) of the fusion layer, and finally outputs a one-dimensional predicted value.
[0032] The product network and residual neural network hybrid model has the following advantages:
[0033] This invention helps data analysts conduct interactive exploratory analysis of large-scale datasets by constructing machine learning-based data distribution models. It transforms common SPJA queries into inference calculations targeting the model, avoiding the high latency of executing precise queries on the original database. This results in faster query responses, improving user experience and decision-making effectiveness. Experimental results show that this invention can achieve sub-second response times (from 2 milliseconds to less than 500 milliseconds) when executing an aggregation query on a large dataset with hundreds of millions of records, while maintaining a low error level. Attached Figure Description
[0034] Figure 1 This is an architecture diagram of the hybrid model of sum-product network and residual neural network for approximate query processing in this invention.
[0035] Figure 2 This is a schematic diagram of the learning process based on sum-product networks in this invention.
[0036] Figure 3 This is a flowchart showing how a user performs a query and obtains the results using this model.
[0037] Figure 4 This is a comparison chart showing the query accuracy of this invention on the Flights dataset.
[0038] Figure 5 This is a comparison chart of query latency for the present invention on the Flights dataset. Detailed Implementation
[0039] The invention will be further illustrated below using a commonly used exploratory analysis dataset, and its advantages will be demonstrated by comparing it with sampling-based approximate query processing methods. The dataset is described below:
[0040] The Flights dataset is a US flight delay dataset containing statistics such as flight delay times and passenger volume, and includes six categorical attributes and six numerical attributes. In this invention, the Flights dataset was augmented to 500 million data tuples using the IDEBench method (a data augmentation method).
[0041] Figure 3 This paper demonstrates a comparison of the user query response processes using a sampling-based approximation method and the present invention. When data analysts interact with data analysis tools and execute interactive queries, for the sampling-based approximation method, the query is first rewritten as a calculation based on the samples, and after prediction using the constructed samples, the approximate result is returned to the data analysis tool. For the present invention, the query is first rewritten as a probabilistic expression based on the model, and after the model performs inference, the approximate result is returned to the analysis tool. In the experiment:
[0042] For the training of the join network model, the original dataset is the Flights data containing 500 million tuples, three data samples are obtained by sampling at a sampling rate of 1%, and one join network model is trained based on each of the three samples according to the foregoing method;
[0043] For the training of the residual neural network, 120,000 SPJA queries are generated for the Flights dataset, and the foregoing join network model is used to obtain preliminary prediction results and feature vectors of the join network, wherein 100,000 data are used as a training set, and 20,000 data are used as a validation set.
[0044] During testing, the RandomSampling-based approximate query method, the StratifiedSampling-based approximate query method, and the present application are compared on the Flights dataset, and the commonly used analytical workloads (containing 10 queries, denoted as F1-F10) on the Flights dataset are tested.
[0045] Figure 4 Fig. 4 shows a comparison of the relative errors of the test workloads when the RandomSampling-based approximate method, the StratifiedSampling-based approximate method, and the present application are used. On the Flights dataset, the query accuracy of the present application is better than that of the other two methods, and the relative errors of the queries based on the present application are all within 5%; while the average relative errors of the RandomSampling-based and StratifiedSampling-based methods are 9.68% and 5.22% respectively, and the relative errors of multiple queries are greater than 10%.
[0046] Figure 5 Fig. 5 shows a comparison of the query delays of the test workloads when the RandomSampling-based approximate method, the StratifiedSampling-based approximate method, and the present application are used. On the Flights dataset, the query delays of the present application are all within 50 milliseconds, while the average query delays of the RandomSampling-based and StratifiedSampling-based approximate methods are 11 times and 64 times that of the present application respectively, far exceeding the acceptable query waiting time of users. Therefore, compared with the other two sampling-based approximate query processing methods, the present application can greatly improve the response time of the aggregation query while providing query results with high accuracy.
Claims
1. A hybrid system of Hadamard product network and residual neural network for approximate query processing, characterized in that, Comprise two modules: an unsupervised learning module based on a product network and a supervised learning module based on a residual neural network, wherein: The unsupervised learning module based on the product network uses a product network model to construct a plurality of density estimators of a data set, so as to calculate an approximate result for a SPJA query, the SPJA query being a Select-Project-Join query containing an aggregate operator in a SELECT clause; in the module: In an offline learning stage, firstly, a plurality of data samples are randomly generated using a sampling rate specified by a user for a given data set; then, a hybrid product network is trained for each data sample, the hybrid product network being a probability model for mixed type attributes, and each product network learns the distribution rule of the data set according to the corresponding data sample, and finally a plurality of base density estimators are obtained; In an online prediction stage, when a user query is input, the SPJA query of the user is parsed and converted into a probability expression, and the trained product network model calculates the result of the expression through model inference to obtain a prediction value for the query; The supervised learning module based on the residual neural network is a deep neural network with a residual structure, which integrates the aforementioned plurality of product network models and fuses the learned data distribution information to correct the prediction result of the product network; in the module: In an offline learning stage, a large number of queries are generated according to a SPJA query template, for each query, the aforementioned plurality of product network models will calculate a plurality of prediction values, and the query result predicted by the plurality of product network models and the feature vector in the product network prediction process are used as inputs, and the true result of the query is used as a label to train the deep neural network; In an online prediction stage, the query input by the user is sequentially subjected to the aforementioned product network model and the trained neural network model, and finally a query prediction result with higher accuracy after correction is output.
2. The hybrid system of sum-product network and residual neural network for approximate query processing according to claim 1, wherein, Generation of data samples for constructing a plurality of product networks: according to the overall distribution characteristics of the original data set, a specific sampling method is selected, random uniform sampling is performed on uniformly distributed data, and stratified sampling or weighted biased sampling is performed on skewed data; at the same time, according to the spatial overhead budget of the user, a proper number of samples is selected to improve resource utilization efficiency.
3. The hybrid system of sum-product network and residual neural network for approximate query processing according to claim 2, wherein, When the product network model is constructed from the data samples, the complete data set is recursively decomposed by rows and columns to speed up the model inference; in each recursive decomposition operation, there are three cases: (1) If the data of the current node contains a plurality of column attributes, first calculate the correlation between columns according to the RDC correlation coefficient, group according to the correlation degree, decompose the column attributes with low correlation degree, and connect the decomposed data clusters using product nodes; (2) If the number of data records of the current node exceeds a specified threshold, use the K-Means algorithm to cluster the data records by rows, and connect each data cluster using a summation node; (3) If the data of the current node involves only one column attribute and the number of data records is less than the specified decomposition threshold, return a univariate distribution on the column attribute, which constitutes a leaf node.
4. The hybrid system of sum-product network and residual neural network for approximate query processing according to claim 3, wherein, The prediction of the query by the product network model is converted into a bottom-up probability and expectation calculation. For a relational data table, i.e., a two-dimensional table stored by rows and columns, the conversion process is as follows: (1) for queries of the form SELECT COUNT(*) FROM R WHERE filter, whose result is denoted as where N is the total number of data records of the dataset, computed bottom-up from the trained sum-product network; (2) For a query of the form SELECT AVG(A) FROM R WHERE filter, A is the attribute on which the aggregate query needs to be performed, the query is represented as the formula for the computation of the conditional expectation and P(filter) are both computed bottom-up by a product-sum network. (3) For the SUM query, it is expressed as COUNT*AVG, and the result is obtained by executing the model inference calculation twice by the methods (1) and (2) mentioned above.
5. The hybrid system of sum-product network and residual neural network for approximate query processing according to claim 3, wherein, The residual neural network model is composed of an encoder layer, a fusion layer and a decoder layer, wherein: The encoder layer is composed of six fully connected neural networks, and its task is to individually encode and represent the outputs of multiple product network models; the input of the residual neural network includes the predicted values of the multiple product network models and the feature vectors extracted in the inference process of the product network model, the feature vectors representing a specific state of the given model under the given query, which contains both the data distribution information learned by the model itself and the feature information corresponding to the query, providing more learnable information for the residual neural network; after the feature vectors of each product network are input into the encoder layer, they are mapped to the same dimensional vector space after feature extraction and compression; The fusion layer is composed of two linear layers and four residual blocks, which fuse the outputs of the encoder layer and input them into a fully connected neural network with a residual structure to learn more comprehensive data distribution features; residual blocks are added during model training to protect the integrity of information; by directly passing the input to the output, the loss of information during propagation is reduced; The decoder layer is composed of six fully connected neural networks, and its task is to take the feature vectors output by the previous two layers as input and fit the relationship between the predicted results of the product network, the feature vectors and the real query results, so as to predict the real query result for the given query input; residual structure is also added at the end of the network to limit the large fluctuation of errors, and the mean value of the preliminary results predicted by multiple product networks is input into the decoder layer as guidance, so that the decoder layer learns a residual distribution rather than a complete output, the learning goal of the network is simplified and easy to train, and more accurate prediction results are obtained.