Search Query Result Count Estimation via Bayesian Networks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Database systems face inefficiencies in estimating search query result sets due to the large number of segments generated in disjunctive normal form, leading to significant resource consumption and time, especially for complex queries with many clauses and literals.
Innovation Solution
A data science model is employed to estimate search query result sets by parsing queries, identifying hierarchies of connected nodes, and using cross-hierarchy probabilities to generate an estimated count without executing the query, leveraging Bayesian networks to model attribute dependencies and independencies.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a basic trained model counts and persists the number of results for every possible segment in disjunctive normal form, then the result set count estimation is accurate, but the model size and computation time increase significantly
Solution Approach 1:
The patent extracts only the necessary information needed for estimation by maintaining counts for complete segments (segments involving all attributes) rather than all possible segments. This selective extraction reduces model size while preserving estimation capability through the use of complete segment extensions.
Solution Approach 2:
The patent segments the query evaluation process into two distinct phases: model training (which counts and persists results for complete segments only) and model use (which expands queries to complete segments and their extensions). This segmentation allows the model to be compact during storage while enabling comprehensive estimation during query processing.
2Device complexity
If a basic trained model maintains counts for complete segments and expands queries to complete segments and extensions, then the model size is reduced, but the time to get overall results count increases exponentially
Solution Approach 1:
The patent performs preliminary action during model training by pre-counting and persisting the number of results for every complete segment. This pre-computation stores the foundational data needed for rapid query estimation, eliminating the need for complex real-time calculations during model use and reducing computation time.
3Measurement precision
If a database system executes complex queries with many clauses and literals to get accurate result counts, then the estimation accuracy is high, but the resource consumption and time increase significantly
Solution Approach 1:
The patent uses copying by maintaining pre-computed count data for complete segments during model training. During model use, the system copies and combines these pre-stored counts for the relevant complete segments and their extensions, providing accurate estimation without executing the full complex query and consuming significant resources.
Data Source
AI summary
Search query result set count estimation is described. A system parses data set query that includes first query attribute and second query attribute. The system identifies first hierarchy of connected nodes including a first node representing a first query attribute, and a second hierarchy of other connected nodes including a second node representing a second query attribute. The system identifies a directed arc connecting first correlated node in first hierarchy to second correlated node in second hierarchy. The system identifies cross-hierarchy probabilities of correlations between values of a first attribute represented by the first correlated node and values of a second attribute represented by the second correlated node. The system outputs query result set estimated count generated from cross-hierarchy probabilities, probabilities that values of first attribute are associated with values corresponding to first node, and probabilities that values of second attribute are associated with values corresponding to second node.