Database Statistics Estimation via Historical Trends
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current database systems face challenges in efficiently collecting and maintaining statistics, as frequent collection is costly and may not be feasible, especially for large databases, leading to stale data that hampers query optimization.
Innovation Solution
The use of historical data trends to make estimations and predictions about database data, allowing for less frequent collection of statistics and optimizing query execution by selecting stable data trends based on historical records.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If statistics are collected frequently to maintain data accuracy for query optimization, then query optimization effectiveness is improved, but system cost and resource consumption increase
Solution Approach 1:
The system performs preliminary analysis of historical statistics data to identify stable trends before actual query execution. By pre-processing historical data and establishing trend models in advance, the system can make accurate predictions without needing to collect fresh statistics frequently, thus reducing the cost of frequent statistics collection while maintaining optimization effectiveness
Solution Approach 2:
Instead of collecting actual current statistics data, the system creates copies or predictions based on historical trends. The prediction module generates estimated statistics data that replicates the information value of actual statistics without requiring actual data collection, thereby reducing system cost while maintaining measurement precision
2Loss of energy
If statistics collection frequency is reduced to lower system cost, then system cost decreases, but statistics become stale and query optimization effectiveness deteriorates
Solution Approach 1:
The prediction module acts as an intermediary between historical statistics data and current query optimization needs. Instead of directly collecting fresh statistics, the system uses the prediction module to bridge the gap by generating accurate predictions from historical data, thus maintaining statistics freshness without frequent collection
Solution Approach 2:
The system performs preliminary trend analysis on historical data to establish prediction models before statistics would normally become stale. This preliminary action enables the system to maintain accurate statistics predictions even when collection frequency is reduced, preventing staleness without increasing collection cost
3Measurement precision
If multiple data trends are analyzed to improve prediction accuracy, then prediction accuracy improves, but processing complexity increases
Solution Approach 1:
The system analyzes multiple data trends (more than the minimum single trend) to improve prediction accuracy. By examining several trends simultaneously and selecting the most relevant ones, the system achieves higher prediction accuracy while the increased processing complexity is managed through efficient trend selection and combination strategies
Data Source
AI summary
Data trends that are based on historical data of a database can be used to make estimations and/or predications about the database. In other words, historical trends of a database can be used to make estimation about the data of the database. By way of example, a prediction can be made regarding the data change in the data since the Statistics has been collected. An estimation about data in a database can be used instead of actual data that would have to be collected as Statistics for the database. As a result, Statistics can be collected less frequently but estimation and/or predictions about the database can be used to optimize the execution of queries of the database. Also, collection of Statistics for database can be altered by using estimation about the Statistics of the database by collecting Statistics less frequently or for longer periods of time.