Data Store Statistics Normalization via Rule Evaluation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Data store statistics become stale and compromised in integrity due to infrequent updates, leading to non-optimal query response plans, as the relationships among statistics are not maintained effectively, causing resource inefficiencies and suboptimal query execution.
Innovation Solution
A data store system that includes an array of persistent storage devices and a processor to generate, retrieve, and evaluate statistics based on predefined rules, adjusting values as necessary to maintain relationship integrity and store updated statistics for query response planning.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If statistics are collected infrequently to save resources, then resource consumption is reduced, but the statistics become stale and lose integrity
Solution Approach 1:
The system performs preliminary actions by collecting raw statistic data periodically and storing it without immediate processing. The normalization process is deferred until query planning time, allowing raw data to be accumulated without frequent expensive processing operations while ensuring fresh statistics are available when needed.
Solution Approach 2:
Instead of fully processing and normalizing all statistics immediately after collection, the system performs only partial action by storing raw statistics. The complete normalization is performed selectively only when queries require statistics, avoiding unnecessary processing when statistics are not needed, thus reducing resource consumption while maintaining integrity on-demand.
2Reliability
If statistics are updated frequently to maintain freshness, then statistics integrity is improved, but resource consumption increases
Solution Approach 1:
The system implements periodic action by collecting raw statistic data at regular intervals without performing expensive normalization operations during each collection cycle. This separates the frequent data gathering from less frequent processing, maintaining statistics freshness while improving resource efficiency by batching the computationally intensive normalization work.
Solution Approach 2:
The system enables self-service by automatically performing normalization only when queries trigger the need for statistics. The query optimizer detects when statistics are needed and initiates the normalization process autonomously, eliminating the need for continuous resource-intensive updates while ensuring statistics are fresh when actually used.
3Measurement precision
If individual statistic integrities are maintained independently, then individual statistic accuracy is improved, but relationship integrity among statistics is compromised
Solution Approach 1:
The system merges the validation of individual statistic integrities with the checking of relationship integrities into a unified normalization process. By combining these checks and performing joint adjustment of statistics that violate relationships, the system maintains both individual accuracy and relationship consistency simultaneously, resolving the contradiction between the two integrity requirements.
Solution Approach 2:
The system implements feedback by evaluating relationship integrities after individual statistics are determined and using this information to adjust statistics that violate relationships. The normalization process uses feedback from relationship validation to iteratively refine statistic values until both individual and relationship integrities are satisfied, ensuring comprehensive integrity maintenance.
Data Source
AI summary
A data store system may include an array of persistent storage devices configured to store a plurality of data store tables and a set of rules associated with a plurality of statistics. Each rule may include a relationship between at least two statistics. The data store system may further include a processor in communication with the storage device. The processor may generate the plurality of statistics on a data store table and retrieve the set of rules. The processor may evaluate the set of rules based on the plurality of statistics and, in response to the determination that at least one rule of the set of rules is violated, adjust a value of at least one of the statistics. The processor may store values of the plurality of statistics in at least one of the persistent storage devices, receive a query on the data store table, and use the stored statistics to generate a response to the query. A method and computer-readable medium are also disclosed.