Order-Independent Hashing for Database Performance Comparison
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing database performance evaluation methods face inaccuracies due to simulated workloads not accurately reflecting production workloads, leading to false positives and inaccuracies in comparing database systems under changed parameters or software versions.
Innovation Solution
The implementation of preprocessing techniques involving independent hashing and filtering of database queries to generate order-independent hash values, excluding non-deterministic data, which reduces false positives and improves accuracy in comparing database performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If simulated or emulated workload is run on test system, then performance evaluation can be conducted, but accuracy of results compared to production system deteriorates
Solution Approach 1:
The patent applies preliminary action by pre-processing the captured workload to remove non-deterministic data elements before replaying it on the test system. This ensures that the workload characteristics are preserved and standardized in advance, allowing accurate performance comparison between production and test systems without being affected by timing variations or system-specific non-deterministic behaviors.
2Device complexity
If traditional hashing is used for result comparison, then comparison process is simplified, but false positives increase due to order dependence
Solution Approach 1:
The patent applies segmentation by dividing the result set into individual record components and hashing each record independently. This allows the comparison process to evaluate each record separately regardless of its position in the result set, eliminating order dependence while maintaining a relatively simple overall comparison mechanism through aggregation of individual record hash values.
3Adaptability or versatility
If non-deterministic data is included in workload replay, then workload completeness is maintained, but false positives increase
Solution Approach 1:
The patent applies the extraction principle by identifying and removing non-deterministic data elements from the captured workload before replay. This extraction process separates deterministic workload characteristics that are relevant for performance comparison from non-deterministic elements that would cause false positives, thereby improving comparison accuracy while maintaining the essential workload characteristics.
Data Source
AI summary
A process is provided for independently hashing and filtering a data set, such as during preprocessing. For the data set, one or more records, separately having one or more fields, may be identified. A record hash value set, containing one or more record hash values for the respective one or more records, may be generated. Generating a given record hash value may be accomplished as follows. For a given record, a hash value set may be generated, having one or more field hash values for the respective one or more fields of the given record. The record hash value for the given record may be generated based on the hash value set. A total hash value for the data set may be generated based on the record hash value set. The records of the data set may be filtered based on classification of the query that generated the records.