Dynamic Database Partitioning for Faster Query Response
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The increasing computational load and response time in databases due to the growing amount of data being processed, leading to longer wait times for clients receiving responses.
Innovation Solution
A computer-implemented engine that generates and updates partition statistics based on correlations between data records, relocating records to optimize database partitions and reduce the number of partitions needed to search for a response, thereby minimizing computational load.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the database processes all data records to generate responses, then the completeness of data retrieval is improved, but the computational load and response time increase
Solution Approach 1:
The patent divides the database into multiple partitions based on correlation analysis of data records. By segmenting the database structure and using partition statistics to identify which partitions contain relevant data, the system retrieves only necessary partitions rather than scanning all data records, thus maintaining completeness while reducing response time and computational load.
2Measurement precision
If the database scans all partitions to ensure complete data retrieval, then the accuracy of response is improved, but the computational load increases
Solution Approach 1:
The patent performs preliminary correlation analysis and partition statistics generation to pre-identify which partitions are likely to contain relevant data for given queries. This preliminary action creates partition statistics that guide subsequent query operations, allowing the system to scan only relevant partitions rather than all partitions, thereby maintaining accuracy while reducing computational load.
3Device complexity
If the database maintains fixed partitions, then the structural simplicity is improved, but the adaptability to changing data access patterns deteriorates
Solution Approach 1:
The patent implements dynamic partitioning where partition structures and statistics are continuously updated based on observed data access patterns and correlations. The system monitors query workloads and recalibrates partition assignments accordingly, allowing the database to adapt to changing access patterns while maintaining a relatively simple overall partitioning framework.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
The present disclosure relates to database technology and in particular to dynamically updating and customizing database partitions. A computer-implemented engine is disclosed for identifying and retrieving a number of data records applicable to generate a response to a request, the engine having access to at least two partitions. Partition statistics are generated indicating correlations between the data records and, based on that partition statistics, the data records having the strongest correlation with each other are relocated to partitions so that the number of partitions which have to be queried in order to generate a response to a data request is minimized. Furthermore, the computational load caused when generating responses is more equally distributed across the partitions.