Database Hotspot Key Detection With LRU Cache QPS Counting
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing database systems struggle to accurately count query per second (QPS) for hotspot primary keys, especially when they do not exist or are deleted, leading to inaccurate and non-real-time QPS counting.
Innovation Solution
Implement a method using a cache based on the least recently used (LRU) algorithm to filter out low-frequency primary keys and accurately count QPS by determining whether the primary key exists in a real-time hotspot primary key set, accumulating query frequencies, and outputting results after each time period.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional database query counting methods are used, then all primary keys can be counted, but the counting is not accurate for non-existent or deleted primary keys and cannot provide real-time results
Solution Approach 1:
The patent pre-allocates a fixed-size array to store hotspot primary key information before actual querying begins. This preliminary preparation enables O(1) time complexity for both adding new primary keys and querying existing ones, achieving real-time detection without compromising accuracy for non-existent or deleted keys.
Solution Approach 2:
The patent creates a separate hotspot primary key set that copies only the most frequently queried primary keys from the full database. This copying approach allows the system to track QPS for hotspot keys in real-time while ignoring non-existent or deleted keys that don't appear in the hotspot set, thus achieving both real-time performance and accurate counting.
2Productivity
If all primary keys are tracked for QPS counting, then comprehensive coverage is achieved, but system load increases and performance decreases
Solution Approach 1:
The patent extracts only the hotspot primary keys from the full set of database primary keys and stores them in a dedicated fixed-size array. By taking out only the necessary subset of frequently queried keys, the system reduces the tracking burden from potentially millions of keys to a manageable fixed number, thereby improving query performance while maintaining accurate QPS counting for hotspots.
Solution Approach 2:
The patent segments the primary key tracking into two parts: a fixed-size array for hotspot primary keys and the rest of the database keys. This segmentation allows the system to apply optimized tracking only to the hotspot segment, reducing overall system load while maintaining comprehensive monitoring capabilities for the critical subset.
3Speed
If a fixed-size array is used to store hotspot primary keys, then real-time O(1) detection is achieved, but the system cannot accommodate more than the fixed number of hotspot keys
Solution Approach 1:
The patent implements a replacement strategy where when the fixed-size array is full and a new hotspot primary key needs to be added, the least recently used or least frequent entry is discarded to make room for the new key. This discarding and recovering mechanism allows the system to maintain fixed-size O(1) performance while adapting to changing hotspot patterns over time.
Data Source
AI summary
According to the embodiments of the present disclosure, a method and apparatus for hotspot data detection for a database and a storage medium are provided. The method includes: receiving a database query instruction, and determining whether a target primary key of data to be queried exists in a real-time hotspot primary key set corresponding to a current time period; if the target primary key does not exist, controlling, through a target cache, whether to add the target primary key to the real-time hotspot primary key set, the target cache is a cache; if the target primary key exists, accumulating, a query frequency of the target primary key in the current time period; and determining, after the current time period ends, a hotspot primary key of the current time period based on the real-time hotspot primary key set, and outputting a query frequency of the hotspot primary key.