Database Query Latency Reduction via Segmented Indexing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The large volume of metric data generated by server platforms, stored in multiple copies without optimal indexing, leads to significant latency in locating and retrieving specific log data, causing time and resource consumption during database queries.
Innovation Solution
A system and method that involve receiving data from virtual servers, identifying tokens associated with terms, mapping an index, storing it in a first database, and creating a key-value pair in a second database, optimizing queries, constructing a search key, and retrieving result values to reduce query processing time.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If metric data is stored in multiple copies without optimal indexing, then data redundancy and reliability are improved, but query latency and retrieval time increase
Solution Approach 1:
The patent segments the database into two distinct databases: an index database storing indexing information and a data database storing actual metric data. This segmentation allows the index database to be optimized for fast retrieval operations while the data database maintains data redundancy for reliability, thereby reducing query latency without compromising data reliability
Solution Approach 2:
The patent implements preliminary indexing actions by creating and maintaining an index database that pre-processes and organizes metric data before actual queries are executed. This preliminary organization of data through indexing reduces the time required for data retrieval while maintaining the redundant storage structure for reliability
2Quantity of substance
If large volume of metric data is stored, then data completeness and coverage are improved, but search efficiency and retrieval speed deteriorate
Solution Approach 1:
The patent introduces an intermediary index database that mediates between the large volume of metric data stored in the data database and the search operations. The index database contains pre-computed indexing information that enables efficient search operations without requiring direct scanning of the entire large data volume, thus maintaining search efficiency despite increased data quantity
Solution Approach 2:
By segmenting the storage system into an index database for metadata and indexing information, and a data database for actual metric data, the patent enables efficient searching of large volumes of data through the optimized index structure while maintaining complete data storage capacity
3Ease of manufacture
If metric data is not indexed optimally, then storage simplicity and implementation ease are improved, but query processing time and resource consumption increase
Solution Approach 1:
The patent segments the database system into specialized components (index database and data database) with distinct functions, making the indexing and query processing mechanisms explicit and manageable. This segmentation transforms the complex task of optimizing large data retrieval into manageable components, improving implementation ease while reducing query processing time
Solution Approach 2:
By introducing the index database as an intermediary layer between the application and the data database, the patent provides a structured approach to optimization that is easier to implement and maintain. The index database handles the complex indexing operations separately, making the overall system easier to implement while significantly reducing query processing time
Data Source
AI summary
A system for reducing database query latency, the system comprising: a memory storing instructions; and at least one processor configured to execute the instructions to perform operations comprising: receiving data reflecting performance of a role on a virtual server; identifying tokens associated with terms in the received data; mapping an index comprising the tokens and the terms; storing the mapped index in a first database; storing a key-value pair in a second database, the key corresponding to the mapped index, and the value corresponding to a portion of the received data; receiving a query; optimizing the query to reduce query processing time; constructing a search key based on results obtained by running the optimized query against the first database; retrieving a result value from the second database corresponding to the search key.