Database Query Segmentation for Event Log Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Databases face challenges in quickly responding to heavyweight event log queries without affecting normal operations, leading to potential service disruptions and instability.
Innovation Solution
The approach involves splitting heavyweight event queries into multiple sub-queries, each associated with a different time period, ensuring that the amount of data queried does not exceed the database's processing capability, thereby allowing the database to respond without impacting read and write operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If the database processes a heavyweight event log query involving a large amount of data, then the query can be completed, but the normal operations of the database (read and write operations) are affected and stability deteriorates
Solution Approach 1:
The patent divides a heavyweight query involving a large amount of data into multiple sub-queries, each operating on a smaller subset of data. This segmentation allows the database to process the query in manageable chunks, preventing resource exhaustion and maintaining stability during query execution.
Solution Approach 2:
The system dynamically adjusts query processing by evaluating the size of the query and automatically splitting it into sub-queries when necessary. This dynamic approach allows the database to adapt its processing strategy based on the specific query characteristics, ensuring optimal performance and stability.
2Quantity of substance
If the database processes a heavyweight event log query, then the query result is obtained, but the response time increases and normal operations are impacted
Solution Approach 1:
By segmenting the large query into smaller sub-queries, each sub-query can be executed more quickly and efficiently. The database can process multiple smaller queries in parallel or sequentially without the performance degradation associated with a single large query, thus improving overall response speed.
Solution Approach 2:
The system processes the query in partial steps through sub-queries, completing the full query through multiple incremental actions. This approach allows the database to make progress on the query without being overwhelmed by the total data volume, maintaining acceptable response times.
3Loss of information
If the database handles a large amount of data in a single query, then the complete information is retrieved, but the database resources are overwhelmed and normal operations are disrupted
Solution Approach 1:
The query is divided into multiple sub-queries that collectively retrieve all the required information. Each sub-query operates on a specific subset of data, ensuring that no information is lost while maintaining resource efficiency. The results from all sub-queries are aggregated to provide the complete query answer.
Solution Approach 2:
The results from multiple sub-queries are merged and combined to form the final comprehensive query result. This merging process ensures that all required information is preserved and presented to the user, while the individual sub-queries maintain database resource efficiency.
Data Source
AI summary
A query is received from a user. A query event type and a query time range associated with the query are determined. An estimated amount of data to be queried associated with the determined query time range is determined based on at least a historical number of the query event type of the user. An allowable amount of data to be queried supported by a database for a single query is determined. One or more sub-queries for the received query are generated. Each sub-query is associated with a different time period within the determined query time range. A corresponding amount of data to be queried associated with each time period is less than, or equal to, the determined allowable amount of data to be queried. The database is queried with the generated one or more sub-queries.