Query Management System for BI Platforms
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current business intelligence platforms face challenges in managing large volume data requests, leading to inefficient processing, server timeouts, and inconsistent performance due to their limitations in handling complex data and network traffic issues.
Innovation Solution
A method and system that convert queries exceeding certain thresholds into Extract, Transform, and Load (ETL) queries, compatible with big data platforms, allowing for data feeds and metadata conversion, thereby enhancing data processing efficiency and scalability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional web-based BI tools are used to process large volume data requests, then the platform can handle basic queries, but the processing efficiency deteriorates and server timeouts occur
Solution Approach 1:
The patent introduces an intermediary component that sits between the BI platform and data sources. This intermediary evaluates incoming queries, determines whether they exceed processing thresholds, and routes complex queries to ETL programs while keeping simple queries handled by the BI platform. This mediator architecture resolves the contradiction by preventing direct processing of large queries that would cause timeouts while maintaining efficient handling of smaller queries.
Solution Approach 2:
The patent segments query processing into two distinct pathways: simple queries are handled directly by the BI platform, while complex queries exceeding thresholds are segmented out and processed by external ETL programs. This segmentation allows the BI platform to maintain high efficiency for routine operations while offloading the burden of complex, resource-intensive queries that would otherwise cause server timeouts and performance degradation.
2Adaptability or versatility
If complex queries are processed directly by the BI platform, then query functionality is maintained, but server load increases and performance becomes inconsistent
Solution Approach 1:
The patent implements dynamic query routing that adapts to query complexity. The system evaluates each incoming query against defined thresholds and dynamically determines the appropriate processing path. This dynamic approach maintains versatility by handling all query types while ensuring consistent performance by adjusting the processing route based on query characteristics rather than using a static processing model.
Solution Approach 2:
The intermediary component acts as a smart router that assesses query complexity and directs appropriate queries to suitable processing systems. This mediator preserves the BI platform's versatility in handling various query types while maintaining performance consistency by preventing complex queries from overwhelming the platform's processing capabilities.
3Productivity
If ETL programs are used to process large data requests, then data processing efficiency improves, but system complexity increases
Solution Approach 1:
The patent introduces a management system that acts as an intermediary between the BI platform and ETL programs. This intermediary handles the complexity of coordinating with external ETL systems, managing query routing, and handling data feed subscriptions. By centralizing these complex management functions in a dedicated intermediary layer, the system achieves high processing efficiency through ETL while containing architectural complexity in a manageable component rather than distributing it throughout the entire system.
4Reliability
If query thresholds are set low to improve performance, then server load is reduced, but more queries are converted to ETL format increasing processing overhead
Solution Approach 1:
The patent implements adjustable query thresholds that can be tuned to optimize the balance between server load reduction and conversion overhead. By allowing dynamic adjustment of threshold parameters, the system can adapt to different operational conditions, data volumes, and performance requirements. This parameter flexibility enables optimization of server stability while minimizing unnecessary query conversions and their associated overhead.
Data Source
AI summary
A method is provided for managing queries in business intelligence platforms. The method includes receiving, by a processor, a first query requesting data from at least one business intelligence content, the first query having a first format. The method further includes determining, by the processor, at least one requirement to complete the first query. The method further includes converting, by the processor, the first query to a second query configured to be used by an Extract, Transform, and Load (ETL) program, in response to determining that the at least one requirement exceeds a threshold, the second query having a second format.