Query Service Isolates Data Analysis Engine in Multi-Tenant Log Systems
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Multi-tenant cloud architectures face challenges in securely granting access to shared electronic log data while preventing unauthorized access and overloading of computing resources.
Innovation Solution
A data analysis engine is isolated from direct network access through a query service that places requests on a queue for subsequent retrieval by a batch server, using batch processing and scheduling to manage resource demands and prevent overloading.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If direct network access is granted to the data analysis engine for log data queries, then query processing speed is improved, but system security and protection against unauthorized access deteriorate
Solution Approach 1:
A query service is introduced as an intermediary component between the data analysis engine and the log data storage. This query service receives queries from clients, validates authentication/authorization, formats queries appropriately, and submits them to the data analysis engine. The query service acts as a mediator that protects the data analysis engine from direct network exposure while maintaining efficient query processing through the queue mechanism.
2Ease of operation
If real-time query processing is implemented, then responsiveness is improved, but computing resource overload and system instability worsen
Solution Approach 1:
The system performs preliminary actions by pre-processing and validating queries before they reach the data analysis engine. The query service authenticates users, authorizes access, and formats queries in advance. Queries are also prioritized and scheduled appropriately, with critical queries processed immediately and less critical ones batched, preventing resource overload while maintaining responsiveness.
Solution Approach 2:
The system implements periodic batch processing for non-critical queries, where queries are collected and processed in periodic cycles rather than continuously. This allows the data analysis engine to process queries in manageable batches, preventing resource exhaustion while maintaining system responsiveness through asynchronous processing and notification mechanisms.
3Reliability
If comprehensive access control is implemented for log data, then data security is improved, but access complexity and operational difficulty worsen
Solution Approach 1:
The query service provides universal access control functionality that handles multiple security concerns through a single unified mechanism. It performs authentication, authorization, and query formatting in one service layer, making access control manageable rather than complex. The service maintains a centralized security context that handles all access decisions consistently, reducing operational complexity while maintaining comprehensive security.
4Productivity
If batch processing with scheduling is implemented, then resource utilization efficiency is improved, but query processing time worsens
Solution Approach 1:
The system implements dynamic scheduling that adjusts batch processing behavior based on real-time conditions. Critical queries can bypass the batch queue and be processed immediately, while non-critical queries wait for batch processing. The query service dynamically prioritizes queries based on user role, query type, and system load, optimizing both resource utilization and processing time rather than using a fixed static approach.
Data Source
AI summary
Systems and methods process log data relating to usage of a multi-tenant application server. An input module receives a request from a user that identifies requested log data by a particular user or tenant of the multi-tenant application system. A log query service automatically formats the database query for the requested log data based upon the request received from the user, and stores the formatted database query on the storage device. A batch server subsequently retrieves the database query from the storage device, submits the database query to the data analysis engine, obtains the requested information from the data analysis engine after the query is completed, and notifies the user that the requested information is available for output.