MVaaS Architecture for Cloud Analytics Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current database systems lack a pluggable means to create and manage materialized views efficiently, leading to compute-intensive processes that impact existing transactions and result in latency during data updates and queries, especially in cloud-based analytics environments.
Innovation Solution
Implementing a Materialized View-as-a-Service (MVaaS) architecture that separates the management of materialized views into independent compute units, allowing for scalable and serverless execution, independent of transaction compute units, and enabling on-demand processing and data replication.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If materialized views are created using traditional database approaches, then query performance is improved through pre-computation, but system complexity increases and existing transactions are impacted during view refresh operations
Solution Approach 1:
The patent segments the materialized view management into separate compute units that are independent from transaction compute units. Each materialized view has its own dedicated compute unit that can be created, updated, and refreshed independently, preventing complexity propagation to existing transactions while maintaining query performance benefits.
Solution Approach 2:
The patent introduces an intermediary layer between the transaction compute units and materialized view computations. The materialized view compute units act as mediators that handle pre-computation tasks separately, allowing transactions to query results without being impacted by the computational complexity of view maintenance.
2Reliability
If materialized views are refreshed frequently to maintain synchronization, then data freshness is improved, but latency increases due to compute-intensive processes
Solution Approach 1:
The patent implements dynamic resource allocation where compute units are created on-demand for materialized views and can be scaled independently. This allows the system to adjust computational resources based on actual refresh needs, maintaining data freshness when required while minimizing latency during normal operations through flexible resource management.
Solution Approach 2:
The patent changes the operational parameters of materialized view maintenance by using separate compute units with independent scaling capabilities. This allows optimization of refresh frequency and computational intensity based on specific view requirements, balancing data freshness with operational latency.
3Device complexity
If materialized views are managed within the same compute unit as transactions, then system simplicity is maintained, but scalability is limited and resource utilization is inefficient
Solution Approach 1:
The patent divides the computing infrastructure into separate transaction compute units and materialized view compute units. This segmentation enables independent scaling of each component based on its specific workload requirements, improving scalability while maintaining operational simplicity through clear functional separation.
Solution Approach 2:
The patent creates a universal materialized view compute unit framework that can handle multiple views and queries simultaneously. This multi-functional approach allows efficient resource utilization across different workloads while maintaining scalability, as the same compute unit infrastructure can serve multiple purposes.
4Productivity
If data is persisted in both base tables and materialized view tables, then query performance is improved through pre-computation, but storage requirements and data redundancy increase
Solution Approach 1:
The patent extracts the pre-computation functionality into separate materialized view compute units that operate independently from transaction processing. This extraction allows the system to maintain data in base tables for transactions while using materialized views for analytical queries, reducing unnecessary redundancy by clearly separating transactional and analytical data pathways.
Data Source
AI summary
Embodiments may facilitate data analytics for a cloud computing environment. A plurality of system tables may store system metadata that associates a first materialized view with a first view and a first table. A transaction compute unit may process an update request relevant to the first view. A query parser may capture the update request from the transaction compute unit and detect which system tables are associated with the update request relevant to the first view. Responsive to the request, the query processor arranges for first table data to be replicated and transmit a materialized view request to a materialized view compute unit. The materialized view compute unit may be created as a Materialized View-as-a-Service (“MVaaS”) independent of the transaction compute unit. The materialized view compute unit may receive the materialized view request, refresh the first materialized view, compute the first materialized view, and store a result of the computation.