Point-in-time Materialized View for Object Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Object storage environments face challenges in supporting applications with stringent Service Level Agreements (SLAs) due to the complexity and limited functionality of existing materialized views, which require collaborative distributed systems and two-phase processing, lacking control over data structures and load processing.
Innovation Solution
A method and system for point-in-time materialization in object storage environments, where storage object values are transformed into a relational database format within a materialized view, maintaining object identifiers and timestamps, allowing the view to satisfy queries without re-accessing the object storage, and enabling periodic updates and automatic refreshes to ensure data freshness.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If materialized views are implemented using collaborative distributed systems with two-phase processing, then data consistency is improved, but system complexity increases and functionality remains limited
Solution Approach 1:
The patent introduces an intermediary component that sits between the object storage environment and the relational database, translating object storage concepts into relational database terms. This intermediary handles the complexity of distributed system coordination internally while presenting a simplified interface to users, thereby maintaining data consistency without exposing system complexity.
Solution Approach 2:
The patent segments the materialized view implementation into distinct functional components: object storage interface, transformation layer, and relational database interface. This segmentation allows each component to handle specific aspects of the two-phase processing independently, reducing overall system complexity while maintaining data consistency through coordinated operation of segments.
2Loss of information
If frequent access to object storage is performed to ensure data freshness, then data up-to-dateness is improved, but access time and system performance deteriorate
Solution Approach 1:
The patent implements preliminary action by pre-materializing data from object storage into the relational database format before it is needed. The system proactively refreshes materialized views at optimized intervals or triggered by specific events, so that when queries are executed, the data is already fresh and available locally without requiring time-consuming access to object storage.
Solution Approach 2:
The patent employs periodic action by scheduling regular refresh operations for materialized views. Instead of continuously or on-demand accessing object storage, the system performs updates at predetermined intervals, balancing data freshness with performance. This periodic refresh strategy ensures data remains current while minimizing the frequency and duration of object storage accesses.
Data Source
AI summary
A query that is frequently processed to access an object storage is identified. Results from the query returned from the object storage is transformed into a relational database format as a materialized view. When the query is submitted a subsequent time, updated results are managed from the materialized view, other materialized views, and/or the object storage when needed.