Materialized View Management Platform for Heterogeneous Data Replication

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

The increasing complexity and cost of managing and integrating data from multiple disparate data storage systems, particularly in scenarios where near real-time updates are required, pose challenges due to the need for custom code and complex integration workflows.

Innovation Solution

A materialized view management platform that enables the creation and maintenance of integrated views across heterogeneous data sources, allowing for near real-time updates without relying on batch-based changes, and providing a serverless experience with automatic scaling and simplified operational workloads.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If data is stored across multiple different locations and types of storage systems to optimize performance and analysis benefits, then data optimization for individual data sets is improved, but system complexity and integration difficulty increase

Engineering Contradiction:

Improvedata optimizationVSAvoidsystem complexity

Core Design Contradiction:

ProductivityVSDevice complexity

Solution Approach 1:

The patent introduces a materialized view management platform as an intermediary layer between disparate data storage systems and applications. This platform automatically manages data replication, synchronization, and access across heterogeneous storage systems, eliminating the need for applications to directly handle complex integrations while maintaining optimized data access paths.

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

The materialized view management platform provides universal access to data from multiple storage systems through a unified interface. It supports various data storage technologies and provides consistent data access methods, allowing applications to query and update data without needing to understand the underlying storage complexity.

Inventive Principle:

Principle #6Universality (Multi-functionality)

2Adaptability or versatility

If custom code and complex integration workflows are used to access data from multiple data storage systems, then data access capability is improved, but implementation cost and operational workload increase

Engineering Contradiction:

Improvedata access capabilityVSAvoidimplementation cost

Core Design Contradiction:

Adaptability or versatilityVSEase of manufacture

Solution Approach 1:

The materialized view management platform provides self-service capabilities by automatically detecting data sources, creating materialized views, and managing data synchronization without requiring manual coding or complex integration workflows. The system autonomously handles data replication, update propagation, and access optimization across multiple storage systems.

Inventive Principle:

Principle #25Self-service

Solution Approach 2:

The platform creates materialized views as copies of data from source storage systems, allowing applications to access replicated data without directly querying the original sources. These materialized views can be updated incrementally and provide the same data access capabilities while eliminating the need for custom integration code.

Inventive Principle:

Principle #26Copying

3Device complexity

If batch-based changes are used for data integration, then system simplicity is maintained, but update speed and near real-time performance deteriorate

Engineering Contradiction:

Improvesystem simplicityVSAvoidupdate speed

Core Design Contradiction:

Device complexityVSSpeed

Solution Approach 1:

The materialized view management platform dynamically adapts its update strategy based on the specific data source and target requirements. It can switch between batch updates for large datasets and incremental near-real-time updates for time-sensitive data, automatically optimizing the update frequency and method for each materialized view without requiring manual configuration.

Inventive Principle:

Principle #15Dynamics

4Productivity

If data is replicated across multiple target systems, then data accessibility and performance are improved, but data synchronization complexity and maintenance overhead increase

Engineering Contradiction:

Improvedata accessibilityVSAvoidsynchronization complexity

Core Design Contradiction:

ProductivityVSDevice complexity

Solution Approach 1:

The materialized view management platform implements feedback mechanisms that continuously monitor data changes in source systems and automatically propagate updates to all materialized views and target systems. The system tracks update states, detects conflicts, and resolves synchronization issues automatically, providing real-time feedback on data consistency across the distributed system.

Inventive Principle:

Principle #23Feedback

Data Source

PatentUS11544286B2Replicating materialized views across heterogeneous target systems

Publication Date: 2023.01.03 AMAZON TECH INC

AI summary

A materialized view created by a materialized view management platform may be replicated across many different target systems. The materialized view management platform allows users to specify different target systems. To create the materialized view in the different target systems, the materialized view management platform can translate the materialized view into different formats in order to store the materialized view in the different formats. Updates to the different instances of the materialized view are then made to keep the different materialized view formats in the different target systems up-to-date.