Data-Aware Orchestration for Stateful Application Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Stateful applications face challenges when moved from on-premise data centers to cloud environments due to potential data loss during node restarts or reboots, and the use of remote data stores introduces latency and data consistency issues.
Innovation Solution
The implementation of data-aware orchestration facilitates the lifting and shifting of existing applications and their data into a distributed or cloud platform without developer intervention. This involves encapsulating the application in a container, using a driver to create a local storage volume, and enabling automatic partitioning and replication of data across compute nodes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If an external store is utilized in place of local storage to prevent data loss, then data reliability is improved, but data access speed deteriorates due to remote access latency
Solution Approach 1:
The patent segments storage into local and remote portions, with the local portion providing fast access for frequently accessed data and the remote portion providing backup for data protection. This segmentation allows the system to simultaneously achieve fast data access and high reliability.
Solution Approach 2:
The patent applies local quality by providing different storage characteristics for different data access patterns. Local storage with fast access speed is used for active data, while remote storage with higher reliability is used for backup and archival data, optimizing both speed and reliability for their respective use cases.
2Speed
If local storage is used for stateful applications, then data access speed is improved, but data loss occurs during node restarts or reboots
Solution Approach 1:
The patent implements preliminary action by creating a local portion of the data store before the application needs to access data, and by continuously synchronizing this local portion with the remote store. This ensures data is available locally for fast access while maintaining persistence through the remote backup.
Solution Approach 2:
The patent provides beforehand cushioning by maintaining a local copy of data that serves as a buffer between the application and the remote store. This local cushion allows the application to access data quickly while the remote store provides protection against data loss during node failures or reboots.
3Reliability
If remote data stores are used to prevent data loss, then data reliability is improved, but data consistency issues arise across multiple nodes
Solution Approach 1:
The patent implements feedback mechanisms through continuous synchronization between the local and remote data store portions. The system monitors changes and automatically updates both locations to maintain consistency, providing feedback loops that ensure data integrity across distributed nodes.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Data-aware orchestration with respect to a distributed system platform enables at least lifting and shifting of pre-existing applications and associated data without developer action. A volume of a local store is created automatically in response to a container comprising a user application that is non-native with respect to the distributed system platform. The volume is then exposed to the container for use by the application to save and retrieve data. The container and local store are co-located on a compute node providing at least high availability. The application and local store can be duplicated on one or more replicas providing reliability in case of a failure. Further, partitions can be created automatically in response to declarative specification.