Serverless CDP System Using Object Storage and Egress Triggers
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional Continuous Data Protection (CDP) techniques require significant resources and are expensive, especially for cloud-based systems, as they necessitate costly block devices and compute resources, which are inefficient for organizations with low IO operations.
Innovation Solution
Implementing a serverless or Function-as-a-Service (FaaS) based CDP system that stores data in object storage, allowing compute costs to be minimized by only paying for object storage and applying compute costs only when data arrives, with data recovery possible from any point in time, leveraging object storage's scalability and lower costs compared to block storage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional CDP techniques are used with block devices and compute resources, then data protection is achieved, but costs and resource consumption increase significantly
Solution Approach 1:
The patent extracts the compute function from the storage system by using egress triggers that automatically invoke serverless functions when data is written to object storage. This separates the continuous data protection mechanism from traditional block devices, allowing data to be stored in cheaper object storage while compute resources are only consumed when needed for data retrieval or protection operations.
Solution Approach 2:
The patent uses serverless functions that are ephemeral and on-demand, replacing persistent compute resources. These serverless functions are created only when data arrives or is needed, and are automatically terminated afterward, eliminating the need for continuously running compute instances and reducing resource consumption while maintaining data protection capabilities.
2Reliability
If block devices are used for CDP, then data protection is provided, but the cost of storage and compute resources increases
Solution Approach 1:
The patent creates copies of data in object storage with metadata that tracks the state of protected data. Instead of using expensive block devices for storage, the system stores data copies in cheaper object storage while maintaining protection capabilities through serverless functions that can restore data to any point in time based on the metadata.
Solution Approach 2:
The patent changes the storage parameter from block storage to object storage, and the compute parameter from persistent to on-demand. This parameter change allows the system to use cheaper storage tiers while maintaining data protection through event-driven serverless compute that only activates when needed, significantly reducing overall cost while preserving reliability.
3Reliability
If traditional CDP systems are deployed, then continuous data protection is achieved, but efficiency decreases for low IO operations
Solution Approach 1:
The patent uses event-driven periodic action where serverless functions are invoked only when data is written to object storage (through egress triggers) or when data retrieval is needed. This eliminates continuous compute resource consumption and inefficiency associated with traditional CDP systems that run constantly, even when there are no IO operations, thereby improving efficiency while maintaining continuous data protection capability.
Data Source
AI summary
A serverless application is provided to a cloud site of a cloud services provider. The cloud services provider offers backend services that include an object store and a database. Input/output (IO) writes sent to a volume of a block storage device at a production site are intercepted and aggregated. The aggregated IOs and metadata for the IOs are transmitted from the production site to the cloud site of the cloud services provider. Upon receipt of the aggregated IOs and metadata at the cloud site, the aggregated IOs are stored in an object in the object store at the cloud site, and a function of the serverless application is triggered to write the metadata to the database offered by the cloud services provider.