SaaS Backup System Batch Data Versioning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Businesses face high costs and resource inefficiencies in processing and retaining historical data from cloud-based SAAS applications, as most services overwrite past data with current information, lacking a comprehensive data collection system that preserves change history.
Innovation Solution
A system that fetches consistent datasets in batches for a given period, allowing retrieval of past data by creating new versions without overwriting existing batches, enabling the preservation of change history and providing historical data to tenants.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If new data overwrites past data in cloud-based SAAS applications, then storage cost is reduced, but loss of information occurs
Solution Approach 1:
The system segments data storage into multiple isolated batches, each representing a specific time period. Instead of a single monolithic storage structure where new data overwrites old data, the system divides data into discrete temporal segments (batches) that can be independently retained. This segmentation allows the system to maintain historical data without requiring proportional increases in storage resources, as only relevant batches are retained based on retention policies.
Solution Approach 2:
The system performs preliminary actions by capturing and storing data in isolated batches at the time of generation, rather than relying on later archival processes. By proactively creating time-stamped batches as data is generated, the system ensures historical data is preserved in its original state before any potential overwriting occurs. This preliminary capture mechanism enables cost-effective retention of historical data without requiring continuous additional storage allocation.
2Measurement precision
If detailed historical data is retained for all tenants, then measurement precision is improved, but device complexity increases
Solution Approach 1:
The system applies local quality by allowing each tenant to have customized data retention configurations specific to their needs. Different tenants can have different batch retention periods, data types retained, and access policies. This localized approach to data quality and retention means that not all tenants store all historical data indefinitely, reducing overall system complexity while maintaining high measurement precision for each individual tenant's specific requirements.
Solution Approach 2:
The batch data storage system serves multiple functions simultaneously: it provides historical data retention, enables data recovery, supports analytics, and manages storage resources. The same batch storage mechanism serves all tenants across different SAAS applications, providing a universal solution that reduces complexity compared to implementing separate historical data storage systems for each tenant or application.
3Loss of information
If data is fetched in batches for multiple time periods, then loss of information is reduced, but productivity decreases
Solution Approach 1:
The system implements periodic action by fetching and processing data in regular batches corresponding to specific time periods, rather than continuously processing all data. Data is collected, stored, and made available in discrete periodic intervals (batches), which maintains data completeness for historical analysis while improving processing efficiency. This periodic batch processing allows the system to maintain complete historical data records without the overhead of continuous real-time processing of all accumulated data.
Data Source
AI summary
A system fetches consistent datasets in batches for a given period of time and provides the ability to retrieve each batch. Batches of data may be fetched for an interval of time. The present system may fetch new or changed data from different cloud/on-premise applications. It will store this data in the cloud or on-premise to build data history. As the system fetches new data, existing batches of data will not be overwritten. New batches of data are created as new versions so that change history is preserved. Past batches of data for a past time period may be provided to one or more tenants.