DBaaS Orphan Data Cleanup Using Event-Driven Validation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The failure to remove orphan data from cloud computing platforms during deprovisioning leads to resource wastage, contractual violations, and legal non-compliance due to leftover objects and data, necessitating an efficient identification and cleanup process.
Innovation Solution
Implementing an argo cron workflow with dedicated service accounts for orphan data detection and cleanup, utilizing event service data to identify potential orphan data, and validating through service manager and SaaS manager databases to confirm and remove orphan instances and subscriptions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If automated deprovisioning processes are implemented to remove customer data, then data removal efficiency is improved, but errors may occur leaving orphan data behind
Solution Approach 1:
The system implements a feedback mechanism by periodically scanning the database for orphan data using event service databases that track deletion events. The orphan data identification process continuously monitors and detects data that should have been removed but remains, providing feedback to the cleanup process to correct these oversights.
Solution Approach 2:
The system performs preliminary actions by maintaining event service databases that record all deletion events before they are fully processed. This preliminary tracking allows the system to identify and correct orphan data before it causes problems, rather than waiting for issues to manifest.
2Reliability
If manual verification processes are added to ensure complete data removal, then data removal completeness is improved, but processing time increases
Solution Approach 1:
The system implements self-service by automatically identifying and cleaning up orphan data without requiring manual verification. The orphan data identification process autonomously scans the database, compares against deletion events, and removes identified orphan data, making the verification process self-executing rather than manual.
Solution Approach 2:
The automated feedback loop continuously monitors for orphan data and triggers cleanup operations automatically, eliminating the need for time-consuming manual verification while maintaining high completeness through systematic automated checking.
3Loss of energy
If orphan data is not removed, then resource consumption continues, but implementing cleanup processes adds system complexity
Solution Approach 1:
The system achieves multi-functionality by integrating orphan data identification and cleanup capabilities into existing database-as-a-service infrastructure. The event service databases serve dual purposes: tracking deletion events for legitimate data removal and identifying orphan data for cleanup, eliminating the need for separate dedicated systems.
Solution Approach 2:
The cleanup process operates autonomously, automatically identifying and removing orphan data without requiring complex external management systems. This self-service approach reduces overall system complexity while effectively addressing resource consumption from orphan data.
Data Source
AI summary
Disclosed herein are system, method, and computer program product embodiments for identifying and removing orphan data in a Database-as-a-Service (DBaaS) cloud computing platform. In embodiments, a workflow is executed that accesses an event service database of the platform to obtain a list of deleted customer entities, queries a control plane of the platform to obtain a list of existing DBaaS instances, determines that a particular existing DBaaS instance is associated with a particular deleted customer entity, identifies the particular existing DBaaS instance as a potential orphan DBaaS instance, validates that the particular existing DBaaS instance is an orphan DBaaS instance, and in response to validating that the particular existing DBaaS instance is an orphan DBaaS instance, performs one or more of generating an alert that identifies the particular existing DBaaS instance as an orphan DBaaS instance or removing the particular existing DBaaS instance from the DBaaS cloud computing platform.