Soft Deletion Shard for Sharded Database Performance
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Sharded databases face performance issues due to time-consuming processes when adding or deleting data, as rebuilding indexes is required, and past soft deletion methods negatively impact performance by requiring additional operations on the primary shard and indexes.
Innovation Solution
Introducing a soft deletion shard (SDS) that stores soft-deletion documents referencing soft-deleted primary data, allowing for deletion and restoration without modifying the primary shard or its indexes, thus eliminating the need for write, delete, or index update operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional deletion methods are used in sharded databases, then data can be removed from the database, but index rebuilding and write operations cause performance degradation
Solution Approach 1:
The database is divided into primary shards for data storage and a separate soft-deletion shard for tracking deleted documents. This segmentation allows deletion operations to occur in the soft-deletion shard without affecting the primary shards or their indexes, thus maintaining performance while enabling deletion functionality.
Solution Approach 2:
A soft-deletion document acts as an intermediary between the deletion request and the actual data. Instead of directly deleting data from primary shards, the system inserts soft-deletion documents into the soft-deletion shard that reference the deleted documents, thereby mediating the deletion process without impacting primary shard performance.
2Adaptability or versatility
If soft deletion is implemented on primary shards, then data can be restored, but additional write and index update operations are required
Solution Approach 1:
Restoration operations are segregated to the soft-deletion shard by removing soft-deletion documents, rather than performing write operations on primary shards. This segmentation eliminates the time-consuming index update operations that would otherwise be required on the primary shards.
Solution Approach 2:
The soft-deletion shard maintains a copy of deletion information (soft-deletion documents) that reference primary data. Restoration is achieved by manipulating these copies in the soft-deletion shard rather than directly modifying primary shards, thereby avoiding costly write and index operations on the primary data.
3Reliability
If indexes are updated during deletion operations, then data integrity is maintained, but the process becomes time-consuming
Solution Approach 1:
Index management is segmented from data deletion. The soft-deletion shard maintains its own indexing structure independent of the primary shards. Deletion operations only affect the soft-deletion shard's index, not the primary shards' indexes, thereby maintaining data integrity while simplifying the operational complexity.
Data Source
AI summary
An embodiment includes receiving a request to delete a specified document from a primary shard of a sharded database. The embodiment also includes inserting a soft-deletion document to a soft-deletion shard, wherein the soft-deletion document identifies the specified document, which remains in the primary shard. The embodiment also includes receiving a query from a client application, wherein the specified document satisfies the query. The embodiment also includes blocking the specified document from being returned in response to the query while the soft-deletion document associated with the specified document remains in the soft-deletion shard.