Tenant Data Truncation in Multi-Tenant Database Nodes
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In database systems, truncating a tenant from a multi-tenant table without impacting other tenants is challenging due to the need to wait for a truncate record to be flushed from the in-memory cache, which can take a considerable amount of time, preventing the truncated tenant from writing new records until the flush is complete.
Innovation Solution
Implementing a mechanism where the database node checks for truncate records in the in-memory cache before processing transactions, allowing it to determine if records were committed before or after the truncate record, and only returning records committed after the truncate record, enabling writes to the table without flushing the truncate record from the cache.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the database system waits for the truncate record to be flushed from the in-memory cache to the database, then data consistency is maintained, but the truncated tenant cannot write new records for a considerable amount of time
Solution Approach 1:
The system creates a copy of the truncate record and places it in the in-memory cache, allowing the database to observe the truncation immediately without waiting for the flush to complete. This copying mechanism enables the truncated tenant to write new records while maintaining data consistency through the cached truncate record observation
Solution Approach 2:
The truncate record is flushed to the database in advance before the tenant needs to write new records. By performing the flush operation preliminarily, the system ensures that when the tenant attempts to write, the truncation has already been persisted, eliminating the waiting time while maintaining consistency
2Reliability
If the database system traverses records to remove tenant data, then complete truncation is achieved, but many database operations are required impacting other tenants
Solution Approach 1:
The system extracts only the essential truncate record information (tenant identifier and timestamp) rather than traversing and removing individual data records. This extraction approach achieves complete truncation by marking the tenant for exclusion while avoiding the overhead of examining and deleting each record, thus minimizing impact on other tenants
Solution Approach 2:
The truncate record acts as an intermediary mechanism that mediates between the tenant deletion request and the actual data access operations. Instead of directly removing records, the system introduces the truncate record as a mediator that signals which records should be excluded, reducing database operations and protecting other tenants from extensive traversal
3Quantity of substance
If the in-memory cache is not flushed until it becomes full, then cache utilization is maximized, but the truncated tenant waits a considerable amount of time
Solution Approach 1:
The system performs the flush operation preliminarily, immediately after the truncate record is created in the in-memory cache, rather than waiting for the cache to become full. This preliminary flush ensures that the truncation is persisted before the tenant needs to write new records, eliminating wait time while the cache still has capacity for other operations
Data Source
AI summary
Techniques are disclosed relating to truncating a tenant's data from a table. A database node may maintain a multi-tenant table having records for tenants. Maintaining the table may include writing a record for a tenant into an in-memory cache and performing a flush operation to flush the record to a shared storage. The database node may write a truncate record into the in-memory cache that truncates a tenant from the table such that records of the tenant having a timestamp indicating a time before the truncate record cannot be accessed as part of a record query. While the truncate record remains in the in-memory cache, the database node may receive a request to perform a record query for a key of the tenant, make a determination on whether a record was committed for the key after the truncate record was committed, and return a response based on the determination.