Non-Relational Database Drop Column via Index and Prioritization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Non-relational databases face performance issues due to the need to scan entire tables to identify items for drop column operations, which can be resource-intensive and interfere with read/write requests, leading to inefficient data management and storage utilization.
Innovation Solution
Implementing efficient drop column techniques that create a backup of the table, allowing evaluation of the backup to identify items with the column to be dropped, thereby reducing the need for scanning the entire table and minimizing interference with other requests, and using prioritization and allocation schemes to manage deletion operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the entire table is scanned to identify items for drop column operations, then all items with the column can be identified, but computational resources are heavily consumed and the operation interferes with read/write requests
Solution Approach 1:
The system maintains a pre-computed index that stores the mapping between column names and item identifiers. When a drop column operation is requested, the system queries this pre-existing index to quickly identify all items containing the target column, rather than scanning the entire table at that moment. This preliminary preparation of the index structure enables fast identification of affected items without disrupting ongoing read/write operations.
Solution Approach 2:
The patent introduces an index structure as an intermediary data structure that sits between the storage engine and the drop column operation. This index acts as a mediator that pre-organizes column-item relationships, allowing the system to efficiently retrieve the list of items containing a specific column without directly accessing or scanning the main table data. The index structure handles the complexity of item identification, isolating the drop column operation from the bulk data.
2Reliability
If the entire table is scanned to identify items for drop column operations, then all items with the column can be identified, but storage resources are wasted scanning items that may not have the column
Solution Approach 1:
The system pre-computes and maintains an index that maps column names to the specific items containing those columns. This preliminary organization of data relationships allows the drop column operation to directly query the index for the exact subset of items that need modification, avoiding the waste of reading and processing items that do not contain the target column. The pre-computed index eliminates redundant computational work.
Solution Approach 2:
Instead of treating the entire table uniformly during the drop column operation, the system uses the index to identify and operate only on the local subset of items that actually contain the column to be dropped. This localized approach ensures that computational resources are concentrated only on the relevant items, avoiding the energy waste of processing the entire table when only a portion of it is affected.
3Ease of manufacture
If drop column operations are performed without prioritization schemes, then the operations can be executed, but they interfere with read/write requests and degrade overall system performance
Solution Approach 1:
The system implements dynamic prioritization that adjusts the execution schedule of drop column operations based on current system conditions. When read/write request loads are high, the drop column operations are automatically deprioritized or scheduled for lower-utilization periods. This dynamic adjustment allows the system to maintain simple operation semantics while adapting to changing workload conditions, ensuring that drop column operations do not consistently interfere with time-sensitive read/write requests.
Solution Approach 2:
The patent employs periodic scheduling mechanisms where drop column operations are executed during designated time windows or when system resources are available, rather than immediately upon request. This periodic execution pattern allows read/write operations to proceed without interruption during critical periods, while still ensuring that drop column operations are eventually performed. The system balances the simplicity of operation execution with the need to protect concurrent request performance through time-based separation.
Data Source
AI summary
Columns of a non-relational data store may be efficiently dropped. A drop column request for a table stored in a non-relational database may be received. In response to the drop column request, schema information for the table may be updated to filter out the column from subsequent read responses to read requests. A backup of the table is created and evaluated to identify those items in the table that include the column. The column is then deleted from the identified items.