Deletion Vectors for Concurrent Table Updates Without Transaction Retries
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Concurrent transactions in data processing systems often fail due to intervening transactions altering the physical representation of data tables, leading to significant waste of time and resources as repeated transactions are necessary.
Innovation Solution
The use of deletion vectors to determine logical and physical prerequisites of concurrent transactions, allowing for their serialization and preventing failures by maintaining metadata of record validity, thus enabling efficient transaction commitment.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If concurrent transactions are allowed to access the same data table simultaneously, then system throughput and productivity are improved, but transaction reliability deteriorates due to conflicts from intervening transactions altering physical data representation
Solution Approach 1:
The patent introduces deletion vectors as an intermediary data structure that mediates between concurrent transactions. The deletion vector tracks which records have been deleted by winning transactions, allowing current transactions to detect conflicts without directly interfering with the physical data files. This intermediary mechanism enables conflict detection while maintaining concurrent access, resolving the contradiction between high throughput and transaction reliability.
Solution Approach 2:
The system implements feedback by having transactions check the deletion vector before committing. When a current transaction detects that its target records have been marked as deleted in the deletion vector (indicating a winning transaction modified them), the transaction aborts with a conflict error. This feedback loop prevents unreliable commits while allowing concurrent access to proceed, maintaining both productivity and reliability.
2Reliability
If transactions abort upon detecting concurrent conflicts, then data consistency is maintained, but time and resources are wasted due to repeated transaction attempts
Solution Approach 1:
The patent applies preliminary action by pre-tracking all deletions in the deletion vector before current transactions commit. Instead of discovering conflicts during execution, transactions can check the deletion vector in advance to determine if their target records have been deleted by winning transactions. This preliminary detection allows transactions to abort immediately without performing unnecessary work, reducing time loss while maintaining consistency.
3Productivity
If the physical representation of data tables is altered by winning transactions, then updated data is made available, but current transactions fail because they cannot locate or access the original data files
Solution Approach 1:
The deletion vector serves as an intermediary that decouples the physical data representation from the logical data access. Winning transactions can freely alter physical data files (improving update speed) while the deletion vector maintains a record of which logical records were affected. Current transactions query the deletion vector to determine accessibility without needing to know or access the physical file structure, maintaining ease of operation despite physical alterations.
Data Source
AI summary
A disclosed configuration receives a first indication that a first transaction is committed to update a first subset of records in a data table at a first version to generate a second version of the data table and receiving a second indication to commit a second transaction to update a second subset of records in a data file of the data table at the first version. The configuration determines a logical prerequisite based on whether the first subset of records changes content of one or more records in the second subset of records and determining a physical prerequisite on whether the second subset of records corresponds to respective data records in data files of the second version of the data table. The configuration commits the second transaction to generate a third version of the data table by updating elements of the deletion vector if the prerequisites are satisfied.