Concurrent Database Transaction Handling via Segmented Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In scenarios with high concurrency, existing database systems face delays and inconsistencies due to the need for preceding transaction requests to be completed before subsequent requests can access data, leading to unsatisfactory user experiences and potential losses in web-based applications.
Innovation Solution
A system that classifies transaction requests as simple or complex, using a key-value store for pre-calculated results and a relational store for complex transactions, with a synchronizer ensuring data consistency between the two, allowing for efficient handling of concurrent requests.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a single database system processes all transaction requests sequentially to ensure data consistency, then data accuracy is maintained, but processing speed and user satisfaction deteriorate under high concurrency
Solution Approach 1:
The patent segments the database system into two independent storage engines: a key-value store engine for simple transactions and a relational store engine for complex transactions. Each engine operates independently with its own transaction processing path, allowing parallel execution of simple and complex transactions simultaneously. This segmentation eliminates the sequential bottleneck while maintaining data consistency through separate transaction logs and synchronization mechanisms.
Solution Approach 2:
The key-value store engine pre-calculates and stores results for simple transactions in advance, enabling rapid retrieval and execution without requiring complex query processing at request time. This preliminary preparation of data in the key-value store allows the system to handle simple transactions with minimal latency, significantly improving overall throughput under high concurrency.
2Productivity
If transaction requests are processed in parallel to improve speed, then productivity increases, but data consistency and accuracy worsen due to concurrent access conflicts
Solution Approach 1:
By dividing the transaction processing into separate paths for simple and complex transactions, the system allows parallel execution without conflict. Simple transactions use the key-value store engine with its own isolation mechanisms, while complex transactions use the relational store engine with full ACID compliance, eliminating concurrency conflicts while maintaining parallelism.
Solution Approach 2:
The patent introduces transaction logs and synchronization mechanisms as intermediaries between the key-value store engine and relational store engine. These intermediaries coordinate data changes and ensure consistency across both engines, allowing parallel processing while preventing data accuracy degradation through controlled synchronization.
3Reliability
If a complex relational database is used to handle all transactions to ensure data integrity, then reliability is maintained, but system complexity and processing overhead increase
Solution Approach 1:
The patent segments the database system into two specialized engines: a simple key-value store engine for straightforward transactions and a complex relational store engine for intricate transactions requiring full SQL capabilities. This segmentation allows each engine to be optimized for its specific purpose, reducing overall system complexity while maintaining data integrity through targeted design.
Solution Approach 2:
Different parts of the system are given different qualities appropriate to their function: the key-value store engine uses simplified storage and retrieval mechanisms for speed, while the relational store engine uses full relational capabilities for complex queries. Each component has locally optimized characteristics matching its transaction type requirements.
4Loss of time
If simple transactions use pre-calculated results for speed, then processing time is reduced, but data freshness and consistency may worsen
Solution Approach 1:
The key-value store engine pre-calculates and stores results for simple transactions in advance, enabling rapid retrieval without complex query processing. This preliminary preparation significantly reduces execution time while maintaining data freshness through synchronization with the relational store engine.
Solution Approach 2:
The system implements feedback mechanisms where the relational store engine monitors changes in the underlying data and triggers updates to the key-value store when data freshness requirements are met. This feedback loop ensures that pre-calculated results remain synchronized with the latest data state, maintaining reliability while benefiting from pre-computation speed.
Data Source
AI summary
A request handler may receive transaction requests for transactions to be executed using data of a database, and may classify a first transaction request of the transaction requests as a simple transaction request, and a second transaction request of the transaction requests as a complex transaction request. A key-value store engine may execute a first transaction satisfying the first transaction request, using a key-value store of pre-calculated results determined prior to receipt of the first transaction request, and based on the data, and may update a key-value delta reflecting a change, if any, of the key-value store caused by the first transaction. A relational store engine may cause the at least one processor to execute a second transaction satisfying the second transaction request, using a relational store including a subset of the data, and may update a relational delta reflecting a change, if any, of the relational store caused by the second transaction. A synchronizer may execute a synchronization of the key-value store and the relational store, based on the key-value delta and the relational delta.