Multi-Blob Consistency Middleware for Atomic Data Transactions
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional database software, designed with ACID rules, is inflexible and inadequate for storing large or scalable data, particularly in cloud storage systems where data blobs require atomicity and consistency across multiple servers.
Innovation Solution
A tiered middleware framework with modular data transaction components ensures atomicity and consistency by using a master blob to manage data blob sets, ensuring that data transactions are executed as a single, consistent unit across multiple data stores, and employing a transactionally consistent indexer to facilitate efficient data retrieval and storage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional database software with ACID rules is used, then data consistency and atomicity are ensured, but the system lacks flexibility and scalability for large data storage
Solution Approach 1:
The system divides the database management functionality into separate modular components: a transaction manager that handles ACID compliance and a storage manager that handles blob storage operations. This segmentation allows each component to specialize in its function while working together, maintaining data consistency through the transaction manager while achieving scalability through the storage manager's ability to work with distributed blob storage systems.
Solution Approach 2:
The patent introduces a transaction manager as an intermediary layer between the application and the blob storage system. This mediator translates high-level transaction operations into low-level blob operations, ensuring ACID compliance without requiring the underlying storage system to natively support these rules. The intermediary handles complexity internally while presenting a simplified interface to applications.
2Reliability
If traditional database software is used, then strict ACID rules are enforced, but the system cannot easily store large chunks of data or scale to multiple servers
Solution Approach 1:
The system uses master copies and version copies of blob data to enable atomic transactions across multiple storage locations. When a transaction needs to modify blobs, the system creates version copies and uses master copies to coordinate the changes. This copying mechanism allows the system to maintain transaction atomicity while distributing data across multiple servers and scaling storage capacity without being constrained by traditional database limitations.
3Adaptability or versatility
If cloud storage servers are added to increase scalability, then data storage capacity increases, but ensuring atomicity and consistency across multiple servers becomes difficult
Solution Approach 1:
The transaction manager implements feedback mechanisms by continuously monitoring the state of blobs across multiple servers and adjusting its operations accordingly. When coordinating transactions across distributed storage, the system uses feedback from version copies and master copies to determine whether transactions have completed successfully, ensuring consistency even as the system scales to multiple servers. This feedback loop enables the system to maintain reliability while achieving scalability.
Data Source
AI summary
A multi-blob consistency component of a tiered middleware framework ensures data blobs are transacted in an atomic manner. The component determines a data blob of a data store to be modified based on an application request. The component then reads a master blob to locate a stored version number of the data blob to be modified and a version number of the master blob. A new data blob with a new version number that replaces the data blob to be modified is written to the data store. The component then reads the master blob again to re-obtain the version number of the master blob. Thus, when the obtained and re-obtained version numbers match, the component replaces the stored version number of the data blob with the new version number of the new data blob. Further, the component deletes the data blob to be modified using the stored version number.