Tiered Middleware Blob Transaction Framework
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional database software, designed with ACID rules, is inflexible and inadequate for storing large chunks of data or scalable data storage needs, particularly in cloud-based blob storage systems where data must be stored in a manner that ensures atomicity, consistency, and durability while accommodating varying performance and cost considerations.
Innovation Solution
A tiered middleware framework with modular data transaction components that facilitates data transactions across arbitrary data blobs, using a master blob to ensure atomicity and consistency, and employing a transactionally consistent indexer to manage data access and storage efficiently, allowing for flexible configuration based on latency, availability, and cost tradeoffs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional database software with ACID rules is used, then data consistency and reliability are ensured, but flexibility and scalability are reduced
Solution Approach 1:
The patent segments the database system into multiple independent components: ACID-compliant relational database for critical data, NoSQL database for flexible data storage, and in-memory caching system for performance. Each component handles specific data types and access patterns, allowing the system to maintain reliability for critical operations while gaining flexibility through diverse storage options.
Solution Approach 2:
The patent implements dynamic data routing that automatically directs data operations to appropriate storage systems based on data characteristics, access patterns, and consistency requirements. This dynamic allocation allows the system to adapt to varying workloads and data types, providing both ACID compliance when needed and flexible NoSQL operations when appropriate.
2Reliability
If traditional database software with ACID rules is used, then data consistency is ensured, but performance and cost efficiency deteriorate
Solution Approach 1:
The patent applies different consistency models to different data types and operations based on their specific requirements. Critical financial transactions use strict ACID consistency, while less critical operations like analytics or caching use eventual consistency or relaxed isolation levels. This local optimization of consistency requirements improves overall system performance while maintaining data integrity where essential.
Solution Approach 2:
The system dynamically adjusts transaction isolation levels and consistency guarantees based on workload characteristics, data importance, and performance requirements. This allows the database to operate at higher performance levels for non-critical operations while maintaining strong consistency for essential transactions, optimizing the balance between reliability and productivity.
3Adaptability or versatility
If cloud-based blob storage is used, then scalability is improved, but data consistency and atomicity are compromised
Solution Approach 1:
The patent introduces an intermediary layer consisting of the relational database and application logic that mediates between blob storage and data consistency requirements. The relational database maintains ACID-compliant transactions for critical data, while blob storage provides scalable capacity for large objects. The application layer coordinates operations between these systems, ensuring atomicity through transaction management and consistency through proper sequencing of operations.
Solution Approach 2:
The patent segments data storage into two distinct systems: relational database for structured data requiring ACID compliance and atomicity, and blob storage for unstructured large objects requiring scalability. This segmentation allows each system to operate in its optimal mode while the application layer ensures overall data consistency by coordinating operations between the two systems.
4Quantity of substance
If cloud-based blob storage is used, then storage capacity is improved, but access performance deteriorates
Solution Approach 1:
The patent adds a temporal dimension to data storage by implementing a multi-layered caching architecture. Frequently accessed data is cached in fast in-memory systems, less frequent data is stored in cloud blob storage. This creates a hierarchical storage system where data can be accessed at different speeds depending on its access pattern, effectively adding a time-based dimension to the storage architecture that resolves the speed-capacity tradeoff.
Solution Approach 2:
The patent implements local caching of frequently accessed blob data in faster storage media at the application server level. This local optimization provides high-speed access for hot data while maintaining the scalable capacity of cloud blob storage for cold data, creating a hybrid access pattern that improves overall performance without sacrificing storage capacity.
Data Source
AI summary
A blob transaction component is a tiered middleware framework component that facilitates data transactions across arbitrary data blobs in an optimistically consistent manner. The component receives a request to transact data across multiple stored data blobs. For each stored data blob, the component creates a duplicate data blob. The component then creates a record blob that includes a pointer to the duplicate data blob. The stored data blob is then modified by the component to point to the record blob using optimistic locking. The component then creates a new data blob that includes modified data. The component further updates the record blob to include a pointer to the new data blob using optimistic locking. Following locking of the record blob, the stored data blob is updated with the modified data by the component, and the record blob, the duplicate data blob, and the new data blobs are deleted.