Cache-Database Decoupling for High Traffic Data Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Large database systems, such as those used in the airline industry for inventory management, face challenges in handling high traffic and maintaining data quality while scaling to meet increasing user inquiries without impacting transaction processing and requiring costly and complex database management systems.
Innovation Solution
A data storage system that independently interfaces with both database systems and cache nodes, allowing read queries to be sent directly to cache nodes and write queries to be synchronized across both systems, ensuring data consistency and scalability without the need for replication components within the database.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If a cache system is introduced to handle high user inquiries, then response speed to users is improved, but data consistency between cache and database becomes difficult to maintain
Solution Approach 1:
The database system sends notifications to the cache system when data is updated, deleted, or inserted. The cache system receives these notifications and invalidates or updates corresponding cached data, ensuring consistency through feedback from the database to the cache
Solution Approach 2:
A notification mechanism acts as an intermediary between the database and cache systems. The database publishes change events that the cache subscribes to, enabling automated synchronization without direct tight coupling between the two systems
2Reliability
If the database system directly handles all user queries, then data quality is maintained, but the system cannot scale to handle high traffic
Solution Approach 1:
The system segments query handling between two components: the cache system handles read queries to protect the database from high traffic, while the database system maintains data quality. This segmentation allows the database to focus on data integrity while the cache handles traffic volume
Solution Approach 2:
The cache system serves as an intermediary layer between users and the database. It handles user queries and only contacts the database when necessary (on misses or updates), protecting the database from direct exposure to high traffic while maintaining data quality
3Reliability
If standard database management systems are used, then data storage is reliable, but the system becomes costly and complex
Solution Approach 1:
The cache system acts as an intermediary that simplifies the overall architecture by handling the complex task of managing high traffic and data synchronization, allowing the use of standard, reliable database systems without requiring specialized high-end database solutions
Solution Approach 2:
The system creates a copy of frequently accessed data in the cache system. This copy mechanism allows standard databases to be used while achieving high performance through the cache, avoiding the need for complex specialized database systems
Data Source
AI summary
Method and system of storing data by a software application. Each read query of a data storage system by a software application is first solely issued to a plurality of cache nodes, which returns the queried data if available. If not available, the software application receives a miss that triggers a fetch of the queried data from one or more database systems on a first dedicated interface. Upon having retrieved the queried data, the software application adds the queried data to at least one cache node on a second dedicated. Each writing of the one or more database systems by the software application is also concurrently performed in the at least one cache node. Hence, population of the at least one cache node is quickly done at each missed read query of the at least one cache node and at each write query of the data storage system.