Hybrid Database Ingestion for Cloud Order Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional order processing systems in cloud computing environments face delays during peak times due to the time-consuming process of writing data to relational databases, which hinders immediate access to order information and affects customer experience.
Innovation Solution
Implementing a system that saves order data initially in a non-relational database for quick storage and then transfers it to a relational database for efficient querying, allowing access to order data before completion of processing and reducing CPU resource utilization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data is written to a relational database for order processing, then data consistency and query efficiency are improved, but storage speed and system response time deteriorate during peak times
Solution Approach 1:
The patent segments the database writing process into two independent parts: (1) asynchronous writing of order data to a non-relational database for immediate storage, and (2) subsequent writing to a relational database for consistency. This segmentation allows the system to achieve both fast storage and data consistency by separating the speed-critical path from the consistency-critical path.
Solution Approach 2:
The patent applies preliminary action by storing order data in a non-relational database before completing the relational database write operation. This preliminary storage enables the system to return order confirmation to customers immediately while the relational database write proceeds in the background, thus improving storage speed without sacrificing eventual data consistency.
2Productivity
If data is written to a relational database for order processing, then query efficiency is improved, but CPU resource utilization increases during peak times
Solution Approach 1:
The patent introduces a non-relational database as an intermediary layer between the order ingestion system and the relational database. This intermediary handles the initial storage and basic query operations, reducing the load on the relational database and its CPU resources. The relational database remains available for complex queries while the non-relational database handles high-volume simple operations.
3Speed
If data is stored in a non-relational database for quick access, then storage speed and access time are improved, but data consistency and query efficiency deteriorate
Solution Approach 1:
The patent merges two different database systems with complementary strengths: a non-relational database for fast storage and access, and a relational database for data consistency and complex queries. By combining these two systems working in parallel, the patent achieves both fast access time and strong data consistency, resolving the contradiction between speed and reliability.
4Reliability
If order data is processed sequentially in a relational database, then data integrity is maintained, but processing time increases during peak times
Solution Approach 1:
The patent applies dynamics by making the data storage system adaptive and flexible. Instead of a fixed sequential processing model, the system dynamically routes different types of operations to appropriate database systems: non-relational database for immediate storage and relational database for integrity-critical operations. This dynamic approach maintains data integrity while reducing processing time through parallel operations.
Data Source
AI summary
Disclosed are some implementations of systems, apparatus, methods and computer program products for performing data ingestion. A request received from a client device is processed, where the request has a payload including a plurality of data items pertaining to a data object. An identifier associated with the data object is generated. Information including the data items is obtained from the request. The information including the data items is stored in association with the identifier in a first data store, where the first data store is not a relational database. The data items and identifier are obtained from the first data store and stored in a second data store, where the second data store is a relational database. A query including the identifier is processed to obtain the identifier from the query. It is determined whether the identifier obtained from the query is in the first data store. One or more of the data items associated with the identifier are retrieved from the first data store according to a result of determining whether the identifier obtained from the query is in the first data store.