High-Throughput Message Generation via In-Memory Key-Value Caching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional methods for generating personalized messages in marketing campaigns face scalability issues and high latency when dealing with large numbers of customer records, as they require random database look-ups, which become inefficient at high throughputs.
Innovation Solution
Storing data in relational databases but performing relational database look-ups during the design phase and transforming tables into key-value data structures for efficient look-ups during the execution phase, allowing for high-throughput message generation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If random database look-ups are performed during message generation for each customer record, then data accuracy is maintained, but throughput decreases and latency increases when dealing with millions or billions of customer records
Solution Approach 1:
The patent applies preliminary action by transforming relational database tables into key-value data structures during the publishing phase, before the execution phase begins. This pre-processing step prepares the data in an optimized format that enables high-speed lookups during message generation without requiring random database access during execution, thereby resolving the contradiction between data accuracy and throughput
Solution Approach 2:
The patent creates copies of relational database tables in the form of key-value data structures. These copies contain the same data but are organized for efficient retrieval during message generation. The copying approach allows the system to maintain data accuracy while achieving high throughput, as the key-value structures can be quickly accessed without random database look-ups
2Adaptability or versatility
If relational database look-ups are performed during execution phase for each message, then data freshness is ensured, but system scalability deteriorates when customer records increase to millions or billions
Solution Approach 1:
The patent changes the data structure parameters by transforming relational tables into key-value pairs. This parameter change simplifies the data access model from complex relational queries to simple key-based lookups, enabling the system to scale to millions or billions of customer records while reducing access complexity
Solution Approach 2:
The key-value data structure acts as an intermediary between the relational database and the message generation process. During the publishing phase, data is transformed into this intermediate format, which then serves as the source for message generation during execution, eliminating the need for direct database access and improving scalability
Data Source
AI summary
Personalized messages are generated from a message template with high throughput by transforming data sources of the message template to key-value data structures that are stored in-memory and performing data look-ups using the in-memory key-value data structures.