Message Replay via Persistent Storage and Timestamps
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional message-based communication systems lack replay capability, leading to service disruptions during temporary processing delays or failures, as messages can only be read once and not replayed.
Innovation Solution
A method where messages from an input stream are persisted with timestamps and identifiers, allowing for replay by creating a replay function when a replay triggering event is detected, enabling messages to be resent to consumers from persistent storage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If messages are read once and dropped in conventional message-based communication systems, then the system operates with simple architecture and fast processing, but replay capability is lost leading to service disruptions during processing delays or failures
Solution Approach 1:
The patent applies preliminary action by persisting messages in persistent storage before they are consumed. This allows messages to be retained and replayed if needed, rather than being dropped after single consumption. The message broker stores messages in persistent storage as a preliminary step, enabling future replay operations without requiring the original message stream to be available.
Solution Approach 2:
The patent introduces persistent storage as an intermediary between the message stream and consumers. This intermediary layer allows messages to be buffered and made available for replay to multiple consumers over time, resolving the contradiction between simple fast processing and reliable replay capability.
2Adaptability or versatility
If messages are persisted in persistent storage for replay capability, then replay functionality is enabled, but system complexity increases and processing overhead is added
Solution Approach 1:
The patent applies universality by designing the persistent storage mechanism to serve multiple functions: it acts as a message buffer for normal operation, a replay source for failed messages, and a persistence layer for audit trails. This multi-functional approach enables replay capability without requiring separate dedicated systems for each function.
Solution Approach 2:
The message broker automatically manages message persistence and replay operations without requiring external intervention. When a consumer requests replay, the broker self-service by retrieving messages from persistent storage and forwarding them, eliminating the need for complex manual replay management systems.
3Productivity
If message queues are dimensioned for normal operational circumstances, then the system handles regular message rates efficiently, but queue overruns occur during unusual conditions such as processing delays or failures
Solution Approach 1:
The patent transitions from a single-dimension message queue approach to a two-dimensional architecture combining in-memory queues for normal operation with persistent storage for exceptional cases. This dimensional change allows the system to handle both high-speed normal processing and reliable replay during failures without compromising either productivity or reliability.
Data Source
AI summary
A technique for processing messages in a message-based communication scenario is described. In the communication scenario, messages are received from an input message stream, multiplied and forwarded to multiple message consumers, and persisted together with at least one of a time stamp and a message identifier in a persistent storage for later replay to the message consumers. A method aspect of this technique comprises detecting a replay triggering event associated with a message consumer. The method aspect further comprises, in response to detecting the replay triggering event, creating a replay function for the message consumer. The replay function is configured to receive persisted messages from the persistent storage and to send the persisted messages to the message consumer.