Dynamic Persistence Manager for Messaging Systems
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current messaging systems have an inflexible persistence policy, leading to excessive resource investment in ensuring message integrity for all messages, regardless of their business value, which results in performance bottlenecks and inefficient use of resources.
Innovation Solution
A messaging system with a persistence manager that evaluates the cost and benefit of persisting messages based on their value and risk of loss, allowing for dynamic determination of whether to save messages to persistent storage, thereby optimizing resource allocation and reducing unnecessary disk writes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If persistent messaging is used to ensure message delivery and recoverability, then message integrity and reliability are improved, but system performance and throughput deteriorate due to frequent disk writes
Solution Approach 1:
The patent applies local quality by differentiating persistence requirements across different messages based on their individual business value. High-value messages receive persistent storage while low-value messages use non-persistent storage, allowing the system to optimize reliability for critical messages without sacrificing overall throughput for all messages
Solution Approach 2:
The patent implements dynamics by making the persistence policy adaptive rather than static. The system dynamically evaluates message value, system state, and risk factors to determine persistence decisions in real-time, allowing the persistence strategy to adjust based on current conditions rather than following a fixed rule for all messages
2Reliability
If all messages are persisted to ensure integrity, then message recoverability is improved, but resource allocation efficiency deteriorates due to excessive disk I/O operations
Solution Approach 1:
The patent applies local quality by assigning different persistence treatments to different messages based on their individual characteristics. Instead of uniformly persisting all messages, the system evaluates each message's business value and applies persistence only where necessary, reducing unnecessary disk I/O operations while maintaining recoverability for valuable messages
Solution Approach 2:
The patent implements partial action by applying persistence to only the necessary subset of messages rather than all messages. The system performs partial persistence based on message value thresholds and risk assessments, avoiding excessive disk operations for low-value messages while ensuring adequate protection for high-value messages
3Reliability
If persistent storage is used for every message operation, then data integrity is improved, but system responsiveness and performance deteriorate
Solution Approach 1:
The patent applies local quality by providing differentiated data integrity protection based on message importance. Critical messages receive full persistent storage protection while less important messages use faster non-persistent storage, allowing the system to maintain data integrity for essential operations without incurring time penalties for all operations
Solution Approach 2:
The patent implements dynamics by making the data integrity mechanism adaptive. The system dynamically determines the appropriate level of persistence based on real-time evaluation of message value, system state, and risk factors, allowing response times to be optimized for low-risk operations while maintaining integrity for high-risk operations
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Described are methods, apparatus and computer programs for managing persistence within a data processing system such as a messaging system, database system or file system. The method for managing persistence includes a deferred evaluation (230,330,430) of at least one criterion associated with costs and/or benefits of saving to persistent storage, instead of persistence behaviour being fully predefined. The evaluation may be performed when a disk write is about to be performed (240,340,440), or at various times during processing of a data update and at various points within a data processing network. In a messaging solution, the method for managing persistence includes a dynamic evaluation (230,330,430) of costs and/or benefits of saving to persistent storage, with the evaluation being performed at various points in the messaging network after the message has been created and sent by the originating entity. The method includes determining whether message data and/or log records relating to the message require saving to persistent storage, in accordance with the costs and/or the benefits of saving to persistent storage. The benefits of persisting may be determined with reference to a message value (400).