Message System Write Reduction via Conditional Logging

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Message systems face inefficiencies due to excessive write operations to persistent storage, which consume additional input/output resources and can lead to data loss during system crashes, especially when messages are written to multiple intermediate systems.

Innovation Solution

Implementing a method that selectively writes messages to persistent memory by delaying log writing until a trigger is met, such as consumer availability or bandwidth, and selectively logging across message systems, reducing the number of writes to persistent storage.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If messages are written to recovery log for every consumer request, then data reliability is improved, but system performance deteriorates due to excessive write operations

Engineering Contradiction:
Improvedata reliabilityVSAvoidsystem performance
Core Design Contradiction:
ReliabilityVSProductivity

Solution Approach 1:

The patent applies partial action by selectively writing messages to the recovery log based on specific conditions rather than writing every message. The system evaluates whether a message should be logged by checking conditions such as message importance, consumer availability, and current system state, thereby performing logging only when necessary to maintain reliability while avoiding unnecessary writes that would degrade performance.

Inventive Principle:
Principle #16Partial or excessive action

Solution Approach 2:

The patent implements dynamic logging behavior where the decision to write to the recovery log is not static but adapts based on real-time system conditions. The system dynamically adjusts logging frequency by monitoring consumer availability, message characteristics, and system state, allowing it to optimize between reliability and performance based on current operational context rather than following a fixed logging policy.

Inventive Principle:
Principle #15Dynamics

2Reliability

If messages are written to persistent storage immediately, then data loss prevention is improved, but I/O resource consumption increases

Engineering Contradiction:
Improvedata loss preventionVSAvoidI/O resource consumption
Core Design Contradiction:
ReliabilityVSUse of energy by moving object

Solution Approach 1:

The patent applies preliminary action by preparing messages for potential logging without immediately writing them to persistent storage. The system pre-evaluates messages and determines logging necessity based on anticipated conditions, allowing it to defer actual write operations until confirmation is needed. This preliminary assessment mechanism prevents unnecessary I/O operations while ensuring that messages requiring persistence are logged in a timely manner.

Inventive Principle:
Principle #10Preliminary action

Solution Approach 2:

The patent employs a lightweight in-memory message representation that can be discarded if logging is not required, avoiding the cost of persistent storage writes. The system maintains messages in a low-cost in-memory format and only transitions them to persistent storage when conditions dictate, effectively using a cheap temporary state to avoid expensive I/O operations for messages that don't require durability.

Inventive Principle:
Principle #27Cheap short-living objects (Disposable)

3Reliability

If messages are logged at every intermediate message system, then recovery capability is improved, but device complexity increases

Engineering Contradiction:
Improverecovery capabilityVSAvoidsystem complexity
Core Design Contradiction:
ReliabilityVSDevice complexity

Solution Approach 1:

The patent applies local quality by making each intermediate message system independently evaluate whether to log messages based on its local conditions and the specific message characteristics. Rather than requiring all systems to uniformly log every message, each system applies local judgment to determine logging necessity, reducing overall system complexity while maintaining recovery capability where needed. This decentralized decision-making eliminates the need for complex coordination between systems.

Inventive Principle:
Principle #3Local quality

Solution Approach 2:

The patent extracts the logging decision-making process from a centralized control mechanism and distributes it to individual message systems. Each system independently extracts and evaluates the necessary information from messages to determine if logging is required, removing the complexity of centralized coordination while preserving recovery capabilities. This extraction of decision logic to the edge simplifies the overall system architecture.

Inventive Principle:
Principle #2Taking out (Extraction)

Data Source

PatentUS11537454B2Reducing write operations in middleware
Publication Date: 2022.12.27 INTERNATIONAL BUSINESS MACHINE CORPORATION
  • US11537454B2 patent drawing
  • US11537454B2 patent drawing
  • US11537454B2 patent drawing

AI summary

Disclosed is a computer-implemented method for reducing write operations in a message system. The method includes receiving, by a message system, a message from a producer, wherein the message includes a first flag, the first flag indicating the message should be written to a recovery log, and the message is configured to be sent to a consumer. The method also includes adding the message to a queue. The method further includes determining, at a first time, the consumer is not available. The method includes verifying, in response to the consumer not being available at the first time, a trigger is not met. The method also includes determining, at a second time, the consumer is available. The method also includes sending, in response to determining the consumer is available, the message to the consumer.