Message Batching for Network Overhead Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Concurrent messaging systems face inefficiencies due to increased network and storage overhead from multiple round-trips, which reduce system throughput and increase CPU utilization, especially when handling small operations in durable stores.
Innovation Solution
Implementing batching techniques to collect and process messages or acknowledgments in batches, grouping related messages by session IDs, and optimizing storage operations to reduce network calls and storage communications, thereby minimizing overhead and improving latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If independent calls to the message server are made by different clients, then message delivery is achieved, but the number of round-trips to the message server increases
Solution Approach 1:
Multiple independent client calls are merged into a single batch operation. The server collects messages from multiple clients in one operation and processes them together, reducing the number of round-trips while ensuring reliable delivery of all messages.
Solution Approach 2:
The server performs preliminary actions by pre-allocating resources and preparing message handling infrastructure before actual message processing. Batch operations are prepared in advance, allowing multiple messages to be processed efficiently in a single round-trip cycle.
2Loss of information
If independent calls with extra headers are made, then message metadata is transmitted, but network activity and CPU utilization increase
Solution Approach 1:
Message headers and metadata from multiple independent calls are merged into a single batch operation. Instead of processing headers separately for each message, the system consolidates metadata handling, reducing redundant CPU operations and network activity while preserving all necessary message information.
3Reliability
If independent operations are performed on the durable store, then message persistence is achieved, but the number of round-trips to the durable store increases
Solution Approach 1:
Multiple independent persistence operations to the durable store are merged into a single batch operation. The system consolidates multiple write operations into one atomic batch transaction, reducing the number of round-trips to the durable store while ensuring all messages are persisted reliably.
Solution Approach 2:
The system performs preliminary buffering of messages in memory before writing to the durable store. This allows multiple messages to be prepared and staged for persistence, then written in a single batch operation, improving productivity while maintaining persistence reliability.
4Ease of operation
If small I/O operations are performed frequently, then message operations are executed, but overall system throughput reduces
Solution Approach 1:
Multiple small I/O operations are merged into larger batch operations. The system consolidates frequent small message operations into periodic batch operations, reducing the total number of I/O calls while maintaining ease of operation through the batch interface, thereby improving overall system throughput.
Data Source
AI summary
Network and storage calls are reduced by batching messages. Messages are collected from a client and sent to the gateway or backend application in one round trip. Alternatively, the messages are collected for different entities, and the batched messages are durably stored. Related messages, which may have the same sessionID, are grouped into a logical unit or session. This session may be locked to a single message consumer. A session may be associated with an application processing state as an atomic unit of work while other operations are performed on the messages in the session. Acknowledgements are accumulated by a broker on a message server, but the messages are not immediately deleted. Instead, multiple messages in a selected range are periodically truncated from a message store in a single operation. Expired messages for active sessions are proactive cleaned up to prevent sessions from reaching a quota or limit.