Batched Remote Procedure Call for Message Queue Propagation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional messaging techniques require each message to be encapsulated in a separate remote procedure call (RPC) for propagation from a source queue to one or more destination queues, leading to inefficient use of system resources and reduced data sharing, as multiple copies of messages and metadata are transmitted repeatedly.
Innovation Solution
A computer-implemented remote procedure call that streams multiple messages from a source queue to one or more destination queues using a single RPC, with a first message header indicator, message body indicators, and an end-of-batch token, allowing for efficient propagation by reusing metadata and reducing redundant data transmission.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If each message is encapsulated in a separate RPC for propagation, then message delivery reliability is ensured, but system throughput and resource efficiency deteriorate
Solution Approach 1:
The patent merges multiple individual RPC calls into a single batched RPC call. Instead of sending n separate RPCs for n messages, the invention encapsulates multiple messages within one RPC structure, allowing the destination queue to process multiple messages in a single operation. This significantly improves throughput while maintaining reliable message delivery through the batched approach.
2Ease of operation
If separate RPCs are used for each message, then message processing simplicity is maintained, but data transmission redundancy increases
Solution Approach 1:
The invention combines multiple message transmissions into a single RPC structure, eliminating redundant metadata transmission. By batching messages together, the overhead of repeated metadata sends is avoided, reducing data transmission redundancy while the destination queue processes each message individually to maintain processing simplicity.
Solution Approach 2:
The single RPC structure serves multiple functions simultaneously - it transports multiple messages, carries shared metadata once, and enables batch processing at the destination. This multi-functional design reduces redundancy while maintaining operational simplicity through a unified processing mechanism.
3Adaptability or versatility
If multiple copies of messages are transmitted to multiple destination queues, then message routing flexibility is achieved, but memory usage and processing overhead increase
Solution Approach 1:
The patent merges multiple message routing operations into a single batched RPC. Instead of sending separate RPCs for each destination queue, the invention includes all destination queue references in one batched structure, allowing the destination to route messages to multiple queues from a single transmission. This reduces memory usage by avoiding duplicate message copies while maintaining full routing flexibility.
Data Source
AI summary
A computer-implemented method of streaming a plurality of messages from a source queue to at least one destination queue over a computer network. The method may include steps of generating a remote procedure call, the remote procedure being configured to encapsulate at least header data, message data for each of the plurality of messages and an end of batch token, the header data being common to each of the plurality of messages and including an identification of the at least one destination queue, the end of batch token signaling that no further message data follows; sending the generated remote procedure call over the computer network to the destination queue identified in the header data, and dequeueing at least the message data for each of the plurality of messages from the source queue and streaming the dequeued messages over the network, and successively enqueueing each streamed message data into the at least one destination queue identified by the header data until the end of batch token is received.