MPI Tag-Matching Architecture for Multi-Domain Resource Utilization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current MPI communication protocols face optimization challenges due to unbounded resource requirements, particularly in supporting wildcard sources or tags, which limits multi-domain tag-matching implementations and leads to inefficient resource utilization in network traffic handling.
Innovation Solution
The proposed solution involves a tag-matching architecture that utilizes multiple tag-matching engines concurrently, allowing systems with multiple host channel adapters and software matching engines to exploit all resources simultaneously, rather than selecting one and idling the others, by establishing lists of expected and unexpected messages in both master and client domains, and employing shared memory for interprocess communication.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If a single matching engine is used for all data destined to a given target, then resource requirements are bounded, but resource utilization efficiency deteriorates as other engines remain idle
Solution Approach 1:
The patent segments the tag-matching function by creating multiple matching engines (first matching engine, second matching engine, etc.) that can operate concurrently. Each engine handles specific data streams or communication domains, allowing parallel processing of MPI messages without requiring a single engine to handle all matching operations, thus improving resource utilization while maintaining manageable complexity through modular architecture
Solution Approach 2:
The patent introduces a new dimension of parallelism by deploying matching engines across multiple communication domains or host channel adapters. Instead of using one engine sequentially for all data, the system utilizes multiple engines simultaneously operating in parallel domains, transforming the single-threaded matching approach into a multi-dimensional parallel architecture that improves throughput and resource utilization
2Productivity
If multiple host channel adapters are used, then resource utilization improves, but system complexity increases
Solution Approach 1:
The patent implements a universal tag-matching architecture where multiple matching engines follow the same standardized interface and processing logic. Each engine is multi-functional, capable of handling different types of MPI communication patterns (point-to-point, collective operations, wildcard matching) through a unified mechanism. This universality allows the system to scale to multiple host channel adapters without proportionally increasing complexity, as each additional engine replicates the proven single-engine design
Solution Approach 2:
The patent employs copying by replicating the successful single-engine design across multiple instances. Instead of creating a fundamentally new complex system for multi-engine operation, the solution copies the validated single-engine architecture multiple times and orchestrates them in parallel. Each matching engine is an independent copy that can be deployed on separate host channel adapters, reducing the complexity burden of scaling by reusing proven design patterns
3Adaptability or versatility
If wildcard sources or tags are supported, then message routing flexibility improves, but resource requirements become unbounded
Solution Approach 1:
The patent applies preliminary action by pre-processing and categorizing wildcard matching requirements before actual message routing occurs. The system performs advance setup of wildcard patterns, pre-computes matching criteria, and prepares routing tables in advance. This preliminary preparation allows the system to support flexible wildcard sources and tags without requiring unbounded resources during runtime, as the complex matching logic is resolved beforehand when resources can be allocated efficiently
Solution Approach 2:
The patent introduces intermediary structures such as pattern-matching tables, routing caches, and intermediate buffer zones that mediate between wildcard messages and destination buffers. These intermediaries absorb the complexity of wildcard matching by providing structured lookup mechanisms and temporary storage, preventing the system from requiring unbounded direct resources for every possible wildcard combination. The intermediaries act as buffers that manage resource consumption while maintaining routing flexibility
Data Source
AI summary
Network communication is carried out by transmitting messages in accordance with a predefined data exchange protocol among nodes that include a master domain and a plurality of client domains. A list of expected messages has a tail portion in the master domain and respective head portions in the client domains. A search is conducted for a match between the tag of a received message to tags in a list of unexpected messages that is maintained in the master domain. Upon a failure to find the match the receive is added to the list of expected messages. If a match is found then data in the message is written into a data buffer.