Origin Messaging Module Minimizes Buffering in Data Transfer
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data transfer methods in parallel computing networks require significant buffering, leading to increased overhead and reduced transmission speed due to the need for temporary buffers to handle complex data types between nodes, which hampers efficient communication.
Innovation Solution
The method involves an origin messaging module that receives storage identifiers and complex data types, creates metadata to select and transmit subsets of data directly to target nodes without buffering, using the complex target data type to store the data in the target application storage, thereby eliminating the need for temporary buffers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If temporary buffers are used to handle complex data types during data transfer between nodes, then data transfer reliability is improved, but transmission speed decreases and overhead increases
Solution Approach 1:
The patent extracts the data type interpretation function from the target node and relocates it to the origin node. The origin node packs complex data types into a standardized format before transmission, eliminating the need for temporary buffers at the target node that would otherwise be required for data type handling. This extraction resolves the contradiction by maintaining reliability through proper data encoding while removing the speed-reducing buffering overhead.
Solution Approach 2:
The patent applies preliminary action by performing data type packing and validation at the origin node before data transmission begins. The origin messaging module prepares the data in the correct format advance, so that the target node can directly write to application storage without intermediate buffering. This preliminary preparation ensures data integrity while eliminating transmission delays.
2Reliability
If temporary buffers are used for data transfer between nodes, then data integrity is maintained, but latency increases
Solution Approach 1:
The patent removes the buffering step from the data transfer path by extracting the data type handling function to the origin node. Data is packed into a standardized format at the origin and written directly to the target node's application storage, eliminating the time-consuming buffer copy operation while maintaining data integrity through proper encoding.
Solution Approach 2:
The patent introduces a standardized data format as an intermediary representation between the origin and target nodes. This standardized format acts as a universal mediator that eliminates the need for node-specific buffering, allowing direct transmission while preserving data integrity through consistent encoding rules.
3Reliability
If message buffers are used in data communications networks, then communication reliability is improved, but device complexity and memory requirements increase
Solution Approach 1:
The patent extracts the buffering function from the network infrastructure and relocates it to the application layer at the origin node. By packing data into a standardized format before transmission, the system eliminates the need for complex message buffers in the network, reducing device complexity while maintaining communication reliability through proper data encoding.
Solution Approach 2:
The patent creates a universal data format that can be transmitted across different network configurations without requiring node-specific buffering infrastructure. This standardized format provides multi-functionality, working reliably across various network topologies and data types without increasing device complexity.
Data Source
AI summary
Methods, apparatus, and products for minimally buffered data transfers between nodes in a data communications network are disclosed that include: receiving, by a messaging module on an origin node, a storage identifier, a origin data type, and a target data type, the storage identifier specifying application storage containing data, the origin data type describing a data subset contained in the origin application storage, the target data type describing an arrangement of the data subset in application storage on a target node; creating, by the messaging module, origin metadata describing the origin data type; selecting, by the messaging module from the origin application storage in dependence upon the origin metadata and the storage identifier, the data subset; and transmitting, by the messaging module to the target node, the selected data subset for storing in the target application storage in dependence upon the target data type without temporarily buffering the data subset.