MPI Shared Memory Optimization for SPMD Data Redundancy
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In high performance computing using Message Passing Interface (MPI) on shared memory systems, conventional MPI standards lead to redundant data transfer between processes due to complete data copying, which hampers memory bandwidth and cache performance, especially in patterns like single program multiple data (SPMD) where multiple copies of data are maintained.
Innovation Solution
An MPI program optimizer comprising an analyzer, retriever, annotator, library, and parser optimizes data communication by identifying sharable data, annotating it with directives, allocating it in shared memory, and transforming communication patterns to a single-data-copy-access pattern, reducing redundant data transfer and improving cache performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional MPI standards are used for data communication between processes, then data transfer can be achieved, but redundant data copying occurs which hampers memory bandwidth and cache performance
Solution Approach 1:
The patent merges multiple data copies into a single shared memory copy that can be accessed by multiple processes simultaneously. Instead of maintaining separate copies of the same data in different process address spaces, the system consolidates the data into one location in shared memory, eliminating redundancy while maintaining accessibility for all processes that need it.
Solution Approach 2:
The patent replaces physical data copying with virtual address mapping. Rather than actually copying data between process address spaces, the system creates virtual mappings that allow processes to access the same physical memory location, achieving the effect of data sharing without the overhead of actual data duplication and transfer.
2Ease of operation
If multiple copies of data are maintained in SPMD pattern, then each process has independent data access, but cache performance deteriorates due to redundant data storage
Solution Approach 1:
The patent merges multiple process-specific data copies into a single shared memory residence. Multiple processes can access the same data structure in shared memory through their respective address spaces, maintaining operational independence while physically consolidating the data to reduce cache memory consumption and eliminate redundant storage.
3Reliability
If complete data copying is performed between processes, then data integrity is ensured, but communication overhead increases and performance decreases
Solution Approach 1:
The patent replaces actual data copying with virtual address mapping mechanisms. Processes access shared data through mapped addresses in their address spaces that point to the same physical memory location, ensuring data integrity through shared access while eliminating the time-consuming copy operations that characterize traditional MPI implementations.
Data Source
AI summary
Machine readable media, methods, apparatus and system for MPI program optimization. In some embodiments, shared data may be retrieved from a message passing interface (MPI) program, wherein the shared data is sharable by a plurality of processes. Then, the shared data may be allocated to a shared memory, wherein the shared memory is accessible by the plurality of processes. A single copy of the shared data may be maintained in the shared data in a global buffer of the processes of the plurality of processes can read or write the single copy of the shared data from or to the shared memory.