Parallel Processing of Distributed Arrays via Optimum Data Distribution
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current architectures lack a single language presentation for parallel processing of distributed arrays, forcing them to be processed either entirely in parallel or sequentially, which limits efficiency in utilizing closely-coupled processors and hardware resources.
Innovation Solution
The implementation provides a single programming language that identifies and transforms programs into parallel programs with an optimum data distribution scheme, allocating portions to multiple labs for parallel execution and combining results, enabling efficient parallel processing of distributed arrays across multiple resources.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If distributed arrays are processed exclusively in parallel or sequentially, then processing simplicity is maintained, but resource utilization efficiency deteriorates
Solution Approach 1:
The system dynamically switches between sequential and parallel processing modes based on the characteristics of the distributed array and available hardware resources. The controller determines whether to process arrays sequentially or in parallel, and can adjust the degree of parallelization dynamically, allowing the architecture to adapt to different workloads while maintaining simplicity in the programming model.
Solution Approach 2:
The system changes processing parameters such as the number of parallel processors involved, the distribution strategy, and the synchronization frequency based on array size, data locality requirements, and resource availability. This allows optimal resource utilization without requiring the programmer to manually configure complex parallel processing details.
2Ease of operation
If a single language presentation is provided for parallel processing, then ease of operation is improved, but system complexity increases
Solution Approach 1:
A controller acts as an intermediary between the single programming language and the parallel processing hardware. The controller translates high-level array operations into appropriate parallel or sequential execution plans, manages data distribution across processors, and handles synchronization automatically. This shields the programmer from complexity while enabling efficient parallel processing when beneficial.
Solution Approach 2:
The single programming language provides universal constructs that can express both sequential and parallel processing needs. Array operations, loops, and conditional statements in the language can automatically leverage parallel hardware resources when appropriate, while maintaining the same syntax and semantics as traditional sequential programming, thus simplifying the programming model.
3Productivity
If distributed arrays are partitioned into segments, then parallel processing capability is improved, but data distribution optimization becomes more difficult
Solution Approach 1:
The system automatically segments distributed arrays into appropriate partitions for parallel processing based on data access patterns, processor capabilities, and communication overhead considerations. The controller determines optimal segmentation strategies, such as block distribution, cyclic distribution, or hierarchical distribution, without requiring manual intervention from the programmer.
Solution Approach 2:
The system performs preliminary analysis of data access patterns and computational requirements before executing parallel processing operations. The controller pre-determines the optimal data distribution scheme and prepares data partitions in advance, optimizing communication efficiency and load balancing before the actual parallel computation begins.
Data Source
AI summary
A computing device-implemented method includes initiating a single programming language, and identifying, via the single programming language, one or more data distribution schemes for executing a program. The method also includes transforming, via the single programming language, the program into a parallel program with an optimum data distribution scheme selected from the one or more identified data distribution schemes, and allocating the parallel program to two or more labs for parallel execution. The method further includes receiving one or more results associated with the parallel execution of the parallel program from the two or more labs, and providing the one or more results to the program.