Agile Communication Operator for Data Parallel Task Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing computer systems face inefficiencies in executing data parallel algorithms due to general purpose processors lacking optimized hardware resources, making it complex to manage and distribute large datasets across multiple computing platforms.
Innovation Solution
A high-level programming language with an agile communication operator that decomposes computational space into segments, assigns them to compute nodes, and allows central management and automation of segment movement, using either a full global-view or local-global-view representation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If general purpose processors are used to execute data parallel algorithms, then programming flexibility is maintained, but execution efficiency deteriorates due to lack of optimized hardware resources
Solution Approach 1:
The patent introduces a hybrid processing architecture that acts as an intermediary between general purpose processors and specialized data parallel compute nodes. The system automatically partitions computational tasks, with the general purpose processor handling control logic and the specialized compute nodes executing data parallel operations, thus resolving the contradiction between programming flexibility and execution efficiency.
2Productivity
If data parallel compute nodes are enhanced with more processing capabilities, then data parallel processing performance is improved, but programming complexity increases
Solution Approach 1:
The patent implements automatic task partitioning and resource allocation mechanisms where the system itself manages the complexity of programming data parallel compute nodes. The hybrid architecture automatically divides computational work between general purpose processors and specialized nodes, eliminating the need for programmers to manually manage the complexity while still utilizing enhanced processing capabilities.
3Productivity
If large datasets are distributed across multiple computing platforms, then processing capacity is increased, but data management complexity increases
Solution Approach 1:
The patent applies segmentation by dividing large datasets into smaller partitions that can be independently managed and processed across multiple computing platforms. The hybrid architecture automatically manages data distribution and movement between platforms, reducing data management complexity while maintaining increased processing capacity through parallel processing of data segments.
Data Source
Figure 1~2
Figure 3A~3C
Figure 4
AI summary
A high level programming language provides an agile communication operator that generates a segmented computational space based on a resource map for distributing the computational space across compute nodes. The agile communication operator decomposes the computational space into segments, causes the segments to be assigned to compute nodes, and allows the user to centrally manage and automate movement of the segments between the compute nodes. The segment movement may be managed using either a full global-view representation or a local-global-view representation of the segments.