Heterogeneous Data Processing System with Actor Network Topology
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current deep learning architectures face inefficiencies in data processing due to high communication overhead and complex data routing, especially in large-scale models, where existing parallel processing methods like data parallelism, model parallelism, and hybrid parallelism do not adequately address the challenges of data movement and resource utilization in heterogeneous architectures.
Innovation Solution
A data processing system for a heterogeneous architecture that decomposes jobs into tasks, generates a task topology, and creates actors to process data efficiently, with a focus on data movement and routing, utilizing a dedicated actor network to fragment and process job data continuously, thereby reducing the need for manual scheduling and improving data movement efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data parallelism is used to process large-scale deep learning models, then processing speed is improved, but communication overhead increases significantly
Solution Approach 1:
The patent segments the deep learning model into multiple operators and organizes them into a computation graph. Each operator can be independently processed and optimized, allowing the system to divide the large model into manageable units that can be distributed across multiple devices while reducing the communication overhead between them.
Solution Approach 2:
The patent introduces a computation graph as an intermediary structure that represents the model's operators and their relationships. This computation graph enables efficient scheduling and optimization of data flow between devices, reducing communication overhead while maintaining processing speed.
2Adaptability or versatility
If model size is increased to handle larger datasets, then model capacity is improved, but device memory capacity is exceeded
Solution Approach 1:
The patent segments the model parameters and computation across multiple devices using the segmented data parallel mode. Each device holds only a portion of the model parameters and processes a specific subset of operators, allowing the system to handle models that exceed the memory capacity of individual devices while maintaining high model capacity.
Solution Approach 2:
The patent introduces a new dimension of parallelism by organizing operators into a computation graph structure. This allows the system to distribute computation not only across devices (spatial dimension) but also across different operator stages (temporal dimension), effectively increasing the system's capacity to handle large models without requiring proportional increases in individual device memory.
3Ease of operation
If manual scheduling is used for data parallel or model parallel processing, then processing control is improved, but programmer workload and error rate increase
Solution Approach 1:
The patent implements self-service mechanisms where the system automatically schedules and optimizes the execution of operators based on the computation graph structure. The framework automatically handles data distribution, device assignment, and synchronization, eliminating the need for manual scheduling while maintaining efficient processing control and reducing programmer workload.
Solution Approach 2:
The patent performs preliminary analysis of the computation graph to automatically determine the optimal execution plan before actual processing begins. This preliminary action includes identifying critical paths, optimizing data flow, and pre-assigning operators to devices, which simplifies the programming process while ensuring efficient execution during runtime.
Data Source
AI summary
Provided is a data processing system for a heterogeneous architecture, including, a job decomposing component, configured to decompose a to-be-completed job into a series of tasks executed by an execution subject in the heterogeneous architecture; a task topology generating component, configured to generate a task relationship topology based on an inherent relationship between the decomposed tasks during the job decomposition, where a task node of the task topology includes all node attributes required to execute a corresponding task; an execution subject creating component, configured to create a corresponding execution subject for each task in a computing resource based on the task relationship topology; and an execution subject network component, configured to include one or more data processing paths including various created execution subjects, and fragment actual job data into task data when receiving the actual job data.