Distributed ANN Training via Edge Device Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Training large artificial neural networks (ANNs) is resource-intensive, limiting the ability to scale due to the high computational and memory requirements, especially when using centralized server farms.
Innovation Solution
Utilizing edge devices for training ANNs through federated learning, where a central server divides the task graph and distributes portions of the ANN model to edge devices, allowing them to perform local training and share activation signals, thereby leveraging their local computational resources in parallel.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Power
If centralized server farms are used to train large ANNs, then computational power and memory resources are concentrated, but the system becomes less scalable and more resource-intensive
Solution Approach 1:
The patent divides the ANN training task into segments by partitioning the computational graph into multiple stages. Each stage is executed on different devices (edge devices for early stages, cloud servers for later stages), allowing the system to scale by adding more edge devices without requiring a single large centralized cluster. This segmentation enables flexible resource allocation across the distributed network.
Solution Approach 2:
The patent introduces a spatial dimension to training by executing different portions of the computational graph at different locations (edge devices and cloud servers simultaneously). This multi-location execution model adds a spatial dimension to the traditional sequential training process, enabling parallel processing across distributed devices and improving overall scalability.
2Productivity
If centralized server farms train large ANNs, then training can be performed, but resource consumption increases significantly
Solution Approach 1:
The training workload is segmented across multiple edge devices and cloud servers, with each device performing only a specific portion of the computational graph. This segmentation reduces the resource consumption at any single location while maintaining overall training productivity through parallel execution across multiple devices.
Solution Approach 2:
Edge devices perform local training computations using their own computational resources rather than requiring constant centralized processing power. This self-service capability allows edge devices to contribute to training independently, reducing the overall resource consumption burden on centralized server farms while maintaining training productivity.
3Adaptability or versatility
If edge devices are used for training, then resource distribution is improved, but coordination complexity increases
Solution Approach 1:
The computational graph is segmented into distinct stages, each with clear input and output interfaces. This segmentation simplifies coordination by defining clear data flow boundaries between edge devices and cloud servers, reducing the complexity of managing distributed training compared to fully decentralized approaches.
Solution Approach 2:
The patent introduces an intermediary coordination layer that manages the distribution of training tasks and aggregation of results between edge devices and cloud servers. This intermediary layer abstracts the complexity of coordinating multiple devices, providing a simplified interface for resource distribution while maintaining the benefits of distributed training.
Data Source
AI summary
Training the ANN can include providing an initial ANN model to a plurality of groups of edge devices and providing an input to a group of edge devices from the plurality of groups of edge devices. Training the ANN can also include, responsive to providing the input, receiving activation signals from a first portion of the plurality of groups. Training the ANN can include providing the activation signals to a second portion of the plurality of groups and provide commands to the plurality of groups of edge devices to train the initial ANN model to generate a trained ANN model based on training feedback generated using different activation signals received from the second portion of the plurality of groups. Training the ANN can also include receiving the trained ANN model from the plurality of groups of edge devices.