Split-Layer Model Training for Heterogeneous Edge Devices
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional methods for training artificial intelligence models in edge networks either underutilize the powerful computing capabilities of base stations or incur significant latency overhead due to resource constraints and heterogeneous device capabilities.
Innovation Solution
A method for model training that involves splitting the global model into sub-models based on the capabilities of terminal devices, with the network device and terminal devices collaboratively training these sub-models, allowing for adaptive partitioning and asynchronous aggregation to optimize training efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Power
If conventional training methods are used in edge networks, then model training can be performed, but the computing capabilities of base stations are underutilized
Solution Approach 1:
The global model is segmented into multiple sub-models that are distributed to different terminal devices for parallel training. Each terminal device trains a specific sub-model based on its capability, and the base station coordinates the aggregation of these sub-models to form the updated global model. This segmentation enables full utilization of distributed computing resources while maintaining training efficiency.
2Power
If distributed training across heterogeneous devices is implemented, then computing capabilities are better utilized, but significant latency overhead occurs
Solution Approach 1:
The system dynamically adjusts the training process based on real-time device capabilities and training progress. The base station monitors the training status of each terminal device and adaptively manages the aggregation process. Training duration of sub-models is used to determine whether a training round is current, enabling dynamic synchronization that reduces latency while maintaining full resource utilization.
3Adaptability or versatility
If model training is performed on resource-constrained terminal devices, then distributed intelligence is achieved, but training efficiency decreases due to resource limitations
Solution Approach 1:
Each terminal device is assigned a specific sub-model tailored to its local computing capabilities and resource constraints. The base station determines the appropriate split layer and sub-model configuration for each device based on its capability assessment. This local optimization ensures that each device operates at its maximum efficiency while contributing to the overall distributed training objective.
4Power
If the global model is split into sub-models for distributed training, then resource utilization improves, but system complexity increases
Solution Approach 1:
The base station serves as an intermediary that manages the complexity of distributed training. It is responsible for segmenting the global model, assigning sub-models to terminal devices, monitoring training progress, and aggregating results. This centralized coordination layer abstracts the complexity from individual terminal devices while enabling full utilization of distributed computing resources.
Data Source
AI summary
A method for model training, a resource management method for model training, and related devices are provided. One example method for model training includes: receiving a first sub-model from a global model, wherein the first sub-model is determined according to a first split layer; and training the first sub-model; wherein the global model further comprises a second sub-model, and at least one of the following is true: training of the first sub-model and training of the second sub-model are jointly used to determine a first local model, and the first split layer is determined according to a capability of the first terminal device; or training duration of the first sub-model is used to determine whether a training round in which the first local model participates in model aggregation is a current training round