Distributed Hyperparameter Search for Low-Latency SON Edge Learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine learning (ML) systems face challenges in hyperparameter tuning due to high signaling/latency costs, complex hyperparameter search spaces, and scarcity of training data, which hinder efficient implementation in self-organizing networks (SONs) like 5G communication networks.
Innovation Solution
A method involving a hyperparameter server and edge devices that extract distribution features from local training data, train a shared hyperparameter ML model, and use heuristic search to identify optimal hyperparameters, reducing the need for large data transfers and minimizing latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional centralized hyperparameter search is used, then comprehensive model training can be achieved, but high signaling costs and latency occur due to large data transfers between edge devices and central server
Solution Approach 1:
The patent implements local hyperparameter search at edge devices using distributed machine learning. Each edge device maintains local training data and performs hyperparameter optimization locally, extracting only necessary performance metrics and sharing minimal information with the central server. This local execution eliminates the need for large-scale data transfers, reducing latency while maintaining search accuracy through distributed collaborative optimization.
2Measurement precision
If exhaustive hyperparameter search is performed, then optimal hyperparameters can be identified, but the complex search space requires excessive computational resources and time
Solution Approach 1:
The patent segments the hyperparameter search space into multiple distributed search tasks executed across different edge devices. Each device handles a portion of the search space using local computational resources, dividing the exhaustive search into manageable segments. This segmentation enables parallel processing of hyperparameter combinations, reducing overall search time while maintaining comprehensive coverage of the search space through coordinated exploration.
Solution Approach 2:
The patent implements partial hyperparameter search by focusing on the most promising regions of the search space identified through distributed exploration. Rather than exhaustively evaluating all possible hyperparameter combinations, the system uses local performance feedback to guide selective exploration of critical parameter subsets, achieving sufficient optimization accuracy with reduced computational effort and improved search efficiency.
3Productivity
If local training data is used at edge devices, then training speed improves, but data scarcity limits model performance
Solution Approach 1:
The patent merges local training experiences across multiple edge devices through distributed collaborative learning. Each device trains models locally using its own data, then shares performance metrics and hyperparameter optimization results with the central server and other devices. This merging of distributed training outcomes effectively combines the benefits of local data utilization with collective knowledge aggregation, improving model performance despite individual data scarcity while maintaining fast local training speeds.
Data Source
AI summary
A method of a hyperparameter server improves hyper-parameter search efficiency for devices in a self-organizing network (SON) includes sending configuration for data feature collection to at least one edge device in the self-organizing network, receiving hyper-parameter performance data from the at least one edge device, and training a shared hyperparameter machine learning model using a global training database including the hyperparameter performance data to identify optimal hyperparameters for use by the at least one edge device. A further method of an edge device improves hyperparameter search efficiency for devices in a SON includes receiving configuration for data feature collection from a hyperparameter server, training an edge machine learning model using local training data and selected hyperparameters, and sending performance data to the hyperparameter server obtained from the training of the edge machine learning model.