Similarity-Based Dataset Selection for Wireless ML Data Transfer
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The high cost of data transfer in wireless networks for machine learning (ML) model training due to the need to retrieve large datasets, especially when datasets are highly correlated, leading to unnecessary data redundancy and increased bandwidth usage.
Innovation Solution
Implementing a method to determine dataset similarity using a data similarity metric to select a subset of datasets for transmission, reducing redundant data transfer by using a cost function to identify less similar data sources.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If all available datasets are transferred to the network node for ML model training, then the model accuracy is improved, but the data transfer cost and bandwidth usage increase significantly
Solution Approach 1:
The patent extracts only the essential and non-redundant datasets from the available data sources. By calculating similarity scores between datasets and selecting only those that provide unique information, the system transfers a reduced subset of datasets to the network node, maintaining model training effectiveness while significantly reducing data transfer costs and bandwidth consumption.
2Reliability
If datasets from multiple highly correlated data sources are retrieved, then the model training data quality is improved, but the redundancy increases and transfer cost increases
Solution Approach 1:
The patent changes the selection parameter from treating all datasets equally to using similarity-based selection. By calculating similarity scores between datasets and applying a selection criterion that prefers datasets with lower similarity to already selected datasets, the system identifies and retrieves only the most informative datasets, reducing redundancy while maintaining training data quality.
3Loss of energy
If similarity-based dataset selection is implemented, then the data transfer overhead is reduced, but the complexity of dataset selection process increases
Solution Approach 1:
The patent performs preliminary similarity calculations and dataset selection before the actual data transfer process. By pre-computing similarity scores between datasets and determining the optimal subset to transfer in advance, the system reduces the complexity during the actual training phase. The selection logic is executed once beforehand, simplifying the overall process while achieving significant reduction in transfer overhead.
Data Source
AI summary
Methods and systems for reducing data sharing overhead based on similarities of datasets. In one aspect, there is a computer implemented method (1200) for reducing the amount of data transmitted to a network node (102) of a wireless communication system (100). The network node is configured to use the data to create or modify a model. The method is performed by the network node and comprises obtaining (s1202) information identifying a set of available datasets (152-158) and obtaining (s1204) a set of similarity scores. The method also comprises based on the obtained similarity scores, selecting (s1206) a subset of the set of available sets. The method further comprises transmitting (s1208) a request for each dataset included in the subset, receiving (s1210) the requested datasets, and using (s1212) the received datasets to construct or modify a model.