Distributed Sensor Data Sampling for High-Value ML Training
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Machine learning models trained on large datasets of sensor data from distributed networks face diminished value when additional data similar to existing data is added, leading to inefficient use of computational resources for transmission, processing, and storage.
Innovation Solution
Implement distributed data sampling by receiving a sampling target, generating sampled data, determining its value based on similarity and cost, and deciding whether to provide it to a remotely disposed computing device, thereby selecting and transmitting only high-value, less common data scenarios.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If additional training data is added to the corpus, then the quantity of training data increases, but the overall value to the training process diminishes when the data is similar to existing data
Solution Approach 1:
The patent applies local quality by evaluating each data sample individually to determine its unique value based on similarity metrics. Instead of treating all data uniformly, the system assigns different weights to different samples based on their local characteristics - specifically, how similar they are to existing training data. This allows the system to prioritize transmission of high-value, unique samples while filtering out redundant ones.
Solution Approach 2:
The system changes the parameter of data selection from simple quantity-based sampling to value-based sampling using similarity metrics. By introducing a similarity parameter that compares new samples against existing training data, the system dynamically adjusts which samples are selected for transmission based on their informational value rather than random or uniform selection.
2Loss of information
If all sampled data is transmitted to the remotely disposed computing device, then the training data coverage is enhanced, but the computational and storage burdens increase
Solution Approach 1:
The patent extracts only the high-value subset of sampled data for transmission to the centralized computing device. By applying similarity metrics locally at edge devices, the system identifies and extracts only those samples that provide unique training value, leaving redundant samples to be processed or discarded locally. This extraction principle significantly reduces the volume of data transmitted and stored centrally while maintaining training effectiveness.
Solution Approach 2:
The system segments the data transmission task by dividing it into local preprocessing at edge devices and centralized training at remote devices. Each edge device independently evaluates its sampled data against local similarity metrics, segments the data into high-value and low-value portions, and transmits only the high-value segments. This segmentation distributes computational burden and reduces network traffic.
3Ease of operation
If data sampling is performed at distributed devices, then the data selection can be based on local similarity metrics, but the coordination with centralized training requires efficient communication protocols
Solution Approach 1:
The patent applies preliminary action by performing data evaluation and selection at distributed edge devices before centralized training occurs. Edge devices pre-compute similarity metrics against their local training corpora, pre-select high-value samples, and pre-transmit them to centralized devices. This preliminary local processing simplifies the coordination required, as centralized devices receive pre-filtered, high-value data ready for training without needing to manage complex real-time selection protocols.
Data Source
AI summary
Distributed data sampling, including: receiving a sampling target; generating, based on one or more sensors, sampled data; determining, based on the sampling target, a value for the sampled data; and determining, based on the value for the sampled data, whether to provide the sampled data to a remotely disposed computing device.