Selective Harvesting for Edge Dataset Building in Partial Networks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Building a robust dataset for machine learning models is challenging due to the difficulty in finding edge devices with similar characteristics in partially observed networks, where querying the entire network is infeasible and costly, and traditional graph search algorithms are unsuited for partially observed scenarios.
Innovation Solution
Model a distributed edge scenario as a partially observed graph, apply a selective harvesting (SH) algorithm to identify and select devices with specified characteristics, and use a pipeline to manage distributed edge devices for dataset construction.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional graph search algorithms are used to find edge devices in a network, then complete network exploration can be achieved, but the querying cost becomes prohibitively high and the process becomes impractical for large-scale networks
Solution Approach 1:
The patent applies partial action by implementing selective harvesting that queries only a subset of network nodes rather than the entire network. The system identifies and queries only those nodes that are relevant to the target domain characteristics, leaving other nodes unqueried. This resolves the contradiction by achieving sufficient device discovery completeness through partial network exploration, thereby reducing querying costs while maintaining practical feasibility for large-scale networks.
Solution Approach 2:
The patent changes the parameter of network exploration from exhaustive to selective by introducing domain-specific filtering criteria. The system modifies the querying process to prioritize nodes based on their relevance to target domain characteristics, transforming the uniform graph search into a targeted exploration strategy. This parameter change enables the system to achieve adequate discovery completeness with significantly reduced querying costs.
2Reliability
If the entire network is queried to find devices with similar characteristics, then a comprehensive dataset can be built, but the time and resources required become unfeasible
Solution Approach 1:
The patent applies preliminary action by pre-training the selective harvesting algorithm on available node features before executing the full network search. The system uses initial feature data to learn and prioritize promising search directions, preparing the model in advance to efficiently identify relevant devices. This preliminary preparation enables the system to construct robust datasets with similar characteristics while significantly reducing the time and resources required compared to exhaustive network querying.
Solution Approach 2:
The patent extracts only the essential node features required for domain similarity assessment, rather than querying all possible node attributes. By selectively extracting relevant features and using them to guide the search process, the system builds comprehensive datasets with similar characteristics while minimizing the time and resources spent on data collection. This extraction approach maintains dataset robustness by focusing on the most informative features.
3Loss of energy
If a limited number of queries are made to reduce costs, then resource consumption decreases, but the ability to find representative devices is compromised
Solution Approach 1:
The patent implements feedback by using the results from each query to inform and adjust subsequent querying decisions. The selective harvesting algorithm learns from observed node features and dynamically updates its search strategy, prioritizing nodes that are more likely to contain relevant domain characteristics. This feedback mechanism ensures that even with a limited number of queries, the system maximizes domain feature coverage by intelligently allocating query resources to the most promising areas of the network.
Solution Approach 2:
The patent changes the querying strategy from uniform to adaptive by introducing domain-specific parameter filtering. The system adjusts query parameters based on learned patterns from initial observations, focusing exploration on regions of the network most likely to contain devices with target characteristics. This parameter adaptation enables the system to maintain comprehensive domain feature coverage while operating within limited query budgets, resolving the contradiction between resource consumption and information coverage.
Data Source
AI summary
One example method includes receiving respective sets of node features from each edge node in a set of edge nodes of a network, identifying edge nodes in the set of edge nodes that contain datapoints corresponding to a specified class, using the datapoints to train an SH model, applying the trained SH model to the network, collecting datapoints from edge nodes in the specified class that were identified by the applying of the SH model to the network, when a threshold number of the edge nodes in the specified class has been identified by application of the SH model, collecting respective data points and features from each of those edge nodes of the specified class, and building a final dataset that comprises the edge nodes of the specified class, and their associated data points and features, that were identified by application of the SH model to the network.