Data Subset Distance Estimation for Transfer Learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for calculating distances between data sets in transfer learning require using all data, leading to excessive processing loads.
Innovation Solution
An information processing device that acquires partial data subsets and their corresponding distances from both data sets, calculates a third distance between these subsets, and generates estimation distance information to approximate the full data set distance, reducing the processing load.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If all data from both data sets are used to calculate distance, then measurement precision is improved, but processing time and computational load increase excessively
Solution Approach 1:
The patent divides the complete data sets into multiple subsets and selects representative subsets for distance calculation. Instead of processing all data points, the method segments the data and uses only necessary portions (subsets) to compute distances, thereby reducing computational load while maintaining acceptable accuracy for transfer learning applications.
Solution Approach 2:
The patent extracts essential features or representative samples from the complete data sets to form subsets used for distance calculation. By taking out only the necessary information needed to estimate data set similarity, the method avoids processing redundant data, thus reducing processing time while preserving the core information needed for accurate distance estimation.
2Measurement precision
If all data from both data sets are used to calculate distance, then measurement precision is improved, but computational resources and complexity increase
Solution Approach 1:
The patent segments the large-scale data processing task into smaller sub-tasks involving subsets. This segmentation reduces the computational complexity of distance calculation by breaking down the problem into manageable pieces that can be processed with fewer computational resources, while still providing sufficient accuracy for determining data set similarity in transfer learning contexts.
Solution Approach 2:
The patent applies partial action by using only a portion (subset) of the available data rather than the complete data sets. This partial processing approach reduces computational complexity and resource requirements while maintaining the essential function of distance calculation needed for transfer learning, accepting that some data is not processed but the overall task is still accomplished effectively.
3Speed
If partial data subsets are used for distance calculation, then processing speed is improved, but measurement precision may deteriorate
Solution Approach 1:
The patent extracts representative subsets from the complete data sets that capture the essential characteristics needed for distance calculation. By carefully selecting subsets that contain the most relevant information, the method maintains measurement precision while achieving faster processing speeds, as the extracted subsets are sufficient to estimate data set similarity without requiring all data points.
Data Source
AI summary
In an information processing device, an information acquisition means acquires a first data subset created by extracting a partial data group included in a first data set, a first distance corresponding to a distance between the first data subset and the first data set, a second data subset created by extracting a partial data group included in a second data set, and a second distance corresponding to a distance between the second data subset and the second data set. An information generation means calculates a third distance corresponding to a distance between the first data subset and the second data subset, and to generate an estimation distance information which is information capable of estimating a fourth distance corresponding to a distance between the first data set and the second data set based on the first distance, the second distance, and the third distance.