Time Series Data Sufficiency Determination via Probability Distribution Convergence
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data sufficiency determination devices for learning models struggle to accurately assess the sufficiency of time series data, particularly due to their inability to flexibly handle data with various characteristics, leading to low accuracy in determining data sufficiency.
Innovation Solution
A data amount sufficiency determination device that acquires time series data, divides it into substring data sets, calculates feature amounts, generates probability distributions, and determines convergence of these distributions to assess data sufficiency, allowing for more accurate determination beyond just pattern numbers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If learning is performed with a large amount of data, then learning accuracy is improved, but learning time increases and over-learning may occur
Solution Approach 1:
The patent implements a feedback mechanism by continuously monitoring the probability distribution of feature amounts during the learning process. The system determines whether to continue collecting data or proceed with learning based on whether the probability distribution has converged, creating a closed-loop control system that adapts the data collection duration to actual learning needs.
Solution Approach 2:
The patent changes the parameter used for determining data sufficiency from simple pattern counting to probability distribution analysis. By using probability distribution convergence as the stopping criterion, the system can more accurately determine when sufficient data has been collected, avoiding both premature termination and excessive data collection.
2Loss of time
If learning is performed with insufficient data, then learning time is reduced, but determination accuracy of data sufficiency is low
Solution Approach 1:
The patent replaces the mechanical approach of counting feature patterns with a probabilistic model that analyzes the distribution of feature amounts. This substitution from discrete pattern counting to continuous probability distribution analysis provides a more accurate and nuanced assessment of data sufficiency.
3Device complexity
If simple pattern counting is used to determine data sufficiency, then device complexity is reduced, but adaptability to various time series data characteristics is poor
Solution Approach 1:
The patent creates a universal determination method based on probability distribution analysis that can handle various types of time series data with different characteristics. The approach is not tied to specific data patterns or domains, making it broadly applicable across different applications while maintaining a relatively simple implementation framework.
Data Source
AI summary
Provided is a data amount sufficiency determination device capable of determining the sufficiency of the data amount of learning data with higher accuracy.A data amount sufficiency determination device according to the present disclosure includes a time series data acquisition unit to acquire time series data, a data division unit to divide the time series data into a plurality of pieces of substring data, a data set generation unit to generate a plurality of substring data sets that are sets of substring data, a feature amount calculation unit to calculate a feature amount of the substring data, a probability distribution generation unit to generate probability distribution of the feature amount for each substring data sets, and a determination unit to determine whether or not the probability distribution has converged.