Dynamic Moving Window Clustering for Prediction Model Data Screening
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional methods for building prediction models discard important sample data related to events like equipment drift, leading to inaccurate model predictions when similar events occur again, as they use a static moving window scheme that replaces older data with new data, potentially discarding critical information.
Innovation Solution
A dynamic moving window method that uses clustering techniques, such as ART2 or WED algorithms, to group similar sample data and retain or discard data based on group size, ensuring that important data related to variance is not lost, thereby maintaining prediction accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a static moving window scheme is used to replace oldest sample data with new data, then the model can be continuously refreshed with new data, but important sample data related to equipment drift events will be discarded
Solution Approach 1:
The patent transforms the static moving window approach into a dynamic one by introducing clustering-based mechanisms. The window is no longer fixed in size or replacement rules, but adapts dynamically based on data characteristics. Clustering algorithms identify patterns and determine which data to retain, making the data retention strategy flexible and responsive to actual data importance rather than following a rigid time-based replacement schedule.
Solution Approach 2:
The patent changes the parameters governing data retention from simple time-based criteria to complexity-based criteria involving clustering metrics. Instead of replacing data purely based on age, the system evaluates clustering coefficients, group sizes, and data distribution characteristics. This parameter transformation allows the system to preserve data with high informational value (such as equipment drift events) while still maintaining continuous model refreshing.
2Reliability
If all sample data are retained for model building, then prediction accuracy for rare events can be maintained, but the computational complexity and data processing burden increase
Solution Approach 1:
The patent extracts and retains only the essential data elements needed for accurate prediction by using clustering to identify and preserve data groups with high informational value. Rather than keeping all data indiscriminately, the system extracts specific clusters that represent important patterns (such as equipment drift events) while discarding redundant data. This selective extraction maintains prediction reliability while reducing the overall data volume that needs to be processed.
Solution Approach 2:
The patent applies partial action by retaining only the necessary portion of data required for accurate prediction. Through clustering analysis, the system identifies the minimal sufficient subset of data that captures essential patterns and variability. This partial retention approach avoids the excessive computational burden of processing all available data while still maintaining sufficient information for reliable predictions of both common and rare events.
3Ease of operation
If a fixed number of sample data are kept in the window, then data management is simple, but the model cannot adapt to varying data importance and loses critical information about equipment events
Solution Approach 1:
The patent introduces dynamic adaptation mechanisms that allow the data window to automatically adjust its composition based on the importance and characteristics of incoming data. Clustering algorithms continuously evaluate new data points and determine whether they represent important patterns (such as equipment drift) that warrant retention. This dynamic behavior enables the system to adapt to varying data importance without requiring complex manual management rules, achieving both adaptability and operational simplicity.
Data Source
AI summary
A method for screening samples for building a prediction model and a computer program product thereof are provided. When a set of new sample data is added to a dynamic moving window (DMW), a clustering step is performed with respect to all of the sets of sample data within the window for grouping the sets of sample data with similar properties as one group. If the number of the sets of sample data in the largest group is greater than a predetermined threshold, it means that there are too many sets of sample data with similar properties in the largest group, and the oldest sample data in the largest group can be deleted; if smaller than or equal to a predetermined threshold, it means that the sample data in the largest group are quite unique, and should be kept for building or refreshing the prediction model.