Data Stream Clustering Starting Model Stabilization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data stream clustering methods experience severe jitter in results due to temporary changes in business situations, leading to instability and inefficiency in business decision-making and process control.
Innovation Solution
A method that determines a starting model for current data partitions by using weighted representative parameters from preceding data partitions, allowing historical data to influence clustering and stabilize results while capturing short-term changes, using algorithms like k-means or Gaussian mixture models.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If clustering is performed on each data segment independently, then the clustering algorithm can process current data quickly, but the clustering results will change severely due to temporary business situation changes, causing severe jitters and instability
Solution Approach 1:
The patent applies preliminary action by using clustering results from historical data segments to pre-establish a starting model for current data segmentation. This starting model serves as a preliminary clustering framework that guides the clustering of current data, avoiding the need to start clustering from scratch and reducing the impact of temporary fluctuations in current data on the final results.
Solution Approach 2:
The patent implements feedback by using the clustering results of current data segmentation to update and refine the starting model for the next data segment. This creates a continuous feedback loop where historical clustering results inform future clustering, smoothing out temporary variations and maintaining stability while still adapting to genuine changes in the data stream.
2Loss of time
If clustering results are updated frequently to reflect current business situations, then the timeliness of business decision-making is improved, but the clustering results become unstable and jump frequently, affecting business stability
Solution Approach 1:
By establishing a starting model based on historical clustering results before processing current data, the patent enables timely clustering of current data segments while maintaining stability. The preliminary model provides a stable foundation that allows quick processing of current data without causing severe jumps in results, thus achieving both timeliness and stability.
Solution Approach 2:
The patent applies parameter changes by dynamically adjusting the starting model parameters based on the aggregation of historical clustering results. This allows the system to adapt to genuine changes in business situations while filtering out temporary fluctuations, achieving a balance between timeliness of response and stability of results.
3Device complexity
If a fixed number of categories is used in clustering, then the clustering process is simpler and faster, but it cannot adapt to dynamic changes in data distribution over time
Solution Approach 1:
The patent applies dynamics by making the number of categories in the starting model dynamic rather than fixed. The starting model is constructed by aggregating clustering results from multiple historical data segments, allowing the number and distribution of categories to adapt automatically to changes in data distribution over time, while maintaining relative simplicity through the use of established clustering algorithms.
Data Source
AI summary
Provided is a method for clustering a data stream. The method comprises: acquiring a plurality of resulting models of a plurality of preceding data partitions prior to a current data partition in a data stream, wherein data partitions in the data stream have a temporal relationship, and wherein each of the plurality of resulting models is generated according to a clustering result of a corresponding preceding data partition, and each of the plurality of resulting models comprises one or more representative parameters in different categories; determining a starting model of the current data partition according to the plurality of resulting models, wherein the starting model comprises one or more representative parameters in different categories determined based on representative parameters of the same category in the plurality of resulting models; and clustering data records in the current data partition by using the starting model.