Multi-source time series data stream concept drift detection method based on dual inspection
By employing a dual-verification method, combining error matrix and historical window comparison, the problem of misjudgment in multi-source time-series data stream drift detection is solved, improving the accuracy and robustness of detection. This method is applicable to dynamic data stream analysis in fields such as finance and industrial monitoring.
Patent Information
- Application Number
- CN202511173279.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-08-21
- Publication Date
- 2025-12-12
AI Technical Summary
Existing methods for detecting drift in multi-source time-series data streams are insufficient in terms of detection accuracy and robustness. In particular, they are prone to misjudging and ignoring the correlation between historical states in time-series data prediction tasks, resulting in low detection accuracy.
A dual-test method is adopted, which involves constructing a multi-output error matrix, performing a first hypothesis test and a second historical window comparison test, and combining Welch's t test and MAED calculation to determine the significance and similarity of error changes, thereby ensuring the accuracy of detection.
It significantly improves the accuracy and robustness of drift detection in multi-source time-series data streams, and is suitable for dynamic data stream analysis in fields such as finance and industrial monitoring.
Smart Images
Figure CN121117872A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of data stream mining, specifically involving a method for detecting concept drift in multi-source time-series data streams based on dual verification. Background Technology
[0002] In typical applications such as industrial monitoring, energy management, financial transactions, and transportation systems, intelligent prediction systems widely rely on modeling and analyzing multi-source time-series data streams to achieve functions such as equipment status monitoring, trend prediction, and anomaly alarms. However, with the continuous evolution of the external environment, data distribution, or business logic, the predictive performance of the original model will inevitably degrade, manifesting as a continuous increase in error or prediction deviation. If concept drift cannot be identified and adjusted in time, it may lead to system anomalies, resource waste, or even business risks. Most existing concept drift detection methods are based on classification scenarios and are suitable for discrete labeled data; there are relatively few drift detection methods specifically for time-series data prediction tasks. A common strategy in existing methods is to detect drift based on changes in prediction error. When the error increases significantly within a continuous window, and this increase is determined to be statistically significant through hypothesis testing, the system determines that concept drift has occurred at the current moment. However, this type of error-driven method has several significant drawbacks. On the one hand, because the model cannot perfectly fit complex real-world data in practical applications and has certain structural residuals, some naturally fluctuating errors are misjudged as drift signals, thus reducing detection accuracy. On the other hand, these methods typically only focus on abrupt changes in error within the current window, ignoring the correlation with historical states, which can easily lead to misjudgments.
[0003] Therefore, a conceptual drift detection algorithm is needed that can significantly improve the drift detection accuracy of multi-source time-series data streams. Summary of the Invention
[0004] In view of the above-mentioned technical problems in the prior art, the present invention proposes a concept drift detection method for multi-source time-series data streams based on dual verification. The method is reasonably designed, overcomes the shortcomings of the prior art, and has good results.
[0005] To achieve the above objectives, the present invention adopts the following technical solution:
[0006] A concept drift detection method for multi-source time-series data streams based on dual verification includes the following steps:
[0007] Step S1: Acquire multi-source time-series prediction error signals and construct a multi-output error matrix;
[0008] Step S2: Divide the current time window and calculate the error sequence of each predicted output within the current time window;
[0009] Step S3: Based on the first hypothesis test, determine whether the prediction error has changed significantly within the current window;
[0010] Step S4: Retrieve the historical time window in the historical window set that is most similar to the time series data of the current window;
[0011] Step S5: Based on the second test, determine whether the prediction error of the current window is significantly higher than the prediction error of the most similar historical window;
[0012] Step S6: Based on the results of the double test, determine whether concept drift has occurred.
[0013] Preferably, step S1 is specifically performed as follows:
[0014] Step S1.1: At each time point t, collect the predicted values of each output dimension k of the prediction model. and the corresponding true observations in multi-source time series data
[0015] Step S1.2: Calculate the absolute error for each time point t and each output dimension k.
[0016] Step S1.3: Integrate the errors of all dimensions at each time point into an error matrix E, where each row represents all output errors at a certain time and each column represents the error sequence of a certain dimension.
[0017] Preferably, step S2 is performed as follows:
[0018] Step S2.1: Set the window length w, and construct the current analysis window W by backtracking w time steps from the current time point each time. t ={t-w+1,...,t};
[0019] Step S2.2: Extract the error submatrix E corresponding to the current window from the error matrix. t The matrix has dimensions w×K and represents the error distribution of all predicted output variables within the current time period.
[0020] Preferably, step S3 is performed as follows:
[0021] Step S3.1: Select a window of the same length preceding the current window as the reference window W. ref ={t-2w+1,...,tw}, extract its error matrix E ref , with the current window error submatrix E t Formation time comparison;
[0022] Step S3.2: For each output dimension k = 1, ..., K, calculate the error vector for the corresponding dimension of the current window error matrix. Error vector corresponding to the dimension of the reference window error matrix The sample mean and variance were used to perform the Welch's test, and the confidence interval was calculated in conjunction with the sample variance.
[0023] Step S3.3: Calculate the p-value for each dimension, which represents the significance of the difference between the mean error of the current window and the mean error of the reference window. By comparing it with the set significance level, determine whether the change is statistically significant.
[0024] Step S3.4: If the p-value of at least one output dimension is lower than the threshold, it is considered that there is a significant increase in prediction error within the current window, indicating a mismatch between the model and the data distribution, which is a candidate signal for concept drift.
[0025] Preferably, step S4 is performed as follows:
[0026] Step S4.1: Maintain the history window library History Window Library Y contains a multi-source time-series data matrix with multiple non-drift windows. i And it is dynamically updated;
[0027] Step S4.2: For each historical window, the multi-source time-series data matrix Y i Calculate its relationship with the current time series data matrix Y in sequence. t Distance between:
[0028]
[0029] in, The current time series data matrix Y is respectively t and multi-source time series data matrix Y i The element in the j-th column and k-th row;
[0030] Step S4.3: Traverse the historical database and calculate the relationship between all historical windows and the current time series data matrix Y. t The window with the smallest distance from the MAED value is selected. As a reference for the current window;
[0031] Step S4.4: If the minimum distance still exceeds the preset similarity threshold α sim If there are not enough similar windows in history, skip the second check to avoid misjudgment.
[0032] Preferably, step S5 is performed as follows:
[0033] Step S5.1: Calculate the current window error submatrix E respectively. t Error matrix with most similar historical window The mean squared error (MSE) in each output dimension is:
[0034]
[0035] in, and E respectively t and The mean squared error in the k-th dimension, and E respectively t and The element in the j-th column and k-th row;
[0036] Step S5.2: Calculate the error variation magnitude for each dimension. Then, the total change Δ is obtained by averaging all dimensions.
[0037] Step S5.3: Based on the given threshold δ, determine whether the error change is significant;
[0038] Step S5.4: If Δ>δ, it means that the error of the current window has increased beyond the reasonable range of change of the most similar historical window. The second test is valid, and the current state is judged to be abnormal.
[0039] Preferably, step S6 is specifically performed as follows:
[0040] Step S6.1: If both the first error significance test and the second historical comparison test are valid, the occurrence of concept drift is confirmed.
[0041] Step S6.2: Record the judgment result to the drift event sequence for subsequent model retraining, system maintenance or alarm purposes;
[0042] Step S6.3: If it is not determined to be a drift, then change the current time series data matrix Y. t and the current window error submatrix E t Add to history window library It also limits the size of the historical pool to avoid information redundancy.
[0043] The beneficial technical effects of this invention are as follows:
[0044] This invention solves the problem of high false positive rate in drift detection in multi-source data by using double verification and similar window comparison, thereby improving the accuracy and robustness of drift detection. It is applicable to dynamic data stream analysis in fields such as finance and industrial monitoring. Attached Figure Description
[0045] Figure 1 This is a flowchart of the method of the present invention.
[0046] Figure 2This is a logical framework diagram for implementing drift detection in this invention. Detailed Implementation
[0047] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments:
[0048] A concept drift detection method for multi-source temporal data streams based on dual verification, the process of which is as follows: Figure 1 As shown, it includes the following steps:
[0049] Step S1: Acquire multi-source time-series prediction error signals and construct a multi-output error matrix; the specific process is as follows:
[0050] Step S1.1: At each time point t, collect the predicted values of each output dimension k of the prediction model. and the corresponding true observations in multi-source time series data
[0051] Step S1.2: Calculate the absolute error for each time point t and each output dimension k.
[0052] Step S1.3: Integrate the errors of all dimensions at each time point into an error matrix E, where each row represents all output errors at a certain time and each column represents the error sequence of a certain dimension.
[0053] Step S2: Divide the current time window and calculate the error sequence of each predicted output within the current time window; the specific process is as follows:
[0054] Step S2.1: Set the window length w, and construct the current analysis window W by backtracking w time steps from the current time point each time. t ={t-w+1,...,t};
[0055] Step S2.2: Extract the error submatrix E corresponding to the current window from the error matrix. t The matrix has dimensions w×K and represents the error distribution of all predicted output variables within the current time period.
[0056] Step S3: Based on the first hypothesis test, determine whether the prediction error has changed significantly within the current window; the specific process is as follows:
[0057] Step S3.1: Select a window of the same length preceding the current window as the reference window W. ref ={t-2w+1,...,tw}, extract its error matrix E ref , with the current window error submatrix E t Formation time comparison;
[0058] Step S3.2: For each output dimension k = 1, ..., K, calculate the error vector for the corresponding dimension of the current window error matrix. Error vector corresponding to the dimension of the reference window error matrix The sample mean and variance were used to perform Welch's t test, and the confidence interval was calculated in conjunction with the sample variance.
[0059] Step S3.3: Calculate the p-value for each dimension, which represents the significance of the difference between the mean error of the current window and the mean error of the reference window. By comparing it with the set significance level, determine whether the change is statistically significant.
[0060] Step S3.4: If the p-value of at least one output dimension is lower than the threshold, it is considered that there is a significant increase in prediction error within the current window, indicating a mismatch between the model and the data distribution, which is a candidate signal for concept drift.
[0061] Step S4: Retrieve the historical time window in the historical window set that is most similar to the time series data of the current window; the specific process is as follows:
[0062] Step S4.1: Maintain the history window library History Window Library Y contains a multi-source time-series data matrix with multiple non-drift windows. i And it is dynamically updated;
[0063] Step S4.2: For each historical window, the multi-source time-series data matrix Y i Calculate its relationship with the current time series data matrix Y in sequence. t Distance between:
[0064]
[0065] in, The current time series data matrix Y is respectively t and multi-source time series data matrix Y i The element in the j-th column and k-th row;
[0066] Step S4.3: Traverse the historical database and calculate the relationship between all historical windows and the current time series data matrix Y. t The window with the smallest distance from the MAED value is selected. As a reference for the current window;
[0067] Step S4.4: If the minimum distance still exceeds the preset similarity threshold α sim If there are not enough similar windows in history, skip the second check to avoid misjudgment.
[0068] Step S5: Based on the second test, determine whether the prediction error of the current window is significantly higher than the prediction error of the most similar historical window; the specific process is as follows:
[0069] Step S5.1: Calculate the current window error submatrix E respectively. t Error matrix with most similar historical window The mean squared error (MSE) in each output dimension is:
[0070]
[0071] in, and E respectively t and The mean squared error in the k-th dimension, and E respectively t and The element in the j-th column and k-th row;
[0072] Step S5.2: Calculate the error variation magnitude for each dimension. Then, the total change Δ is obtained by averaging all dimensions.
[0073] Step S5.3: Based on the given threshold δ, determine whether the error change is significant;
[0074] Step S5.4: If Δ>δ, it means that the error of the current window has increased beyond the reasonable range of change of the most similar historical window. The second test is valid, and the current state is judged to be abnormal.
[0075] Step S6: Based on the results of the double test, determine whether concept drift has occurred; the specific process is as follows:
[0076] Step S6.1: If both the first error significance test and the second historical comparison test are valid, the occurrence of concept drift is confirmed.
[0077] Step S6.2: Record the judgment result to the drift event sequence for subsequent model retraining, system maintenance or alarm purposes;
[0078] Step S6.3: If it is not determined to be a drift, then change the current time series data matrix Y. t and the current window error submatrix E t Add to history window library It also limits the size of the historical pool to avoid information redundancy.
[0079] Figure 2 This is a logical framework diagram for implementing drift detection in this invention.
[0080] Of course, the above description is not intended to limit the present invention, and the present invention is not limited to the examples given above. Any changes, modifications, additions or substitutions made by those skilled in the art within the scope of the present invention should also fall within the protection scope of the present invention.
Claims
1. A method for detecting concept drift in multi-source time series data streams based on dual verification, characterized in that, The method comprises the following steps: Step S1: Collecting multi-source time series prediction error signals and constructing a multi-output error matrix; Step S2: Dividing the current time window and calculating the error sequence of each prediction output in the current time window; Step S3: Based on the first re-hypothesis test, determining whether the prediction error in the current window has changed significantly; Step S4: Searching for the most similar historical time window to the current window in the historical window set; Step S5: Based on the second re-test, determining whether the prediction error in the current window is significantly higher than the prediction error in the most similar historical window; Step S6: Based on the results of the two tests, determining whether a concept drift has occurred.
2. The dual-check based multi-source time-series data stream concept drift detection method according to claim 1, wherein, The specific process of step S1 is: Step S1.1: At each time point t, collect the predicted value for each output dimension k of the prediction model and the corresponding real observed value in the multi-source time series data Step S1.2: For each time point t and each output dimension k, compute the absolute error Step S1.3: Integrating the errors of all dimensions at each time point into an error matrix E, where each row represents the errors of all outputs at a certain time, and each column represents the error sequence of a certain dimension.
3. The dual-check based multi-source time-series data stream concept drift detection method according to claim 1, wherein, The specific process of step S2 is: Step S2.1: Set the window length w and construct the current analysis window W each time by backtracking w time steps from the current time point t = {t - w + 1,..., t}; Step S2.2: Extracting the error sub-matrix E corresponding to the current window from the error matrix t with dimension w x K, representing the error distribution of all predicted output variables in the current time period.
4. The dual-check based multi-source time-series data stream concept drift detection method according to claim 1, characterized in that, The specific process of step S3 is: Step S3.1: Choose a window of the same length as the current window before as reference window W ref = {t-2w+1,...,t-w}, extract its error matrix E ref , and the current window error sub-matrix E t Form a time contrast; Step S3.2: For each output dimension k = 1,..., K, compute the error vector of the current window error matrix corresponding dimension, respectively the sample mean and variance of the error vector of the reference window error matrix corresponding dimension perform Welch's t-test and compute the confidence interval in combination with the sample variance; Step S3.3: Calculate the p-value of each dimension, which represents the significance of the difference between the error mean of the current window and the error mean of the reference window. By comparing with the set significance level, it is determined whether the change has statistical significance; Step S3.4: If the p-value of at least one output dimension is lower than the threshold, it is considered that there is a significant prediction error increase in the current window, indicating a mismatch between the model and the data distribution, which is a candidate signal for concept drift.
5. The dual-check based multi-source time-series data stream concept drift detection method according to claim 1, wherein, The specific process of step S4 is: Step S4.1: Maintaining the history window library History window library A multi-source time series data matrix Y with multiple non-drift windows is stored in the middle i and dynamically updated; Step S4.2: For each historical window, calculate the distance between the corresponding multi-source time series data matrix Y i and the current time series data matrix Y t in turn. wherein, are the current time series data matrix Y t and the multi-source time series data matrix Y i the element in the jth column and the kth row of Step S4.3: Traverse the history library, calculate the MAED value of all history windows with the current time series data matrix Y t , select the window with the smallest distance as the reference control of the current window ; Step S4.4: If the minimum distance still exceeds the preset similarity threshold a sim , consider that there is no enough similar window in the history, skip the second recheck to avoid misjudgment.
6. The dual-check based multi-source time-series data stream concept drift detection method according to claim 1, wherein, The specific process of step S5 is: Step S5.1: Calculate the current window error sub-matrix E t the most similar historical window error matrix Mean Squared Error, MSE, over each output dimension, i.e.: wherein and are E t and the mean squared error in the kth dimension, and are E t and the element in the jth column and kth row in Step S5.2: Calculate the error variation amplitude for each dimension and average all dimensions to get the total variation Δ; Step S5.3: Based on the given threshold δ, determine whether the error change is significant; Step S5.4: If Δ>δ, it means that the error of the current window has increased beyond the reasonable range of variation of the most similar historical window, and the second re-test is established. The current state is determined to be abnormal.
7. The dual-check based multi-source time-series data stream concept drift detection method according to claim 1, wherein, The specific process of step S6 is: Step S6.1: When both the first re-error significance test and the second re-historical comparison test are established, the occurrence of concept drift is confirmed; Step S6.2: Record the judgment result to the drift event sequence for subsequent model retraining, system maintenance or alarm use; Step S6.3: If not drift, then append current timing data matrix Y t and current window error sub-matrix E t to history window library and limit history pool size to avoid information redundancy.