Variable point interval detection algorithm based on multiple stationarity hypothesis testing

Abstract

The invention discloses a change point interval detection algorithm based on multiple stationarity hypothesis testing, which comprises the following steps of: performing missing value filling and abnormal value removal processing on a time sequence, and strictly defining a segmented stationary time sequence and the type of a change point by using a mathematical language; setting the number of initial change points of the segmented stationary time sequence, the interval where the initial change points are located and the shortest time sequence length required by stationary detection; and finding out intervals where all change points are located by using a bisection method and a stability test. The method has the beneficial effects that the change point position is accurately positioned through data segmentation by applying multiple stationarity hypothesis testing, the method can be applied to segmented stationary sequences, excessive model hypotheses are not needed, the application range is wider, and solution implementation is simple.

Description

Technical Field

[0001] The present invention relates to the technical field of data analysis, and in particular to a change point interval detection algorithm based on multiple stationarity hypothesis testing. Background Art

[0002] The change point detection algorithm is a method used to detect the time points at which mutations occur in time series data. The specific algorithm flow is as follows: Data preprocessing: Clean and standardize the time series data to eliminate the influence of noise and outliers; Select a suitable change point detection algorithm: Select a suitable change point detection algorithm based on the data characteristics and application requirements. Different algorithms are suitable for different types of time series data and change point features; Determine algorithm parameters: Determine the parameters of the change point detection algorithm, such as threshold, window size, etc., according to the specific situation. The selection of these parameters will affect the performance and accuracy of the algorithm; Perform change point detection: Run the change point detection algorithm, analyze the time series data, and detect the change point locations; Result evaluation and analysis: Evaluate the change point detection results to judge the performance of the algorithm. At the same time, analyze the detected change points to explore the reasons and impacts behind them.

[0003] Although the existing change point detection algorithms can identify structural changes in time series, by detecting change points, we can better understand the dynamic characteristics of time series and provide a basis for further analysis and decision-making. However, the existing change point detection algorithms are based on piecewise stationary series and require model assumptions and the construction of objective functions for solution. They have many application restrictions and are cumbersome to implement. Summary of the invention

[0004] In order to solve the above problems, especially to address the deficiencies in the prior art, the present invention provides a change point interval detection algorithm based on multiple stationarity hypothesis tests that can solve the above problems.

[0005] To achieve the above purpose, the present invention adopts the following technical means:

[0006] A change point interval detection algorithm based on multiple stationarity hypothesis testing includes the following steps:

[0007] Step (1): Data preprocessing and definition

[0008] Fill missing values ​​and remove outliers in time series, and use mathematical language to rigorously define the types of piecewise stationary time series and change points;

[0009] Step (2): Initial parameter setting

[0010] Set the number of initial change points of the piecewise stationary time series, the interval where the initial change points are located, and the shortest time series length required for stationarity detection;

[0011] Step (3): Detect the set of change point intervals

[0012] Use the bisection method and stationarity test to find the intervals in which all the change points are located.

[0013] A further solution of the present invention is that the method for filling missing values ​​in the time series in step (1) is a forward filling method, a backward filling method or an interpolation method.

[0014] A further solution of the present invention is that in step (1), outliers are removed by identifying them through visual detection, standard deviation method or quartile method, and are replaced by mean or median, or severe outliers are directly deleted.

[0015] A further solution of the present invention is that the method for rigorously defining the types of piecewise stationary time series and change points in step (1) is:

[0016] Assume that there are N+1 time points after the time series fills in missing values ​​and removes outliers. Let this time series be {X i :i=0,…,N};

[0017] Further assume that it is a piecewise stationary time series, that is is a stationary time series;

[0018] Therefore, the change point to be detected is There are K change points in total, which can divide the time series into multiple stable segments, and the statistical characteristics of each segment are relatively stable.

[0019] A further solution of the present invention is that the specific method for setting the parameters in step (2) is:

[0020] Assumptions The change point interval {[a k ,b k ]:k=1,2,…,K} detected;

[0021] The number of change points K is unknown, so we initialize K0 = 1 and first find the interval where the first change point is located. This change point must be in [0, N], so a1 = 0, b1 = N;

[0022] Given the shortest time series length L required for stationarity test, when the length of the time series is greater than or equal to L, stationarity test is performed.

[0023] A further solution of the present invention is that in the step (2), L=30.

[0024] A further solution of the present invention is that the specific method of detecting the set of change point intervals in step (3) is:

[0025] Step 1: Transform the time series {Xi :i=0,…,N} is divided into two time series and Where m1 is the midpoint of [0,N], that is For two time series and Perform stationarity hypothesis test respectively;

[0026] Step 2: Assume that K0 change point intervals have been found, denoted as [a k ,b k ],k=1,2,…,K0,for the min{m k -a k ,m k -b k}>The change point interval of L[a k ,b k ] Each performs the following operations:

[0027] 1) Take the midpoint Where [x] represents the integer closest to x;

[0028] 2) Set the time period Divide into two time periods and For time series and Perform stationarity hypothesis test respectively;

[0029] 3) If and If stable, then m k is the change point, and the change point interval is [m k ,m k ], K0 remains unchanged;

[0030] If only Unstable, indicating that the change point is located at [a k ,m k ], the subsequent time period Continue to look for the change point in , K0 remains unchanged;

[0031] If only Unstable, indicating that the change point is located at [m k ,b k ], the subsequent time period Continue to look for the change point in , K0 remains unchanged;

[0032] like and are not stable, indicating that [a k ,m k ] and [m k ,bk ] all contain change points, let K0 = K0 + 1, and then we need to and Continue to look for change points;

[0033] Step 3: Rearrange the change point interval found in Step 2 and record it as [a k ,b k ], k=1,2,…,K0;

[0034] If there is a change point interval [a k ,b k ] satisfies min{m k -a k ,m k -b k}>L, and then continue to shorten the change point interval according to Step 2;

[0035] If there is no change point interval [a k ,b k ] satisfies min{m k -a k ,m k -b k}>L, the number of change points is K=K0, and the change point interval is [a k ,b k ], k=1,2,…,K.

[0036] A further solution of the present invention is that the two time series in Step 1 and There are four situations for stationary hypothesis test:

[0037] 1) If and are all stable, indicating that the midpoint m1 is the change point we are looking for. At this time, it means that the time series {X i :i=0,…,N} There is only one change point, and the interval where the change point is located is the change point interval [m1,m1], at this time K0=1;

[0038] 2) If Steady, but It is not stable, which means that the change point is in [m1,b1], and the subsequent time period needs to be Continue to look for the change point, at this time k0=1;

[0039] 3) If Steady, but It is not stable, which means the change point is in [a1,m1], and the subsequent time period needs to be Continue to look for the change point, at this time K0=1;

[0040] 4) If and None of them are stable, indicating that there are change points in [a1,m1] and change points in [m1,b1]. and time period There are change points in the time series. The time series contains at least 2 change points, that is, K ≥ 2. and Find the change points respectively, at this time K0=2.

[0041] Beneficial effects of the present invention:

[0042] The present invention uses multiple stationary hypothesis tests to accurately locate the position of the change point through data segmentation. It can be applied to piecewise stationary sequences, does not require too many model assumptions, has a wider application range, and is simple to solve and implement. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 It is a flow chart of the present invention. DETAILED DESCRIPTION

[0044] The technical solution of the present invention will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0045] Example

[0046] like Figure 1 As shown, a change point interval detection algorithm based on multiple stationarity hypothesis testing includes the following steps:

[0047] Step (1), data preprocessing and definition:

[0048] The specific method is to fill missing values ​​and remove outliers in the time series, and use mathematical language to rigorously define the types of piecewise stationary time series and change points. The significance of this step is that missing values ​​will lead to incomplete data, and filling missing values ​​can help maintain the continuity of the time series; outliers are often noise in the data, which may significantly distort statistical characteristics, thereby leading to misjudgment of change point detection. The significance of using mathematical language for rigorous definition is that the definition of change points can take many forms, and it is necessary to clarify in advance the type of change point to be detected and the significance of detecting the change point.

[0049] Step (2), initial parameter setting:

[0050] The specific method is to set the number of initial change points of the piecewise stationary time series, the interval where the initial change points are located, and the shortest time series length required for stationarity detection. The significance of this step is that the number of change points of the time series is unknown in practice, so it is necessary to assume that there is at least one change point in the time series at the beginning. And because a time series that is too short cannot be tested for stationarity, it is necessary to set the shortest time series length required for the test.

[0051] Step (3), detect the set of change point intervals:

[0052] The specific method is to use the dichotomy method and stationarity test to find out the intervals where all the change points are located. The purpose of this step is to continuously reduce the length of the interval where the change points are located through the dichotomy method and stationarity test, so as to accurately locate the interval where the change points are located, and it is suitable for the situation where the number of change points K is unknown.

[0053] Furthermore, the specific method of data preprocessing and definition in step (1) is:

[0054] In practice, the collected time series may contain missing values ​​and outliers. To fill the missing values ​​of the time series, forward filling, backward filling, interpolation and other methods can be used. The selection of the appropriate method depends on the characteristics of the data. Outliers can be identified through visual detection (such as box plots), standard deviation method (using mean and standard deviation to set thresholds), and quartile method (IQR), and can be replaced with mean or median, or severe outliers can be directly deleted.

[0055] Assume that there are N+1 time points after the time series fills in missing values ​​and removes outliers. Let this time series be {X i :i=0,…,N}. Further assume that it is a piecewise stationary time series, that is, is a stationary time series. So the change point we want to detect is There are K change points in total, which can divide the time series into multiple stable segments, and the statistical characteristics of each segment are relatively stable.

[0056] Furthermore, the specific method for setting the parameters in step (2) is:

[0057] Assumptions Our goal is to convert the change point interval {[a k ,b k ]:k=1,2,…,K}. Since the number of change points K is unknown in practice, initialize K0=1 and first find the interval where the first change point is located. The change point must be located in [0,N], so a1=0, b1=N. In addition, the shortest time series length L required for stationarity test must be given, that is, only when the length of the time series is greater than or equal to L, can stationarity test be performed, generally L=30.

[0058] Furthermore, the specific method of detecting the set of change point intervals in step (3) is:

[0059] The core idea of ​​the algorithm is to use the dichotomy method and stationarity test to find the set {[a k ,b k ]k=1,2,…,K} and reduce the interval [a k ,b k ], and keep approaching the position of the change point. When the time series length is too short to test the stationarity hypothesis, it stops. i :i=0,…,N}, a segment from the Time point to The time period of the time point is in Here are the steps:

[0060] Step 1: Transform the time series {X i :i=0,…,N} is divided into two time series and Where m1 is the midpoint of [0,N], that is For two time series and Perform stationary hypothesis tests separately. There are four situations:

[0061] 1) If and are all stable, indicating that the midpoint m1 is the change point we are looking for. At this time, it means that the time series {X i :i=0,…,N], there is only one change point, and the interval where the change point is located is the change point interval [m1,m1], at this time K0=1;

[0062] 2) If Steady, but It is not stable, which means that the change point is in [m1,b1], and the subsequent time period needs to be Continue to look for the change point, at this time K0=1;

[0063] 3) If Steady, but It is not stable, which means the change point is in [a1,m1], and the subsequent time period needs to be Continue to look for the change point, at this time K0=1;

[0064] 4) If and None of them are stable, indicating that there are change points in [a1,m1] and change points in [m1,b1]. and time period There are change points in the time series, and the time series contains at least 2 change points, that is, K ≥ 2. and Find the change points respectively, at this time K0=2.

[0065] Step 2: Assume that K0 change point intervals have been found, denoted as [a k ,b k ],k=1,2,…,K0,for the min{m k -a k ,m k -b k}>The change point interval of L[a k ,b k ] Each performs the following operations:

[0066] 1) Take the midpoint Where [x] represents the integer closest to x;

[0067] 2) Set the time period Divide into two time periods and For time series and Perform stationarity hypothesis test respectively;

[0068] 3) If and If stable, then m k is the change point, and the change point interval is [m k ,m k ], K0 remains unchanged; if only Unstable, indicating that the change point is located at [a k ,m k ], the subsequent time period Continue to look for the change point in , K0 remains unchanged. Unstable, indicating that the change point is located at [m k ,b k ], the subsequent time period Continue to look for the change point in , K0 remains unchanged. and are not stable, indicating that [a k ,m k ] and [m k ,b k ] all contain change points, let K0 = K0 + 1, and then we need to and Continue to look for changing points.

[0069] Step 3: Rearrange the change point interval found in Step 2 and record it as [a k,b k ], k=1,2,…,K0; if there is a change point interval [a k ,b k ] satisfies min{m k -a k ,m k -b k}>L, and then continue to shorten the change point interval according to Step 2; if there is no change point interval [a k ,b k ] satisfies min{m k -a k ,m k -b k}>L, the number of change points is K=K0, and the change point interval is [a k ,b k ], k=1,2,…,K.

[0070] The examples given in the present invention are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here, and the obvious changes or modifications derived therefrom are still within the protection scope of the present invention.

Claims

1. A change point interval detection algorithm based on multiple stationarity hypothesis testing, characterized in that: The following steps are involved: Step (1): Data preprocessing and definition Fill missing values ​​and remove outliers in time series, and use mathematical language to rigorously define the types of piecewise stationary time series and change points; Step (2): Initial parameter setting Set the number of initial change points of the piecewise stationary time series, the interval where the initial change points are located, and the shortest time series length required for stationarity detection; Step (3): Detect the set of change point intervals Use the bisection method and stationarity test to find the intervals in which all the change points are located.

2. The change point interval detection algorithm based on multiple stationarity hypothesis testing according to claim 1 is characterized in that: The method for filling missing values ​​in the time series in step (1) is forward filling method, backward filling method or interpolation method.

3. The change point interval detection algorithm based on multiple stationarity hypothesis testing according to claim 1 is characterized in that: In step (1), outliers are removed by identifying them through visual detection, standard deviation method or quartile method, and are replaced by mean or median, or severe outliers are directly deleted.

4. The change point interval detection algorithm based on multiple stationarity hypothesis testing according to claim 1 is characterized in that: The method for rigorously defining the types of piecewise stationary time series and change points in step (1) is: Assume that there are N+1 time points after the time series fills in missing values ​​and removes outliers. Let this time series be {X i :i=0,…,N}; Further assume that it is a piecewise stationary time series, that is is a stationary time series; Therefore, the change point to be detected is There are K change points in total, which can divide the time series into multiple stable segments, and the statistical characteristics of each segment are relatively stable.

5. The change point interval detection algorithm based on multiple stationarity hypothesis testing according to claim 1 is characterized in that: The specific method for setting the parameters in step (2) is: Assumptions The change point interval {[a k ,b k ]:k=1,2,…,K} detected; The number of change points K is unknown, so we initialize K0 = 1 and first find the interval where the first change point is located. This change point must be in [0, N], so a1 = 0, b1 = N; Given the shortest time series length L required for stationarity test, when the length of the time series is greater than or equal to L, stationarity test is performed.

6. The change point interval detection algorithm based on multiple stationarity hypothesis testing according to claim 5 is characterized in that: In the step (2), L=30.

7. The change point interval detection algorithm based on multiple stationarity hypothesis testing according to claim 1 is characterized in that: The specific method of detecting the change point interval set in step (3) is: Step 1: Transform the time series {X i :i=0,…,N} is divided into two time series and Where m1 is the midpoint of [0,N], that is For two time series and Perform stationarity hypothesis test respectively; Step 2: Assume that K0 change point intervals have been found, denoted as [a k ,b k ],k=1,2,…,K0,for the min{m k -a k ,m k -b k }>The change point interval of L[a k ,b k ] Each performs the following operations: 1) Take the midpoint Where [x] represents the integer closest to x; 2) Set the time period Divide into two time periods and For time series and Perform stationarity hypothesis test respectively; 3) If and If stable, then m k is the change point, and the change point interval is [m k ,m k ], K0 remains unchanged; If only Unstable, indicating that the change point is located at [a k ,m k ], the subsequent time period Continue to look for the change point in , K0 remains unchanged; If only Unstable, indicating that the change point is located at [m k ,b k ], the subsequent time period Continue to look for the change point in , K0 remains unchanged; like and are not stable, indicating that [a k ,m k ] and [m k ,b k ] all contain change points, let K0 = K0 + 1, and then we need to and Continue to look for change points; Step 3: Rearrange the change point interval found in Step 2 and record it as [a k ,b k ], k=1,2,…,K0; If there is a change point interval [a k ,b k ] satisfies min{m k -a k ,m k -b k }>L, and then continue to shorten the change point interval according to Step 2; If there is no change point interval [a k ,b k ] satisfies min{m k -a k ,m k -b k }>L, the number of change points is K=K0, and the change point interval is [a k ,b k ], k=1,2,…,K.

8. The change point interval detection algorithm based on multiple stationarity hypothesis testing according to claim 7 is characterized in that: The two time series in Step 1 and There are four situations for stationary hypothesis test: 1) If and are all stable, indicating that the midpoint m1 is the change point we are looking for. At this time, it means that the time series {X i :i=0,…,N} There is only one change point, and the interval where the change point is located is the change point interval [m1,m1], at this time K0=1; 2) If Steady, but It is not stable, which means that the change point is in [m1,b1], and the subsequent time period needs to be Continue to look for the change point, at this time K0=1; 3) If Steady, but It is not stable, which means the change point is in [a1,m1], and the subsequent time period needs to be Continue to look for the change point, at this time K0=1; 4) If and None of them are stable, indicating that there are change points in [a1,m1] and change points in [m1,b1]. and time period There are change points in the time series. The time series contains at least 2 change points, that is, K ≥ 2. and Find the change points respectively, at this time K0=2.

Images