An unsupervised concept drift detection method based on stack autoencoder and page-hinckley test

By combining stacked autoencoders and Page-Hinckley tests, the problems of label dependence and data distribution fitting difficulties in existing technologies are solved, achieving unsupervised high-accuracy concept drift detection that adapts to dynamic data environments.

CN116151361BActive Publication Date: 2026-05-12NANJING UNIV OF AERONAUTICS & ASTRONAUTICS
View PDF 2 Cites 0 Cited by

Patent Information

Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
NANJING UNIV OF AERONAUTICS & ASTRONAUTICS
Filing Date
2023-02-22
Publication Date
2026-05-12

AI Technical Summary

Technical Problem

Most existing concept drift detection methods rely on labels, especially supervised methods, which waste resources in real-world scenarios. Unsupervised methods, on the other hand, struggle to accurately fit data distributions, and traditional density estimation methods perform poorly with limited data samples.

Method used

By combining a stacked autoencoder and the Page-Hinckley test, unsupervised concept drift detection is achieved through data window partitioning, training representation distribution, using reconstruction error to measure distribution difference, and dynamically setting thresholds.

Benefits of technology

It achieves high accuracy and low latency concept drift detection under unsupervised conditions, reduces dependence on label resources, and improves the model's adaptability in dynamic environments.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN116151361B_ABST
    Figure CN116151361B_ABST
Patent Text Reader

Abstract

The application discloses an unsupervised concept drift detection method based on a stack autoencoder and a Page-Hinckley test, aims to monitor the concept drift phenomenon existing in a dynamic data stream, and timely adjusts a downstream decision model to improve the generalization ability of the model on new data. The method is carried out according to the following steps: 1. data window division, used for organizing single instances in a data stream into different data windows to support subsequent density estimation of data distribution in the windows. 2. training of a stack autoencoder representing distribution, used for extracting hidden statistical features representing data distribution to realize indirect density estimation of window data. 3. distribution difference measurement, using reconstruction error of the stack autoencoder as a measurement value of distribution difference between window data. 4. adaptive threshold setting, formulating a dynamic threshold through the Page-Hinckley test, and using the idea of hypothesis testing to report drift occurrence when the reconstruction error exceeds the threshold, and simultaneously rapidly adjusting the downstream decision model.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention belongs to the field of dynamic data stream mining technology, and relates to an unsupervised concept drift detection method, particularly an unsupervised concept drift detection method based on stack autoencoder and Page-Hinckley test. Background Technology

[0002] In recent decades, big data and the Internet of Things (IoT) have gradually permeated all sectors of society, involving an increasing number of sensors and systems. These sensors and systems continuously generate massive amounts of data in various formats, which are ultimately transmitted to terminals for real-time online analysis and processing in the form of data streams. Due to the characteristics of data streams, such as continuous high-speed arrival, large data volume, and complex data structures, data stream mining methods are usually limited by memory and runtime, and the same data instance cannot be accessed an arbitrary number of times. How to use effective data stream mining techniques to analyze and process this data in real time to obtain potentially valuable and effective information under limited memory and computing resources has attracted widespread attention. Traditional learning methods, including many online learning methods, usually assume that the data is generated in a static environment, that is, the training data and test data are independent and identically distributed. However, data in the real world environment has dynamic behavior. As products, markets, and customer behavior change, the underlying distribution of data is not static, which means that the statistical characteristics of the target variable that the machine learning model is trying to predict will change unpredictably over time. This phenomenon is called concept drift. For example, in spam email classification systems, as the accuracy of spam classifiers improves, spammers will modify their corresponding strategies to try to deceive the spam classifier. As an important decision-making tool, machine learning systems must be able to detect and adapt to changes in the learning environment.

[0003] Concept drift is defined as follows: Given a time interval [0, t], D 0,t ={d0, d1, d2, ..., d t}, where d i =(X i y i Let X be a data instance arriving at time i. The time interval between adjacent instances is not necessarily the same. i ∈R P Let y be an eigenvector in the P-dimensional feature space at time i. i It is time i X i The corresponding tag. D 0,t Follows distribution F 0,t (X, y). When F 0,t (X, y) ≠ F t,∞ When (X, y), it is considered that a concept drift occurred at time t+1, denoted as

[0004] Current streaming data mining methods with concept drift adaptability can be divided into active and passive adaptation. Active adaptation methods design drift detection methods to continuously monitor whether concepts have changed, adjusting the model only when changes occur to adapt to environmental evolution. These methods are more efficient and have a wider range of applications. To date, drift detection methods developed can be mainly categorized into supervised, semi-supervised, and unsupervised methods. Supervised methods assume that labels for all future and historical data are immediately available. Most corresponding detection methods determine whether drift has occurred by tracking the output of the decision model, issuing drift alerts when the model's performance deteriorates. Unsupervised methods assume that labels are unavailable. These methods directly trace the root cause of drift—distribution drift—by studying the statistical properties of features to determine whether drift has occurred. Semi-supervised methods fall between the two, assuming that at a certain point in time, a small portion of data labels are available, while the majority of data labels are unavailable.

[0005] Existing technologies have several methods capable of detecting the location of concept drift in data streams and adjusting downstream decision-making models accordingly. However, these methods have certain shortcomings: 1) Approximately 85% of existing concept drift detection methods are supervised methods. While these methods have high detection accuracy, their assumption of readily available labels is overly optimistic and unsuitable for real-world scenarios. 2) Although retraining models with labeled data is unavoidable, using labels in the drift detection stage is a waste of label resources. 3) Unsupervised methods focus on changes in feature attribute distribution; modeling the data distribution is crucial, and traditional density estimation methods struggle to accurately fit the distribution of a limited number of data samples.

[0006] In summary, the research and development of an unsupervised concept drift detection method that can automatically extract statistical features of representative distributions in a self-supervised manner, without involving the use of any labels during the detection process, has strong practical significance. Summary of the Invention

[0007] The purpose of this invention is to provide an unsupervised concept drift detection method based on stacked autoencoders and Page-Hinckley tests, which can detect concept drift phenomena that may occur in multidimensional data streams with high accuracy, low latency and low false alarms, thereby ensuring that machine learning models can adapt to new data in a timely manner.

[0008] The present invention provides an unsupervised concept drift detection method based on a stack autoencoder and a Page-Hinckley test, comprising the following steps:

[0009] Step 1: Data window partitioning, which organizes individual instances in the data stream into different data windows to support subsequent density estimation of the data distribution within the windows.

[0010] Step 2: Train a stacked autoencoder representing the distribution to learn the underlying distribution of the data, extract hidden statistical features representing the distribution of the data, and achieve indirect density estimation of the window data.

[0011] Step 3: Measurement of distribution difference. The reconstruction error of the stacked autoencoder is used as a measure of the distribution difference between window data.

[0012] Step 4: Adaptive threshold setting. Dynamic thresholds are set through Page-Hinckley testing, and the idea of ​​hypothesis testing is used to report drift when the reconstruction error exceeds the threshold, and the downstream decision model is adjusted quickly.

[0013] Preferably, the data window division specifically includes three types of windows, namely, a reference window S. ref Current window S cur With distribution characterization window S single .

[0014] Preferably, the data stream is represented as stream = {X1, X2, ..., X...} n}, where X i ∈R P It is an eigenvector in the P-dimensional feature space at time i. Assume the last time concept drift occurred was t. c Reference window S ref It consists of the first set of data to arrive after the last drift, with a window size of w1, i.e. When concept drift occurs again, the reference window should also be updated accordingly. Distribution representation window S single The training set of the stacked autoencoder contains the minimum dataset used to represent the data distribution during training, with a window size of w2. This dataset is continuously slid across the reference window to form the training set. The current window S... cur It always contains the latest w2 data entries, which are initialized as the w2 data entries after the reference window is set, and slides continuously in the direction of arrival of new data instances using a first-in-first-out (FIFO) mechanism. The reference window represents a reference data distribution, and the current window represents the latest current data distribution.

[0015] Preferably, the stacked autoencoder for the training representation distribution utilizes a reference window S. ref The data itself serves as a supervisory signal to guide training. When S is filled... ref Then, the distribution representation window S single Initialized to S refThe first data item w2 arrives in the reference window, and the reference window is continuously slid along the direction of data arrival using a FIFO mechanism with a sliding step size of L. As described above, in The training set for the stacked autoencoder is Train, and S corresponds to each value of i. single Different data sets in the train are stretched into one-dimensional vectors and added to the train. in P represents the dimension of the original data instance.

[0016] Preferably, the stacked autoencoder consists of an encoder and a decoder, with a total of L+1 layers. The input and output dimensions are both D, and the structure is symmetrical. Let T... n For a data element in a train, the encoder part can use an encoding function h = f(T) n The encoder is responsible for compressing the input data into a latent spatial representation in a non-linear manner, extracting the most important information from the input data. `h` represents the hidden features extracted by the encoder. The decoder reconstructs the input from the latent spatial representation, denoted by the decoding function `r = g(h)`. The main goal of training the stacked autoencoder is to make the network's output infinitely close to the input, i.e., to minimize `Dist(T)`. n ,g(f(T) n ))), where Dist is the quantization T n With g(f(T) n The loss function is used to determine the difference between the two values. Commonly used loss functions include mean squared error and cross-entropy. This method uses mean squared error, also known as reconstruction error, which is more suitable for continuous data.

[0017]

[0018] In the formula, For the j-th component of the input vector, g(f(T) n )) j Let be the j-th component of the reconstructed output vector. The training of the stacked autoencoder is guided by backpropagation and gradient descent optimization, with LeakyReLU chosen as the activation function.

[0019] Preferably, the distribution difference metric utilizes the data correlation of the autoencoder, assuming a one-to-one correspondence between the trained autoencoder and the reference data distribution represented by the reference window. After the stacked autoencoder training is completed, the current window S is enabled. cur The data is continuously slid along the direction of arrival using a FIFO mechanism with a step size of L, ensuring that it always contains the latest w2 data entries. S curThe data in the current window is also constructed as a one-dimensional vector and input into a stacked autoencoder. The stacked autoencoder attempts to fit the current data distribution and obtains the reconstruction error. If the data in the current window maintains the same distribution as the data in the reference window, the statistical information representing the data distribution will be correctly extracted and become the output of the neural units in the encoder's hidden layer. The decoder then decodes this hidden statistical information to restore the data to be essentially consistent with the original data. The reconstruction error will then fluctuate around a very small value; otherwise, the reconstruction error will experience a large positive jump.

[0020] Preferably, the Page-Hinckley Test (PH Test) is used to dynamically calculate the adaptive threshold. The dynamic threshold setting method combines historical data characteristics to dynamically update the current threshold, compensating for the shortcomings of static threshold scenarios in terms of capability, configuration threshold, and maintenance cost, and possessing generalization and adjustability. The original purpose of the PH Test was to detect abrupt changes in the average value of Gaussian signals; later, researchers used it for online detection of changes in signal processing. We use the PH Test to determine whether drift has occurred by observing the difference between the reconstruction error of the current data block and the overall pattern. If conceptual drift occurs, the distribution in the current data block changes, and the autoencoder cannot recognize the current distribution state and restore the original data, increasing the reconstruction error. When the reconstruction error exceeds a preset threshold α, it indicates that data drift has occurred.

[0021] PH Test defines p i Let p be the current observed value of the random variable p at time i. m represents the average of all observations up to time t. t As a cumulative variable, it stores the cumulative difference between the observed value and the mean of the previous observed values:

[0022]

[0023]

[0024] In the formula, δ is a non-negative real number close to 0, representing the maximum allowable variation. Through continuous observation of M... t With m t The difference between the values ​​is used to detect whether the mean of the variable suddenly increases. When the difference exceeds the threshold α at the current time, the value is considered to be positive. t When the pH test is performed, it will report the changes, including:

[0025] M t =max{m1, m2, ..., m t}

[0026]

[0027] In the formula, λ is a hyperparameter that makes α at each time step... t It can automatically adjust based on historical observations. Warning thresholds are defined separately. With drift threshold Reaching the warning threshold indicates a potential change in the data distribution, requiring the use of subsequent incremental data samples to train a new model M. new And maintain. Reaching the drift threshold indicates a drift has occurred, using M... new Replace the old model to accommodate the new data after the drift.

[0028] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0029] 1) This invention focuses on the distribution characteristics of feature attributes and does not involve the use of any labels in the concept drift detection process, thus falling into the category of unsupervised detection methods.

[0030] 2) This invention utilizes the powerful nonlinear expressive and feature extraction capabilities of autoencoders to capture the underlying distribution information of batch data during training, effectively representing the statistical characteristics of the overall data distribution. This distribution information is hidden and difficult to extract directly using statistical methods, thereby enhancing the flexibility of modeling data distribution information.

[0031] 3) Using a dynamic adaptive threshold setting method can overcome the shortcomings of static thresholds: 1. Static thresholds rely too heavily on manual intervention, requiring a comprehensive analysis of the dataset to select a suitable threshold, resulting in a waste of human resources. 2. Different window sizes, drift types, and data distributions all affect the selection of the threshold. 3. Static thresholds have high maintenance costs, requiring manual adjustments based on business experience. Attached Figure Description

[0032] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention, but do not constitute a limitation thereof. Wherein:

[0033] Figure 1 This is an overall flowchart of an unsupervised concept drift detection method based on a stack autoencoder and Page-Hinckley test according to the present invention.

[0034] Figure 2 This is a diagram illustrating the training process of a stacked autoencoder. Detailed Implementation

[0035] The preferred embodiments of the present invention will be described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0036] Figure 1 This paper presents the overall framework of an unsupervised conceptual drift detection method based on a stacked autoencoder and the Page-Hinckley test. The method employs a windowing strategy to organize data instances into data blocks with certain distribution information. It leverages the superior feature extraction performance of the autoencoder to model the distribution of the window data and uses the reconstruction error as a measure of the distribution difference between the current window data and the reference window data. Furthermore, the Page-Hinckley test is used to dynamically calculate a threshold representing the occurrence of drift; a drift alarm is issued when the reconstruction error exceeds the threshold. The specific steps are as follows:

[0037] Step 1: Divide the data window, organizing the continuously arriving data stream instances into data blocks. Let the last drift time be t. c , t c Initially set to 0, the data stream is represented as stream = {X1, X2, ..., X...} n}, where X i ∈R P It is an eigenvector in the P-dimensional feature space at time i. It specifies the reference window size, the current window size, the distribution representation window size, and the window sliding step number L. For example, the reference window S... ref The window size w1 is set to 1000, and the current window S cur With distribution characterization window S single The capacity w2 is set to 200, and the number of sliding steps L is set to 20. ref being t c The latest 1000 data entries are loaded after the specified time. S single From S ref Starting from the initial position of the window, a first-in-first-out (FIFO) mechanism is used in step size L=20 at position S. ref S slide up continuously, S single The dataset in step two will be processed into the input of the stacked autoencoder and added to the training set of the stacked autoencoder. cur It is first initialized to t. c The 200 newest data points arrive after +1000 time points, and then slide continuously in the direction of new data instance arrivals, so that it always consists of the 200 newest data points.

[0038] Step 2: Train a stacked autoencoder that represents the data distribution. Figure 2 The training process of the autoencoder is demonstrated. After dividing the data stream into different data windows in step one, the stacked autoencoder is trained using the data in the reference window.

[0039] S1: Use the window division method described in the example in step one. single ={X tc+i X tc+i+1 , ..., X tc+i+199},in

[0040] S2: The training set for the stacked autoencoder is Train, and each i value corresponds to S. single Different data sets are stretched into one-dimensional vectors and added to Train, resulting in Train = {T0, T1, ..., T}. 40}, where T i ∈R P*200 P represents the dimension of the original data instance.

[0041] S3: The stacked autoencoder consists of an encoder and a decoder. The specific structure of the stacked autoencoder is defined, for example, with both input and output dimensions being P*200, and a total of 7 layers (0-6). The number of neurons in each layer is P*200, 512, 64, 10, 64, 512, and P*200 respectively. The value of the j-th neuron in the l-th layer is calculated as follows:

[0042]

[0043] T n This is a data element in the Train, used as input to the autoencoder. Let j be the j-th component of the input vector. N l-1 The number of neurons in layer (l-1) The weights between layer (l-1) and layer l are... Let σ(z) be the bias of the l-th layer, and σ(z) be the activation function. LeakyReLU is chosen as the activation function. Its expression is as follows:

[0044] σ(z)=max(0,z)+leak*min(0,z) (2)

[0045] In the formula, leak is taken as 0.01.

[0046] The training objective is to make the network input and output infinitely close, that is, to minimize the loss function Dist(T). n This is also known as reconstruction error.

[0047]

[0048] The autoencoder was trained using backpropagation and gradient descent optimization algorithms for a total of 500 epochs. Each training batch size was 64, and the current batch size was {T}.t T t+1 , ..., T t+63 The SGD optimizer is used with a learning rate of 0.005. Weights and biases are updated according to the following formula:

[0049]

[0050]

[0051] Step 3: Distribution Disparity Measurement. The trained stacked autoencoder is used to fit the data distribution within the current data window. The encoder's reconstruction error is used as a measure of the difference between the current window distribution and the reference window distribution. Enable the current window S. cur S slides continuously along the direction of data arrival with a step size of L=20, ensuring it always contains the latest 200 data points. cur The data in the current window is also constructed into a one-dimensional vector and input into the stacked autoencoder that has been trained in step two, and the reconstruction error is obtained. If the data in the current window is in the same distribution as the data in the reference window, the statistical information representing the data distribution is correctly extracted as the output of the neural units in the encoder's hidden layer. The decoder then decodes this hidden statistical information to restore it to be basically consistent with the original data, and the reconstruction error will fluctuate around a very small value. Otherwise, the reconstruction error will have a large positive jump.

[0052] Step 4: Use the Page-Hinckey test to dynamically calculate the adaptive threshold α at each time step. t The system employs statistical hypothesis testing to detect drift by utilizing the reconstruction error of the current data block. First, a null hypothesis is defined, representing that the data distribution has not changed. This null hypothesis occurs when the reconstruction error of the current window data obtained in step three exceeds the current threshold α. t Rejecting the null hypothesis at that time indicates that data drift has occurred.

[0053] Define p i Let i be the observed value of the current window reconstruction error at time i. m represents the average of all observations up to time t. t For cumulative variables, the calculation method is as follows:

[0054]

[0055]

[0056] In the formula, δ is a non-negative real number close to 0, representing the maximum allowable variation. Through continuous observation of M... t With m t The difference between the values ​​is used to detect whether the mean of the reconstruction error suddenly increases. When the difference exceeds a threshold α, the error is considered to be greater than the threshold value α.t Changes will be reported in due course, including:

[0057] M t =max{m1, m2, ..., m t} (8)

[0058]

[0059] Define warning thresholds separately With drift threshold And determine the value of δ. For example, take λ. w =0.003, λ d =0.007, δ=0.0003. When the reconstruction error reaches the warning threshold, it indicates a potential change in the data distribution. A new model M is then trained using the incremental data samples that arrive later. new And maintain. When the reconstruction error reaches the drift threshold, indicating that drift has occurred, use M... new Replace the old model to adapt to the new data after the drift, and improve the model's accuracy in dynamic environments.

[0060] In summary, the unsupervised concept drift detection method based on a stacked autoencoder and Page-Hinckley test of this invention can accurately detect concept drift in data streams under dynamic environments. After incorporating this method, the accuracy of the decision model can be effectively improved in different dynamic scenarios, which is of great significance in dynamic stream data mining scenarios.

[0061] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. An unsupervised concept drift detection method based on stack autoencoders and Page-Hinckley tests, characterized in that, Includes the following steps: Step 1: Data window partitioning, used to organize individual instances in the data stream into different data windows to support subsequent density estimation of the data distribution within the windows; Step 2: Train a stacked autoencoder representing the distribution to learn the underlying distribution of the data, extract hidden statistical features representing the data distribution, and achieve indirect density estimation of the window data. Step 3: Measurement of distribution difference, using the reconstruction error of the stacked autoencoder as a measure of the distribution difference between window data; Step 4: Adaptive threshold setting. Dynamic thresholds are set through Page-Hinckley testing, and the idea of ​​hypothesis testing is used to report drift when the reconstruction error exceeds the threshold, and the downstream decision model is adjusted quickly.

2. The unsupervised concept drift detection method based on stack autoencoder and Page-Hinckley test as described in claim 1, characterized in that, The data window division specifically includes three types of windows, namely the reference window S. ref Current window S cur With distribution characterization window S single ; The data stream is represented as stream = {X1, X2, ..., X...} n }, where X i ∈R P It is an eigenvector in the P-dimensional feature space at time i; assuming the last time concept drift occurred was t. c Reference window S ref It contains the first set of data to arrive since the last drift occurred, with a window size of w1, i.e. When concept drift occurs again, the reference window should also be updated accordingly; the distribution representation window S single The training set of the stacked autoencoder contains the minimum dataset used to represent the data distribution during training, with a window size of w2. It is continuously slid across the reference window to form the training set. The current window S... cur It always contains the latest w2 data entries, which are initialized as the w2 data entries after the reference window is set, and slides continuously in the direction of the arrival of new data instances using a first-in-first-out (FIFO) mechanism; the reference window represents the reference data distribution, and the current window represents the latest current data distribution.

3. The unsupervised concept drift detection method based on stack autoencoder and Page-Hinckley test according to claim 1, characterized in that, The stacked autoencoder of the training representation distribution utilizes a reference window S ref The data itself serves as a supervisory signal to guide training; when S is filled... ref Then, the distribution representation window S single Initialized to S ref The first data item w2 arrives in the reference window, and the reference window is continuously slid along the direction of data arrival using a FIFO mechanism with a sliding step size of L; as described above, in The training set for the stacked autoencoder is Train, and S corresponds to each value of i. single Different data sets in the train are stretched into one-dimensional vectors and added to the train. in P represents the feature dimension of the original data instance; The stacked autoencoder consists of an encoder and a decoder, with a total of L+1 layers. The input and output dimensions are both D. The value of the j-th neuron in the l-th layer is calculated as follows: In the formula, T n This is a data element in the Train, used as input to the autoencoder; Let j be the j-th component of the input vector. N l-1 The number of neurons in layer (l-1) The weights between layer (l-1) and layer l are... Let σ(z) be the bias of the l-th layer, and σ(z) be the activation function. LeakyReLU is chosen as the activation function, and its expression is as follows: σ(z)=max(0,z)+leak*min(0,z) In the formula, leak is a constant; the training objective of the stacked autoencoder is to make the network input and output infinitely close, that is, to minimize the loss function Dist(T). n This is also known as reconstruction error. The autoencoder training is guided by backpropagation and gradient descent optimization algorithms, with weights and biases updated according to the following formula: In the formula, η is the learning rate, B is the batch size for each training iteration, and the current data batch is {T}. t T t+1 , ..., T t+B-1 This is because training is typically done on a batch of data rather than on a single instance.

4. The unsupervised concept drift detection method based on stack autoencoder and Page-Hinckley test according to claim 1, characterized in that, The distribution difference metric utilizes the data correlation of the autoencoder, assuming a one-to-one correspondence between the trained autoencoder and the reference data distribution represented by the reference window; the current window S is then enabled. cur The data is continuously slid along the direction of arrival using a FIFO mechanism with a step size of L, ensuring that it always contains the latest w2 data entries; S cur The data in the current window is also constructed as a one-dimensional vector and input into the stacked autoencoder. The stacked autoencoder is used to fit the current data distribution and obtain the reconstruction error. If the data in the current window is in the same distribution as the data in the reference window, the statistical information representing the data distribution is correctly extracted and becomes the output of the neural unit of the encoder's hidden layer. The decoder then decodes this hidden statistical information and restores it to be basically consistent with the original data. The reconstruction error will fluctuate around a very small value. Otherwise, the reconstruction error will have a large positive jump.

5. The unsupervised concept drift detection method based on stack autoencoder and Page-Hinckley test according to claim 1, characterized in that, The Page-Hinckley Test (PH Test) is used to dynamically calculate the adaptive threshold. The dynamic threshold setting method combines historical data characteristics to dynamically update the current threshold, making up for the shortcomings of static threshold scenarios in terms of capability, configuration threshold and maintenance cost, and has generalization and adjustability. The original design of the PH Test was to detect abrupt changes in the average value of Gaussian signals. Later, researchers used it for online detection of changes in signal processing. We use the PH Test to determine whether concept drift has occurred by observing the difference between the reconstruction error of the current data block and the overall pattern. If concept drift occurs, the distribution in the current data block changes, and the autoencoder cannot recognize the current distribution state and reconstruct the original data, increasing the reconstruction error. When the reconstruction error exceeds the threshold α at the current time step... t This indicates that data drift has occurred; PH Test defines p i Let p be the current observed value of the random variable p at time i. m represents the average of all observations up to time t. t As a cumulative variable, it stores the cumulative difference between the observed value and the mean of the previous observed values: In the formula, δ is a non-negative real number close to 0, representing the maximum allowable variation; through continuous observation of M t With m t The difference between the values ​​is used to detect whether the mean of the variable suddenly increases. When the difference is greater than a threshold α, the mean is checked. t When the pH test is performed, it will report the changes, including: M t =max{m1,m2,...,m t } In the formula, λ is a hyperparameter that makes α at each time step... t It can automatically adjust based on historical observations; warning thresholds can be defined separately. With drift threshold Reaching the warning threshold indicates a potential change in the data distribution, requiring the use of subsequent incremental data samples to train a new model M. new And maintain; reaching the drift threshold indicates that drift has occurred, using M new Replace the old model to accommodate the new data after the drift.