A concept drift detection method based on bidirectional time series convolution network and multi-stack ensemble learning

Abstract

The application provides a concept drift detection method based on a bidirectional time series convolution network and a multi-stack ensemble learning, and belongs to the concept drift detection field.The method specifically comprises the following steps: step 1, extracting meta features of data in generated concept drift data streams, and dividing the data into a training set and a test set; step 2, taking the training data set obtained in step 1 as the input of a bidirectional time series convolution network model to pre-train the model, and evaluating the performance of the model by using the test set; step 3, performing data cleaning on a real world data set, applying a multi-stack ensemble learning model to predict samples in each timestamp and calculating the average value of error rates of timestamps in the window to obtain feature data; and step 4, inputting the extracted features into the pre-trained bidirectional time series convolution network model to detect concept drift, and retraining the model if concept drift is detected.

Description

Technical Field

[0001] This invention relates to concept drift detection technology, specifically to a concept drift detection method based on bidirectional temporal convolutional networks and multi-stack ensemble learning. Background Technology

[0002] In recent years, the scale of data has experienced explosive growth. Furthermore, the widespread use of new technologies such as the Internet of Things (IoT) and the Industrial Internet, as well as devices like sensors, mobile phones, and tablets, continuously generates data, making multi-source heterogeneous data a major form of data today. In this era of big data, the value of data has reached unprecedented heights. Using data for classification, prediction, and other operations can provide numerous conveniences for people's production and lives. To better utilize data for decision-making, a crucial prerequisite is ensuring the accuracy of the collected data; therefore, anomaly detection has become a major research focus.

[0003] Although many scholars have proposed numerous anomaly detection methods to ensure data accuracy, the complex and ever-changing environment causes data distribution to shift over time, resulting in concept drift. Concept drift not only leads to a decline in model prediction or classification performance but also causes anomaly detection methods to classify data as anomalous. Therefore, it is necessary to detect concept drift in advance to better utilize data in model building and operation, thereby further improving the model's accuracy and robustness.

[0004] Research on concept drift detection originated in the field of statistical quality control. For different types of concept drift phenomena, such as sudden drift, gradual drift, and incremental drift, scholars have successively proposed traditional concept drift detection methods, including those based on data distribution, hypothesis testing, and error rate. In recent years, corresponding improvements to concept drift detection algorithms have emerged, such as: Meta-ADD (Meta-Learning-Based Concept Drift Detection Pre-training Model), Ensemble Learning-Based Concept Drift Detection Algorithm, and Adaptive Random Forest (ARF) algorithm. However, most concept drift detection methods primarily rely on a single learning model to detect and handle a specific type of concept drift phenomenon, failing to effectively capture and analyze long-term, cross-time correlations in time series data, and thus struggling to effectively address different types of concept drift.

[0005] To address the above problems, this invention proposes a concept drift detection model, CD-BTCNMSI, based on bidirectional temporal convolutional networks and multi-stacked ensemble learning, thereby improving the detection accuracy of concept drift models. First, a concept drift data stream with predefined types is generated using a stream ensemble algorithm (SEA), a hyperplane stream generator (HYP), a stochastic radial basis function stream generator (RBF), a stochastic tree stream generator (RTG), and an Agrawal stream generator (AGR), and features are extracted from the data. Then, the extracted features are used to train a BiTCN model as a classifier and detector for concept drift types. In the concept drift detection stage, the real-world dataset is first cleaned, and then the dataset is fed into the multi-stacked ensemble learning model as a data stream for sample prediction and feature extraction. Finally, the extracted features are used as input to the pre-trained BiTCN model for concept drift classification and detection. If concept drift is detected in the current data stream, the multi-stacked ensemble learning model is retrained using the current data stream to better adapt to it. The method proposed in this invention integrates multiple base detection models, which can effectively improve the detection effect of concept drift. At the same time, the use of the BiTCN model can effectively process time series data, making the detection of concept drift more accurate. Summary of the Invention

[0006] To address the problem that most current concept drift detection methods rely on a single machine learning model to detect a certain type of concept drift, which makes it difficult to effectively handle different types of concept drift and cannot effectively process temporal data, this invention proposes a concept drift detection method based on bidirectional temporal convolutional networks and multi-stack ensemble learning.

[0007] This invention provides a concept drift detection method based on bidirectional temporal convolutional networks and multi-stack ensemble learning, comprising:

[0008] Step 1: Generate data streams containing various concept drift types using the stream ensemble algorithm SEA, hyperplane stream generator HYP, random radial basis function stream generator RBF, random tree stream generator RTG, and Agrawal stream generator AGR. Extract meta-features using a fixed-size window. Randomly divide the obtained meta-features into training datasets and test datasets, with the training dataset accounting for 90% of the data and the test dataset accounting for 10% of the data.

[0009] Step 2: Use the training dataset obtained in Step 1 as input to the Bi-Temporal Convolutional Network (BiTCN) model for pre-training. Then, input the test dataset into the trained BiTCN model to obtain detection results and evaluate the model performance using the confusion matrix.

[0010] Step 3: Clean the real-world dataset, apply a multi-stacked ensemble learning model to predict the samples within each time stamp and calculate the average error rate of each time stamp within the window. Then, use the difference in the average error rate of adjacent windows as a dimension of the meta-feature until the specified number of feature dimensions is reached to obtain the feature data.

[0011] Step 4: The extracted features are used as input to the pre-trained BiTCN model for concept drift detection. If the BiTCN model detects concept drift, it updates each classifier in the multi-stacked ensemble learning model to adapt to the current data flow.

[0012] Firstly, the specific steps of step 1 above are as follows:

[0013] Step 1.1: Generate a concept drift data stream with predefined types using the stream integration algorithm stream generator SEA, hyperplane stream generator HYP, random radial basis function stream generator RBF, random tree stream generator RTG, and Agrawal stream generator AGR, and save it in .csv format;

[0014] Step 1.2: Set a fixed-size window, apply a multi-stacked ensemble learning model to predict the samples in the window to obtain the error rate of the data stream in each window, and use the difference in error rates between adjacent windows as a dimension of the meta-feature to obtain data with concept drift type as label and the difference in error rates between adjacent windows as feature;

[0015] Step 1.3: Using uniform random sampling, the obtained error rate data is divided into a training dataset and a test dataset. The training dataset accounts for 90% of the error rate data, and the test dataset accounts for 10% of the error rate data.

[0016] Secondly, the specific steps of step 2 above are as follows:

[0017] Step 2.1: Construct a temporal convolutional network model consisting of residual blocks formed by causal convolution, dilated convolution, and residual connections. This model is abbreviated as TCN. The TCN model uses a 1D fully convolutional network (1D-FCN) architecture to ensure that the length of each hidden layer is the same as the length of the input layer. It also uses common padding to add zeros of length K (kernel size) to previous layers. Furthermore, it introduces dilated convolution, which enables exponentially large receptive fields, allowing for interval sampling of the input during convolution. The sampling rate is controlled by the hyperparameter d, which is often set to 2. n -1, where n is the number of convolutional layers. In the TCN model, F = (f1, f2, ..., f K ) is the set of filter functions corresponding to the model. For a convolution kernel size of K, the input is X = (x1, x2, ..., x...).T The mathematical expression for the dilated convolution function is shown below:

[0018]

[0019] Where F(X) represents the data stream features, F is the set of filter functions for the corresponding model, f1, f2, ..., f K Here, K is the filter function for the corresponding model, K is the kernel size, and X is the initial data flow vector, x1, x2, ..., x... T X is the extracted feature vector, where T is the dimension of the sequence data vector, and X is the feature vector. t-(K-i)d A data stream representing information about the direction of past traffic.

[0020] The TCN model also uses residual connections instead of simple connections between convolutional layers. Within the residual module, the TCN has two layers of dilated causal convolutions and a non-linear mapping of activation functions, and weight normalization and dropout methods are implemented within the module.

[0021] Furthermore, this invention replaces the ReLU activation function in the TCN model with the LeakyReLU activation function shown below to address the gradient problem during backpropagation:

[0022]

[0023] Here, LeakyReLU(x) is the activation function, and a is a parameter introduced by LeakyReLU (the default value is 0.01), which serves as the slope of the function to handle negative inputs.

[0024] The original negative log-likelihood loss function in the TCN model is replaced with the Focal Loss function. A weighting factor α (α∈[0,1]) is introduced into the original Focal Loss function formula to improve it, thereby better suppressing the imbalance between positive and negative samples. The final Focal Loss function is shown below:

[0025]

[0026] FLLoss=-α t (1-p t ) γ log(p t )

[0027] Where p is the probability of correctly predicting the class, and y is the true label value. t To determine the degree of closeness to the correctly predicted label, FLOss is the Focal Loss function, αt γ is a hyperparameter used to harmonize the weight ratio of positive and negative samples, and γ is an adjustable focusing parameter with a range of [0,5].

[0028] Step 2.2, convert the sequence data X = (x1, x2, ..., x...) T The input is fed into the forward TCN to obtain the output result R, and the sequence data X = (x1, x2, ..., x...) is used to generate the output result R. T ) Reversed to X' = (x T ,x T-1 The data (x1, x2, ..., x1) is input into the reverse TCN to obtain the output R'. R and R' are integrated and normalized to obtain a temporal feature that considers both forward and reverse semantics. Based on the finally extracted features, the BiTCN model is pre-trained to obtain the concept drift detection model; where X is the sequence data, X' is the reverse sequence data, and x1, x2, ..., x... T The extracted feature vector is T, where T is the dimension of the sequence data vector.

[0029] Step 2.3: Input the test dataset into the trained BiTCN model to obtain the detection results, and evaluate the model performance using the confusion matrix.

[0030] Thirdly, the specific steps of step 3 above are as follows:

[0031] Step 3.1: Perform data cleaning on the real-world dataset, removing missing and outlier values ​​to improve data quality;

[0032] Step 3.2: Construct a multi-stacked ensemble model with a two-layer stacked structure. The first layer selects six base models: Gaussian Naive Bayes (GaussianNB), Stochastic Gradient Descent (SGD), Perceptron, Random Forest, Gradient Boosting Decision Tree (GBDT), and Extreme Gradient Boosting (XGBoost). These models are trained and predicted on the same data stream training set to maximize the applicability and overall predictive ability of the ensemble model. Then, these predicted values ​​are used as input features for the second-layer meta-model SGD, which outputs the final result.

[0033] Step 3.3: Define a fixed-size window and calculate the average error rate of each timestamp sample within the window using a multi-stacked ensemble learning model, which is the average error rate of the samples within the window.

[0034] Step 3.4: Use the difference in the average error rate of adjacent windows as one dimension of the meta-feature until the specified feature dimension is reached to obtain the feature data.

[0035] Fourthly, the specific steps of step 4 above are as follows:

[0036] Step 4.1: Input the feature data extracted in step 3.4 into the pre-trained BiTCN model to detect concept drift;

[0037] Step 4.2: Use the BiTCN model to detect and classify concept drift phenomena in the data stream. If concept drift is detected, update each classifier in the multi-stacked ensemble model to adapt to the current data stream, and finally obtain the concept drift detection result.

[0038] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0039] 1. The widespread use of the Internet of Things (IoT) and the Industrial Internet results in the collection of various types of concept drift phenomena in the data. However, traditional concept drift detection models can only detect single types of concept drift. To address this, this invention introduces a multi-stacked ensemble learning model that replaces a single learner with six base learners, including GussianNB, SGD, Perceptron, Random Forest, GBDT, and XGBoost. By stacking, different types of base learners can be used to increase the diversity of the model, helping to capture different patterns in the data and reduce the detection bias of the model. This improves the performance and generalization ability of the model in detecting concept drift, ultimately providing an effective concept drift detection framework for IoT and Industrial Internet applications.

[0040] 2. Data generated by applications such as the Internet of Things (IoT) and the Industrial Internet (IIoT) possesses strong long-term dependencies and bidirectional semantic features. However, existing concept drift detection models cannot effectively extract the long-term dependencies of the data and rarely consider the bidirectional semantic features within the data. To address this, this invention utilizes a bidirectional temporal convolutional network (BiTCN) as a concept drift detector, improving the detection performance by capturing the bidirectional semantic features of the data. Furthermore, the ReLU activation function in the TCN model is replaced with the LeakyReLU activation function to avoid the "death" problem of neurons, and the negative log-likelihood loss function is replaced with the FocalLoss loss function to solve the problem of the model being overly biased towards the majority class and neglecting the minority class, resulting in low classification accuracy. This further improves the detection performance of concept drift and ultimately provides effective data support for IoT and IIoT applications. Attached Figure Description

[0041] Figure 1 This is the overall flowchart of the detection model CD-BTCNMSI of this invention.

[0042] Figure 2 This is a detailed flowchart of the detection model CD-BTCNMSI of this invention.

[0043] Figure 3 The results are the detection results of the CD-BTCNMSI model of this invention, as well as the four models META-ADD, ENHASH, STUDD and CDSCDE, on the ELEC dataset.

[0044] Figure 4 The results are the detection results of the CD-BTCNMSI model of this invention, as well as the four models META-ADD, ENHASH, STUDD and CDSCDE, on the SPAM dataset.

[0045] Figure 5 The results are the detection results of the CD-BTCNMSI model of this invention, as well as the four models META-ADD, ENHASH, STUDD and CDSCDE, on the NOAA dataset. Detailed Implementation

[0046] The present invention will be further described below with reference to the accompanying drawings and embodiments. It should be noted that the described embodiments are only intended to facilitate the understanding of the present invention and do not constitute any limitation thereof.

[0047] This invention addresses the problem that most current concept drift detection methods rely on a single machine learning model to detect a certain type of concept drift, making it difficult to effectively handle different types of concept drift and to process temporal data. It proposes a concept drift detection method based on bidirectional temporal convolutional networks and multi-stacked ensemble learning to effectively detect concept drift in data. Extensive experiments demonstrate the feasibility and effectiveness of the model.

[0048] like Figure 1 and Figure 2 As shown, the concept drift detection method proposed in this invention based on bidirectional temporal convolutional networks and multi-stack ensemble learning includes:

[0049] Step 201 uses the Stream Integration Algorithm Stream Generator (SEA), Hyperplane Stream Generator (HYP), Random Radial Basis Function Stream Generator (RBF), Random Tree Stream Generator (RTG), and Agrawal Stream Generator (AGR) to generate data streams containing various concept drift types. Meta-features are extracted using a fixed-size window, and the obtained meta-features are randomly divided into training datasets and test datasets, with the training dataset accounting for 90% of the data and the test dataset accounting for 10%.

[0050] Step 2011 uses the Stream Integration Algorithm Stream Generator SEA, Hyperplane Stream Generator HYP, Random Radial Basis Function Stream Generator RBF, Random Tree Stream Generator RTG, and Agrawal Stream Generator AGR to generate a concept drift data stream with predefined types, and saves it in .csv format;

[0051] Step 2012 sets a fixed-size window, applies a multi-stacked ensemble learning model to predict the samples in the window to obtain the error rate of the data stream in each window, and uses the difference in error rates between adjacent windows as a dimension of the meta-feature to obtain data with concept drift type as label and the difference in error rates between adjacent windows as feature;

[0052] Step 2013 uses a uniform random sampling method to divide the obtained error rate data into a training dataset and a test dataset. The training dataset accounts for 90% of the error rate data, and the test dataset accounts for 10% of the error rate data.

[0053] Step 202 uses the training dataset obtained in Step 201 as input to the Bidirectional Temporal Convolutional Network (BiTCN) model for pre-training. The test dataset is then input into the trained BiTCN model to obtain detection results, and the model performance is evaluated using a confusion matrix.

[0054] Step 2021 constructs a temporal convolutional network model consisting of residual blocks stacked together, including causal convolution, dilated convolution, and residual connections. This temporal convolutional network model is abbreviated as TCN model. The TCN model uses a 1D fully convolutional network (1D-FCN) architecture to ensure that the length of each hidden layer is the same as the length of the input layer. It also uses common padding to add zeros of length K (kernel size) to previous layers. Furthermore, it introduces dilated convolution, which enables exponentially large receptive fields, allowing for interval sampling of the input during convolution. The sampling rate is controlled by the hyperparameter d, which is often set to 2. n -1, where n is the number of convolutional layers. In the TCN model, F = (f1, f2, ..., f K ) is the set of filter functions corresponding to the model. For a convolution kernel size of K, the input is X = (x1, x2, ..., x...). T The mathematical expression for the dilated convolution function is shown below:

[0055]

[0056] Where F(X) represents the data stream features, F is the set of filter functions for the corresponding model, f1, f2, ..., f K Here, K is the filter function for the corresponding model, K is the kernel size, and X is the initial data flow vector, x1, x2, ..., xn. T X is the extracted feature vector, where T is the dimension of the sequence data vector, and X is the feature vector. t-(K-i)d A data stream representing information about the direction of past traffic.

[0057] The TCN model also uses residual connections instead of simple connections between convolutional layers. Within the residual module, the TCN has two layers of dilated causal convolutions and a non-linear mapping of activation functions, and weight normalization and dropout methods are implemented within the module.

[0058] Furthermore, since the ReLU activation function only exhibits a linear relationship, it converges quickly. However, when x < 0, the neuron's gradient is set to 0, causing the neuron to never be activated by any function again. Therefore, this invention replaces the ReLU activation function with the LeakyReLU activation function. This function introduces a small positive number 'a' as the slope of the ReLU activation function to handle negative inputs, thus solving the gradient problem in TCN during backpropagation. The formula for the LeakyReLU activation function is shown below:

[0059]

[0060] Here, LeakyReLU(x) is the activation function, and a is a parameter introduced by LeakyReLU (the default value is 0.01), which serves as the slope of the function to handle negative inputs.

[0061] The original negative log-likelihood loss function in the TCN model is replaced with the Focal Loss function. A weighting factor α (α∈[0,1]) is introduced into the original Focal Loss function formula to improve it, thereby better suppressing the imbalance between positive and negative samples. The final Focal Loss function is shown below:

[0062]

[0063] FLLoss=-α t (1-p t ) γ log(p t )

[0064] Where p is the probability of correctly predicting the class, and y is the true label value. t To determine the degree of closeness to the correctly predicted label, FLOss is the Focal Loss function, α t γ is a hyperparameter used to harmonize the weight ratio of positive and negative samples, and γ is an adjustable focusing parameter with a range of [0,5].

[0065] Step 2022: The sequence data X = (x1, x2, ..., x...) T The input is fed into the forward TCN to obtain the output result R, and the sequence data X = (x1, x2, ..., x...) is used to generate the output result R. T) Reversed to X' = (x T ,x T-1 The data (x1, x2, ..., x1) is input into the reverse TCN to obtain the output R'. R and R' are integrated and normalized to obtain a temporal feature that considers both forward and reverse semantics. Based on the finally extracted features, the BiTCN model is pre-trained to obtain the concept drift detection model; where X is the sequence data, X' is the reverse sequence data, and x1, x2, ..., x... T The extracted feature vector is T, where T is the dimension of the sequence data vector.

[0066] Step 2023 involves inputting the test dataset into the trained BiTCN model to obtain detection results, and evaluating the model performance using the confusion matrix.

[0067] Step 203 involves cleaning the real-world dataset, applying a multi-stacked ensemble learning model to predict samples within each timestamp and calculating the average error rate of each timestamp within that window. The difference in the average error rate of adjacent windows is then used as a dimension of the meta-feature until the specified number of feature dimensions is reached, thus obtaining the feature data.

[0068] The purpose of preprocessing the real-world dataset in the embodiments of this invention is that real-world datasets may contain missing values ​​and outliers due to various reasons during the data collection process. Therefore, it is necessary to preprocess the real-world data before training the detection model.

[0069] Step 2031 involves cleaning the real-world dataset by removing missing and outlier values ​​to improve data quality.

[0070] Step 2032 constructs a multi-stacked ensemble model with a two-layer stacked structure. The first layer selects six base models: Gaussian Naive Bayes (GaussianNB), Stochastic Gradient Descent (SGD), Perceptron, Random Forest, Gradient Boosting Decision Tree (GBDT), and Extreme Gradient Boosting (XGBoost). These models are trained and predicted on the same data stream training set to maximize the applicability and overall predictive ability of the ensemble model. Then, these predicted values ​​are used as input features to the second-layer meta-model SGD, which outputs the final result y.

[0071] Step 2033 defines a fixed-size window and calculates the average error rate of each timestamp sample within the window using a multi-stacked ensemble learning model, which is the average error rate of the samples within the window.

[0072] Step 2034 uses the difference in the average error rate of adjacent windows as a dimension of the meta-feature until the specified feature dimension is reached, thus obtaining the feature data.

[0073] Step 204 uses the extracted features as input to the pre-trained BiTCN model for concept drift detection. If the BiTCN model detects concept drift, it updates each classifier in the multi-stacked ensemble learning model to adapt to the current data stream.

[0074] Step 2041: Input the feature data extracted in step 2034 into the pre-trained BiTCN model to detect concept drift;

[0075] Step 2042 uses the BiTCN model to detect and classify concept drift phenomena generated in the data stream. If concept drift phenomena are detected, the classifiers in the multi-stacked ensemble model are updated to adapt to the current data stream, and finally the concept drift detection results are obtained.

[0076] This invention focuses on concept drift detection in real-world data streams. It proposes a concept drift detection method based on bidirectional temporal convolutional networks and multi-stack ensemble learning, and tests it using the ELEC, SPAM, and NOAA datasets. The ELEC dataset is from the New South Wales electricity market in Australia, containing 45,312 samples, each with 9 features. The SPAM dataset is an email dataset from the Spam AssassinCollection containing natural concept drift, with 9,234 samples and 501 features per sample. The NOAA meteorological dataset is a set of meteorological data maintained by the National Oceanic and Atmospheric Administration (NOAA), including observational data from meteorological stations around the world, as well as remote sensing data from satellites and radar. NOAA provides various datasets for researchers, including global meteorological datasets, US meteorological datasets, and ocean datasets. This invention uses one of these datasets, noaa_042700, which contains 27,594 samples, each with 8 features.

[0077] The concept drift detection model CD-BTCNMSI, based on bidirectional temporal convolutional networks and multi-stack ensemble learning, proposed in this invention is compared with the concept drift detection pre-trained model META-ADD based on meta-learning, the fast ensemble concept drift detection stream algorithm ENHASH, the unsupervised concept drift detection model STUDD based on the teacher-student learning paradigm, and the concept drift and covariate shift detection ensemble algorithm CDSCDE. The detection capabilities of these models are illustrated by calculating the average detection efficiency (including precision, recall, F1-measure, and accuracy) of the five models.

[0078] Figure 3 This demonstrates the detection performance of five models on the ELEC dataset, from... Figure 3 It can be seen that the CD-BTCNMSI model proposed in this invention achieves the highest detection accuracy on the concept drift dataset. However, the STUDD model has the worst detection accuracy, which is because the STUDD model did not retrain and adapt in a timely manner to the concept drift present in the dataset. Compared with META-ADD, the CD-BTCNMSI model improves all four metrics, which also verifies that the BiTCN model proposed in this invention can grasp the temporal features of real-world data and improve the detection accuracy of concept drift through multi-stacked model ensemble. Experimental results demonstrate that the CD-BTCNMSI model can achieve higher detection performance by ensemble learning and capturing the temporal features of real-world traffic.

[0079] Figure 4 This demonstrates the detection performance of five models on the SPAM dataset. Figure 4 It can be seen that the CD-BTCNMSI model proposed in this invention achieves the highest accuracy in detecting concept drift, reaching 94.94%. Concept drift in real-world datasets often lacks direct labels and is generally difficult to detect, but the CD-BTCNMSI model proposed in this invention still demonstrates excellent detection performance for concept drift. Figure 4 It can be seen that, compared with other concept drift detection models, the CD-BTCNMSI model has the highest accuracy across all four metrics. Compared with the META-ADD, ENHASH, STUDD, and CDSCDE models, its accuracy is improved by 2.41%, 15.63%, 6.83%, and 1.32%, respectively.

[0080] Figure 5 This demonstrates the detection performance of five models on the NOAA dataset. Figure 5 It can be seen that the CD-BTCNMSI model proposed in this invention has the highest accuracy in detecting concept drift. It improves accuracy by about 7.61% compared with the META-ADD model and recall by 5.53% compared with ENHASH.

[0081] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0082] Although embodiments of the invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

Claims

1. A concept drift detection method based on bidirectional temporal convolutional networks and multi-stack ensemble learning, characterized in that, Includes the following steps: Step 1: Generate data streams containing various concept drift types using the stream ensemble algorithm SEA, hyperplane stream generator HYP, random radial basis function stream generator RBF, random tree stream generator RTG, and Agrawal stream generator AGR. Extract meta-features using a fixed-size window. Randomly divide the obtained meta-features into training datasets and test datasets, with the training dataset accounting for 90% of the data and the test dataset accounting for 10% of the data. Step 2: Use the training dataset obtained in Step 1 as input to the Bi-Temporal Convolutional Network (BiTCN) model for pre-training. Then, input the test dataset into the trained BiTCN model to obtain detection results and evaluate the model performance using the confusion matrix. Step 3: Clean the real-world dataset, apply a multi-stacked ensemble learning model to predict the samples within each time stamp and calculate the average error rate of each time stamp within the window. Then, use the difference in the average error rate of adjacent windows as a dimension of the meta-feature until the specified number of feature dimensions is reached to obtain the feature data. Step 4: The extracted features are used as input to the pre-trained BiTCN model for concept drift detection. If the BiTCN model detects concept drift, it updates each classifier in the multi-stacked ensemble learning model to adapt to the current data flow.

2. The method as described in claim 1, characterized in that, The specific implementation of step 1 includes the following steps: Step 1.1: Generate a concept drift data stream with predefined types using the stream integration algorithm SEA, hyperplane stream generator HYP, random radial basis function stream generator RBF, random tree stream generator RTG, and Agrawal stream generator AGR, and save it in .csv format; Step 1.2: Set a fixed-size window, apply a multi-stacked ensemble learning model to predict the samples in the window to obtain the error rate of the data stream in each window, and use the difference in error rates between adjacent windows as a dimension of the meta-feature to obtain data with concept drift type as label and the difference in error rates between adjacent windows as feature; Step 1.3: Using uniform random sampling, the obtained error rate data is divided into a training dataset and a test dataset. The training dataset accounts for 90% of the error rate data, and the test dataset accounts for 10% of the error rate data.

3. The method as described in claim 1, characterized in that, The specific implementation of step 2 includes the following steps: Step 2.1: Construct a temporal convolutional network model consisting of residual blocks composed of causal convolution, dilated convolution, residual connections, etc. The temporal convolutional network model is referred to as the TCN model. At the same time, replace the ReLU activation function of the TCN model with the LeakyReLU activation function and replace the negative log-likelihood loss function with the Focal Loss loss function. Step 2.2, convert the sequence data X = (x1, x2, ..., x...) T The input is given to the forward TCN model to obtain the output result R. The sequence data X = (x1, x2, ..., x...) is used to obtain the output result R. T ) Reversed to X' = (x T ,x T-1 The inputs (x1, x2, ..., x1) are fed into the inverse TCN model to obtain the output R'. R and R' are integrated and normalized to obtain a temporal feature that considers both positive and negative semantics. Based on the finally extracted features, the BiTCN model is pre-trained to obtain the concept drift detection model; where x1, x2, ..., x1) are the inputs (x1, x2, ..., x1) and the output R' is obtained. T The extracted feature vector is T, where T is the dimension of the sequence data vector. Step 2.3: Input the test dataset into the trained BiTCN model to obtain the detection results, and evaluate the model performance using the confusion matrix.

4. The method as described in claim 1, characterized in that, The specific implementation of step 3 includes the following steps: Step 3.1: Perform data cleaning on the real-world dataset, removing missing and outlier values ​​to improve data quality; Step 3.2: Construct a multi-stacked ensemble model with a two-layer stacked structure. The first layer selects six base models: Gaussian Naive Bayes (GaussianNB), Stochastic Gradient Descent (SGD), Perceptron, Random Forest, Gradient Boosting Decision Tree (GBDT), and Extreme Gradient Boosting (XGBoost). These models are trained and predicted on the same data stream training set to maximize the applicability and overall predictive ability of the ensemble model. Then, these predicted values ​​are used as input features for the second-layer meta-model SGD, which outputs the final result. Step 3.3: Define a fixed-size window and calculate the average error rate of each timestamp sample within the window using a multi-stacked ensemble learning model, which is the average error rate of the samples within the window. Step 3.4: Use the difference in the average error rate of adjacent windows as one dimension of the meta-feature until the specified feature dimension is reached to obtain the feature data.

5. The method as described in claim 1, characterized in that, The specific implementation of step 4 includes the following steps: Step 4.1: Input the feature data extracted in step 3.4 into the pre-trained BiTCN model to detect concept drift; Step 4.2: Use the BiTCN model to detect and classify concept drift phenomena in the data stream. If concept drift is detected, update each classifier in the multi-stacked ensemble model to adapt to the current data stream, and finally obtain the concept drift detection result.

Images