Data stream concept drift learning method fusing adaptive sample screening

Abstract

This invention belongs to the field of data mining and machine learning, specifically a data stream concept drift learning method and apparatus that integrates adaptive sample selection. It includes: S1: Standardizing and relabeling the streaming data, and calculating corresponding weights based on the number of samples in each category; S2: Real-time monitoring of changes in the distribution of the streaming data and determining the type of data drift; S3: Constructing a neural network model for feature learning and classification prediction; S4: Dynamically adjusting the selection threshold according to the data drift type to filter samples and extracting high-confidence samples to expand the training set; S5: Using the expanded and optimized final training sample set, continuously training and updating the neural network model based on a dynamic class-weighted loss function, and using the weights calculated in S1, applying a weighted penalty mechanism to impose a greater penalty on misclassification of minority class samples during training.

Description

Technical Field

[0001] This invention belongs to the field of data mining and machine learning, specifically a data flow concept drift learning method that integrates adaptive sample selection. Background Technology

[0002] With the digital transformation of society, streaming data is generated in real time in many fields such as industrial production, intelligent transportation, and weather forecasting. Streaming data is characterized by its non-linearity, continuity, and speed; its distribution evolves over time, a phenomenon known as concept drift. The existence of concept drift significantly increases the difficulty of data analysis and model building.

[0003] Traditional concept drift classification algorithms often fail to adequately consider other features of the input data, such as binary and multi-class classification problems in different scenarios, and data with binary features. This leads to poor performance when handling complex datasets. Existing methods are mainly divided into block-based and online learning-based approaches. Block-based methods require storing multiple data blocks, consuming a large amount of memory, and struggle to react to subtle drifts occurring within a single block. While online learning-based methods are fast, they are susceptible to noise contamination and may fall into overfitting because samples are processed only once. Therefore, a learning method that can effectively handle concept drift in different scenarios while balancing efficiency and accuracy is needed. Summary of the Invention

[0004] To address the aforementioned problems, this invention provides a data flow concept drift learning method that integrates adaptive sample selection.

[0005] This invention adopts the following technical solution: a data flow concept drift learning method that integrates adaptive sample selection, comprising: S1: Standardize and relabel the streaming data, and calculate the corresponding weights based on the number of samples in each category of the streaming data; S2: Monitor changes in the distribution of streaming data in real time and determine the type of streaming data drift; S3: Construct a neural network model for feature learning and classification prediction; S4: Dynamically adjust the filtering threshold according to the drift type of the streaming data, filter the samples, and extract high-confidence samples to expand the training set. S5: Using the expanded and optimized final training sample set, the neural network model is continuously trained and updated based on the dynamic class weighted loss function. Using the weights calculated in S1, the model is penalized more severely for misclassification of minority class samples during training through a weighted penalty mechanism.

[0006] In some embodiments, S1 includes: S101: Standardize and remap the streaming data. Standardize the original feature values ​​of the streaming data and scale them to a distribution with a mean of 0 and a standard deviation of 1. S102: Identify the target label category to which the samples in the streaming data belong, and calculate the corresponding weight based on the number of samples in each target label category. , It is the weight of the i-th class. N is the number of samples in class i, N is the total number of samples, and K is the total number of classes.

[0007] In some embodiments, S2 includes: S201: Construct a reference dataset based on the most recently stable data blocks stored in the historical window, and calculate the mean of the reference distribution. and standard deviation ; S202: Divide the streaming data into fixed-size data blocks. For the data block to be detected at the current time, obtain the feature value x of the i-th sample in the current data block. i Calculate the standardized difference between this eigenvalue and the mean of the reference distribution. ; S203: For the i-th sample in the current data block, calculate its statistic. : , and Let represent the positive and negative cumulative sum statistics of the i-th sample, respectively; and This represents the corresponding statistic for the (i-1)th sample; This is the standardized difference calculated by S202; These are drift boundary parameters, used to set the benchmark for drift detection; S204: Drift Score Based Determine whether concept drift has occurred; S205: For a data block where concept drift is determined to have occurred, first mark the first and last potential drift points within the data block as the drift start points. and end point This allows us to determine the location of the drift and calculate its range. Next, the drift severity coefficient is calculated. If the drift range L is less than 10% of the total number of samples in the current data block and If the result is a sudden change, it is considered a sudden drift; otherwise, it is considered a gradual drift.

[0008] In some embodiments, in S204, if the threshold is exceeded... If the number of features is less than 1 / 3 of the total number of features, then no concept drift has occurred; otherwise, it is determined that concept drift has occurred.

[0009] In some embodiments, S3, the neural network model includes: The input layer is used to input features; Batch normalization layers are used to normalize the input features to solve the gradient vanishing or exploding problem that may occur in high-dimensional data training and accelerate network convergence. The embedding layer is designed for datasets containing binary features. This layer is not built for datasets with non-binary features. It converts discrete binary features into continuous vectors. The main classifier layer includes a first fully connected layer, a ReLU activation function, a Dropout layer, and a second fully connected layer connected in sequence. The first fully connected layer maps the input features to a 128-dimensional hidden space, activates them through the ReLU activation function, prevents overfitting through the Dropout layer, and finally maps them to the final output dimension by the second fully connected layer.

[0010] In some embodiments, S4 includes: S401: For binary or multi-class classification problems, calculate the predicted probability of a sample. If the predicted probability is greater than the screening threshold, retain the sample as a high-confidence sample set. ; S402: Drift type and severity coefficient determined based on S2 Dynamically adjust the filtering threshold; S403: The number of samples in the sample set within the current data block based on the target label category j. Total number of samples in the current data block n, sampling ratio and sampling priority Calculate the number of samples that need to be drawn from historical samples to determine the target label category. ; S404: Based on the calculated sample size From the selected high-confidence sample set Randomly select samples of the corresponding category from the data; if there are not enough samples of a certain category, then select all samples of that category; subsequently, stack the selected samples vertically with the current samples in the current data block to form the final expanded and optimized training sample set. .

[0011] In some embodiments, S401 includes: For binary classification problems, use the Sigmoid function to calculate the probability: , The output for the positive class of the model; For multi-class classification problems, use the SoftMax function to calculate the probabilities: , Let C be the output of the model for the k-th class, and C be the total number of classes.

[0012] In some embodiments, S402 includes: If it is determined to be a mutation drift, a higher screening threshold should be set to reduce the participation of historical samples that are not adapted to the current distribution: ; If the distribution is determined to be a gradual shift, a lower screening threshold is set to increase the participation of historical samples similar to the current distribution: .

[0013] In some embodiments, in S403, the number of samples .

[0014] In some embodiments, S5 includes: S501: Construct a dynamic weighted loss function. Based on binary or multi-class classification, construct a loss function that includes weights and directly integrate the weights into the cross-entropy calculation. For binary classification problems, the category-weighted binary cross-entropy loss function is used. For multi-class classification problems, the category-weighted cross-entropy loss function is used; S502: Perform iterative training and parameter updates based on mixed precision, and calculate the loss value of the current batch by combining a dynamic weighted loss function during iterative training and parameter updates; S503: The scheduler is used to adaptively adjust the learning rate.

[0015] Compared with the prior art, the present invention has the following beneficial effects: This invention employs an improved window accumulation algorithm to accurately detect the location, range, and type of concept drift in real time, and adaptively adjusts the model based on the detection results. This allows the model to respond quickly to changes in data distribution, avoiding performance degradation caused by outdated data. Experiments show that in tests across multiple different domains, this method maintains higher classification accuracy when facing both abrupt and gradual drift.

[0016] This invention introduces a batch normalization layer into the neural network model for high-dimensional data, effectively solving the vanishing or exploding gradient problem and accelerating network convergence. Simultaneously, for datasets with binary features, an innovative embedding layer is added, increasing learnable parameters to the model and significantly improving its generalization ability and robustness. The fusion of these technical features enables this method to exhibit superior performance compared to existing techniques when processing complex and diverse streaming data.

[0017] This invention incorporates multiple optimization strategies during model training, including a learning rate optimizer and a scheduler, which dynamically adjust the learning rate according to the needs of the classification task, ensuring rapid and stable convergence of model parameters. Compared with traditional algorithms, this method significantly reduces training time while maintaining high accuracy, thereby improving overall computational efficiency. Attached Figure Description

[0018] Figure 1 This is a general framework diagram of the learning method proposed in this invention to address the phenomenon of data flow concept drift; Figure 2 The window summation algorithm proposed in this invention determines whether a data flow concept drift phenomenon occurs; Figure 3 This is a schematic diagram of the main classifier neural network model of the method proposed in this invention; Figure 4 This is a schematic diagram of the adaptive sample processing strategy proposed in this invention. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below. Obviously, the described embodiments are some embodiments of the present invention, but not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] A dataflow concept drift learning method incorporating adaptive sample selection includes: S1: Standardize and relabel the streaming data, and calculate the corresponding weights based on the number of samples in each category of the streaming data; S2: Monitor changes in the distribution of streaming data in real time and determine the type of streaming data drift; S3: Construct a neural network model for feature learning and classification prediction; S4: Dynamically adjust the filtering threshold according to the drift type of the streaming data, filter the samples, and extract high-confidence samples to expand the training set. S5: Using the expanded and optimized final training sample set, the neural network model is continuously trained and updated based on the dynamic class weighted loss function. Using the weights calculated in S1, the model is penalized more severely for misclassification of minority class samples during training through a weighted penalty mechanism.

[0021] Specifically, step S1 includes: S101: Standardize and relabel the streaming data. Standardize the raw feature values ​​of the streaming data (i.e., unprocessed sample attribute data received in real time from the data stream, such as real-time observed indicators like equipment temperature, workshop humidity, or product defect rate on an industrial production line) and scale them to a distribution with a mean of 0 and a standard deviation of 1. (x is the original feature value mentioned above, The characteristic mean, (characteristic standard deviation); S102: Identify the target label category to which samples belong in the streaming data, and calculate the corresponding weights based on the number of samples in each category to address the target label category imbalance problem in the streaming data. , It is the weight of the i-th class. is the number of samples in class i, N is the total number of samples, and K is the total number of classes, in order to solve the class imbalance problem that may exist in the dataset.

[0022] Specifically, S2 includes: S201: Construct a reference dataset based on the most recently stable data blocks stored in the historical window (which is a sliding window that stores several historical data blocks that have been determined to be stable and have not experienced concept drift), and calculate the mean of the reference distribution. and standard deviation By constructing a reference distribution through historical windows, it can be ensured that the benchmark used for comparison represents the characteristic distribution of the data stream in a stationary state.

[0023] S202: Divide the streaming data into fixed-size blocks. For the data block to be detected at the current time, obtain the feature value x of the i-th sample in the current data block, and calculate the standardized difference between this feature value and the mean of the reference distribution. .

[0024] S203: For the i-th sample in the current data block (i.e., the data instance with index i within the block), calculate its statistic. : , and Let represent the positive and negative cumulative sum statistics of the i-th sample, respectively; and This represents the corresponding statistic for the (i-1)th sample (initially 0). It is the standardized difference calculated in step S202; These are drift boundary parameters, used to set the benchmark for drift detection.

[0025] S204: Drift Score Based Determine whether concept drift has occurred; In S204, if the threshold is exceeded If the number of features is less than 1 / 3 of the total number of features, then no concept drift has occurred; otherwise, it is determined that concept drift has occurred.

[0026] S205: For data blocks where concept drift is determined to have occurred, first mark the first and last potential drift points within the block as the drift start points. and end point This allows us to determine the location of the drift and calculate its range. Next, the drift severity coefficient is calculated. If the drift range L is less than 10% of the total number of samples in the current data block and If the result is a sudden change, it is considered a sudden drift; otherwise, it is considered a gradual drift.

[0027] Specifically, in S3, the neural network model includes: an input layer for input features; a batch normalization layer for normalizing the input features to address the vanishing or exploding gradient problems that may occur during training with high-dimensional data, thus accelerating network convergence; an embedding layer, for datasets containing binary features (such as the switching state of a circuit, logic values, or other discrete features), an optional embedding layer is added to the batch normalization layer (this layer is not built for datasets with non-binary features), converting discrete binary features into continuous vectors; and a main classifier layer, which includes a first fully connected layer, a ReLU activation function, a Dropout layer, and a second fully connected layer connected in sequence. The first fully connected layer maps the input features to a 128-dimensional hidden space, activates them using the ReLU activation function, prevents overfitting using the Dropout layer, and finally maps them to the final output dimension (1 for binary classification and C for multi-class classification).

[0028] Specifically, S4 includes: S401: For binary or multi-class classification problems, calculate the predicted probability of a sample. If the predicted probability is greater than the screening threshold, retain the sample as a high-confidence sample set. ; S401 includes: For binary classification problems, use the Sigmoid function to calculate the probability: The formula The output for the positive class of the model; For multi-class classification problems, use the SoftMax function to calculate the probabilities: The formula Let C be the output of the model for the k-th class, and C be the total number of classes.

[0029] S402: Based on the drift type and severity coefficient determined in step S2 Dynamically adjust the filtering threshold ; S402 includes: If it is determined to be a mutation drift, a higher screening threshold should be set to reduce the participation of historical samples that are not adapted to the current distribution: ; If the distribution is determined to be a gradual shift, a lower screening threshold is set to increase the participation of historical samples similar to the current distribution: .

[0030] S403: The number of samples in the sample set within the current data block based on the target label category j. Total number of samples in the current data block n, sampling ratio and sampling priority Calculate the number of samples that need to be drawn from historical samples for this category. Sample size .

[0031] S404: Based on the calculated sample size From the selected high-confidence sample set Randomly select samples of the corresponding category from the data; if there are not enough samples of a certain category, then select all samples of that category; subsequently, stack the selected samples vertically with the current samples in the current data block to form the final expanded and optimized training sample set. .

[0032] Specifically, S5 includes: S501: Construct a dynamically weighted loss function. Based on the type of classification task (binary or multi-class), construct a loss function that includes class weights, directly incorporating the weights into the cross-entropy calculation: For binary classification problems, the class-weighted binary cross-entropy loss function is used. For problems involving... For a batch of samples, the loss function is defined as: in, For the sample The true label, The original values ​​output by the model. It is the Sigmoid activation function. The weight corresponding to the actual label category (calculated in step S1).

[0033] For multi-class classification problems, the class-weighted cross-entropy loss function is used. The loss function is defined as: in, The total number of categories, For the first Class weight, As an indicator variable (if the sample) Category (The value is 1 if it is 1, otherwise it is 0). The model predicts that the sample belongs to the first... The probability of a class.

[0034] S502: Perform iterative training and parameter updates based on mixed precision. Specifically, this includes: initializing the AdamW optimizer and setting the initial learning rate. (0.001 in this example) and weight decay coefficient (0.01 in this embodiment). In each training batch, the gradients accumulated by the optimizer are first cleared to zero; then, the mixed precision context (AutoCast) is enabled, the training samples are input into the neural network model for forward propagation to obtain the predicted output, and the loss value of the current batch is calculated using the dynamically weighted loss function constructed by S501; next, the loss value is scaled using the gradient scaler to prevent numerical underflow, and backpropagation is performed to calculate the gradient of the model parameters; finally, the model parameters are updated by the AdamW optimizer based on the scaled gradient, and the scaling factor of the gradient scaler is updated.

[0035] S503: Employs the ReduceLROnPlateau scheduler for adaptive adjustment of the learning rate. At the end of each training epoch, the total loss value for that epoch is calculated. If the total loss value is continuously... Each round (in this embodiment) If no decrease occurs within 5 minutes, the model is considered to have entered a plateau. The scheduler automatically reduces the current learning rate (to 0.1 times the original learning rate) to reduce the step size of parameter updates, helping the model escape local optima and achieve more refined convergence.

[0036] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A dataflow concept drift learning method integrating adaptive sample selection, characterized in that, include: S1: Standardize and relabel the streaming data, and calculate the corresponding weights based on the number of samples in each category of the streaming data; S2: Monitor changes in the distribution of streaming data in real time and determine the type of streaming data drift; S3: Construct a neural network model for feature learning and classification prediction; S4: Dynamically adjust the filtering threshold according to the drift type of the streaming data, filter the samples, and extract high-confidence samples to expand the training set. S5: Using the expanded and optimized final training sample set, the neural network model is continuously trained and updated based on the dynamic class weighted loss function. Using the weights calculated in S1, the model is penalized more severely for misclassification of minority class samples during training through a weighted penalty mechanism.

2. The data flow concept drift learning method based on adaptive sample selection according to claim 1, characterized in that, S1 includes: S101: Standardize and remap the streaming data. Standardize the original feature values ​​of the streaming data and scale them to a distribution with a mean of 0 and a standard deviation of 1. S102: Identify the target label category to which the samples in the streaming data belong, and calculate the corresponding weight based on the number of samples in each target label category. , It is the weight of the i-th class. N is the number of samples in class i, N is the total number of samples, and K is the total number of classes.

3. The data flow concept drift learning method based on adaptive sample selection according to claim 1, characterized in that: S2 includes: S201: Construct a reference dataset based on the most recently stable data blocks stored in the historical window, and calculate the mean of the reference distribution. and standard deviation ; S202: Divide the streaming data into fixed-size data blocks. For the data block to be detected at the current time, obtain the feature value x of the i-th sample in the current data block. i Calculate the standardized difference between this eigenvalue and the mean of the reference distribution. ; S203: For the i-th sample in the current data block, calculate its statistic. : , and Let represent the positive and negative cumulative sum statistics of the i-th sample, respectively; and This represents the corresponding statistic for the (i-1)th sample; This is the standardized difference calculated by S202; These are drift boundary parameters, used to set the benchmark for drift detection; S204: Drift Score Based Determine whether concept drift has occurred; S205: For a data block where concept drift is determined to have occurred, first mark the first and last potential drift points within the data block as the drift start points. and end point This allows us to determine the location of the drift and calculate its range. Next, the drift severity coefficient is calculated. If the drift range L is less than 10% of the total number of samples in the current data block and If the result is a sudden change, it is considered a sudden drift; otherwise, it is considered a gradual drift.

4. The data flow concept drift learning method based on adaptive sample selection according to claim 3, characterized in that: In step S204, if the threshold is exceeded... If the number of features is less than 1 / 3 of the total number of features, then no concept drift has occurred; otherwise, it is determined that concept drift has occurred.

5. The data flow concept drift learning method based on adaptive sample selection according to claim 1, characterized in that: In S3, the neural network model includes: The input layer is used to input features; Batch normalization layers are used to normalize the input features to solve the gradient vanishing or exploding problem that may occur in high-dimensional data training and accelerate network convergence. The embedding layer is designed for datasets containing binary features. This layer is not built for datasets with non-binary features. It converts discrete binary features into continuous vectors. The main classifier layer includes a first fully connected layer, a ReLU activation function, a Dropout layer, and a second fully connected layer connected in sequence. The first fully connected layer maps the input features to a 128-dimensional hidden space, activates them through the ReLU activation function, prevents overfitting through the Dropout layer, and finally maps them to the final output dimension by the second fully connected layer.

6. The data flow concept drift learning method based on adaptive sample selection according to claim 3, characterized in that: S4 includes: S401: For binary or multi-class classification problems, calculate the predicted probability of a sample. If the predicted probability is greater than the screening threshold, retain the sample as a high-confidence sample set. ; S402: Drift type and severity coefficient determined based on S2 Dynamically adjust the filtering threshold; S403: The number of samples in the sample set within the current data block based on the target label category j. Total number of samples in the current data block n, sampling ratio and sampling priority Calculate the number of samples that need to be drawn from historical samples to determine the target label category. ; S404: Based on the calculated sample size From the selected high-confidence sample set Randomly select samples of the corresponding category from the data; if there are not enough samples of a certain category, then select all samples of that category; subsequently, stack the selected samples vertically with the current samples in the current data block to form the final expanded and optimized training sample set. .

7. The data flow concept drift learning method based on fusion adaptive sample selection according to claim 6, characterized in that: S401 includes: For binary classification problems, use the Sigmoid function to calculate the probability: , The output for the positive class in the model; For multi-class classification problems, use the SoftMax function to calculate the probabilities: , Let C be the output of the model for the k-th class, and C be the total number of classes.

8. The data flow concept drift learning method according to claim 6, characterized in that: S402 includes: If it is determined to be a mutation drift, a higher screening threshold should be set to reduce the participation of historical samples that are not adapted to the current distribution: ; If the distribution is determined to be a gradual shift, a lower screening threshold is set to increase the participation of historical samples similar to the current distribution: .

9. The data flow concept drift learning method based on fusion adaptive sample selection according to claim 6, characterized in that: In S403, the number of samples .

10. The data flow concept drift learning method based on fusion adaptive sample selection according to claim 1, characterized in that: S5 includes: S501: Construct a dynamic weighted loss function. Based on binary or multi-class classification, construct a loss function that includes weights and directly integrate the weights into the cross-entropy calculation. For binary classification problems, the category-weighted binary cross-entropy loss function is used. For multi-class classification problems, the category-weighted cross-entropy loss function is used; S502: Perform iterative training and parameter updates based on mixed precision, and calculate the loss value of the current batch by combining a dynamic weighted loss function during iterative training and parameter updates; S503: The scheduler is used to adaptively adjust the learning rate.

Images