Data stream dynamic feature selection method, electronic device and medium

Abstract

The present invention relates to a data stream dynamic feature selection method, electronic device and medium, the method comprising: using a Gaussian Copula model to fill in missing feature values ​​in an instance; screening target feature subsets based on the filled data, and dividing the instance into core, auxiliary and complete feature subsets. Assigning weights to each subset according to the feature value, generating corresponding training samples; initializing a classifier and training it using the training samples, assigning an initial prediction weight to each classifier; adding the trained classifier to an ensemble learning framework, and adjusting the ensemble prediction weight according to the classifier type. When receiving a new instance, the missing features are supplemented by a Gaussian Copula model, and the classifier is used to perform weighted prediction and output the final classification result. This method overcomes the limitations of traditional feature selection, such as knowledge loss, high computational cost and reliance on real labels, and achieves continuous and accurate prediction by dynamically screening relevant features and retaining multi-dimensional knowledge.

Description

Technical Field

[0001] The present invention relates to the field of dynamic feature selection learning of stream data, and in particular to a data stream dynamic feature selection method, electronic equipment and medium. Background Art

[0002] In open data stream analysis, data captured by streaming sensors faces complex challenges such as incomplete feature values ​​and changing feature relevance. Feature drift and missing feature values ​​further complicate the problem by causing changes in the subset of features relevant to the decision problem. This variability poses a significant challenge to the continued use of learning models and their ability to accurately predict.

[0003] Existing research work mainly focuses on two types of technologies: feature selection and variable feature space learning. Feature selection technology trains the classification model by screening the features most relevant to the category to reduce the impact of redundant features on classification performance. However, this technology may cause discontinuity in the feature space of the training samples and may cause potential knowledge loss. To this end, researchers have also proposed variable feature space learning technology, which responds to feature drift by selecting features and alleviates knowledge loss by mining the relationship between features. However, due to the need to establish a feature mapping model, this method will incur high computational costs when the number of features in the data stream is large. In order to process the changing features, the above method also needs to obtain the true labels of the data stream samples to detect drift, which will bring expensive labeling costs in practical applications. Summary of the Invention

[0004] The present invention provides a data stream dynamic feature selection method, electronic device and medium, which aims to solve the problem of dynamic changes in feature subsets caused by feature drift and feature value loss in data stream analysis in an open environment.

[0005] To achieve the above object, the present invention provides a method for dynamic feature selection of data streams in a first aspect, comprising the following steps:

[0006] Receiving a currently arriving instance in the data stream, and filling missing feature values ​​in the currently arriving instance using a Gaussian Copula model to obtain processed instance data;

[0007] Based on the processed instance data, multiple decision trees in the sliding window are used to select the target feature subset relevant to the classification task at the current moment;

[0008] According to the target feature subset, the instance is divided into a core feature subset, an auxiliary feature subset and a complete feature subset, and weighted training weights are assigned to different feature subsets based on feature values, and corresponding training samples are generated respectively;

[0009] Initialize the core feature classifier, auxiliary feature classifier and complete feature classifier, use the corresponding training samples as training data for training, and assign initial prediction weights to each classifier to obtain the trained classifiers;

[0010] Add the trained classifiers to the ensemble learning framework and dynamically adjust the ensemble prediction weights according to the classifier type;

[0011] In the ensemble learning framework, when a classifier correctly predicts a labeled sample, its ensemble prediction weight is dynamically increased according to a preset step size;

[0012] Receive newly arrived unlabeled instances, fill in missing features through the Gaussian Copula model, use the classifiers in the ensemble learning framework to make weighted predictions, and output the weighted prediction results as the final classification.

[0013] Furthermore, the method for filling the missing eigenvalues ​​in the currently arrived instance by using the Gaussian Copula model is to iteratively optimize the correlation matrix through an online EM algorithm to reconstruct the missing eigenvalues.

[0014] Furthermore, the online EM algorithm includes:

[0015] Buffer newly arrived instances into a fixed-size data window;

[0016] Estimating a monotonic function to map mixed observation data to the underlying normal distribution space;

[0017] In step E, the conditional expected value of the latent representation of the missing features is calculated;

[0018] Update the correlation matrix by maximizing the log-likelihood function in the M-step;

[0019] Repeat the above steps until the correlation matrix converges;

[0020] Reconstruct missing eigenvalues ​​based on the converged correlation matrix.

[0021] Furthermore, the method of screening the target feature subset related to the classification task at the current moment by using multiple decision trees in the sliding window includes:

[0022] Construct multiple decision trees based on Hoeffding trees. Each Hoeffding tree continuously collects statistical information of data samples within a time period and calculates the information gain of each feature.

[0023] When the difference between the two features with the maximum information gain exceeds the Hoeffding bound, a split node is generated using the feature with the maximum information gain. The calculation formula of the Hoeffding bound is:

[0024]

[0025] where ϵ is the Hoeffding bound, is the possible range of attribute information gain, is the confidence level, Indicates the number of samples collected during the time period;

[0026] The feature selection results of multiple decision trees are integrated to determine the target feature subset at the current moment.

[0027] Furthermore, the updating of the decision tree cluster is achieved through a sliding window: when a newly generated decision tree cluster is added to the sliding window, the earliest generated decision tree cluster in the window is replaced, and each decision tree cluster is generated based on training samples of different time periods.

[0028] Furthermore, the distribution rule of the weighted training weight is:

[0029] The weight of the core feature subset sample is: ;

[0030] The weight of the auxiliary feature subset sample is 1;

[0031] The weights of the complete feature subset samples are: ;

[0032] in, is the number of features in the target feature subset, Indicates the current moment, For the moment The target feature subset is filtered by the decision tree sliding window. is the base of natural logarithms.

[0033] Furthermore, the allocation rule of the initial prediction weight is:

[0034] Initial prediction weights for the core feature classifier:

[0035]

[0036] The initial prediction weights of the auxiliary feature classifier are fixed as:

[0037]

[0038] Initial prediction weights for the full feature classifier:

[0039]

[0040] in, is the base of natural logarithms.

[0041] Furthermore, methods for weighted prediction using each classifier in an integrated learning framework include:

[0042]

[0043] in, It is an integrated learning framework For input samples The prediction results, represents the ensemble learning framework, Represents three classifiers, is the input instance to be predicted, Represents the sample index in the core feature classifier, represents the sample index in the auxiliary feature classifier, represents the sample index in the complete feature classifier, 、 、 Corresponding to the prediction weights of the three classifiers respectively; 、 、 Represent samples of three feature subsets respectively, Indicates the Core feature classifiers for core feature subsets The prediction results, is a set of core feature classifiers, including classifiers, Indicates the Auxiliary feature classifiers for auxiliary feature subsets The prediction results, is a set of auxiliary feature classifiers, including classifiers; Indicates the Full feature classifier for the complete feature subset The prediction results, is a complete feature classifier set, including classifiers.

[0044] To achieve the above-mentioned purpose, the second aspect of the present invention provides an electronic device, including a memory and a processor, wherein the memory is used to store a program that supports the processor to execute the data stream dynamic feature selection method, and the processor is configured to execute the program stored in the memory.

[0045] To achieve the above-mentioned purpose, the third aspect of the present invention provides a computer-readable storage medium, on which a computer program is stored. When the computer program is run by a processor, the steps of the data stream dynamic feature selection method are executed.

[0046] Beneficial effects of the present invention:

[0047] Compared to existing technologies, the present invention provides a method, electronic device, and medium for dynamic feature selection of data streams. By integrating multiple technologies, they effectively address the dynamic feature challenges encountered in data stream analysis in open environments. First, a Gaussian Copula model is used to model mixed data, reconstructing missing features as expected values ​​in a latent normal distribution space. This addresses the issues of missing and incomplete feature values ​​and captures dynamically changing feature correlations. Second, a feature selection mechanism based on a sliding window of decision trees is employed to mitigate the influence of historical feature drift. Subsequently, feature value assessment is used to divide instances into core, auxiliary, and complete feature subsets, assigning higher training weights to core features. An ensemble framework (E3C) consisting of three classifiers is constructed. This framework incorporates a dynamic classifier weight adjustment mechanism: initial weights are assigned exponentially based on the feature subset dimensions, and weights are adjusted in real time based on feedback from correct predictions of labeled samples. This allows the ensemble model to adaptively enhance the decision influence of high-contribution classifiers. Furthermore, the three classifiers retain the discriminative knowledge of the core features, the potential correlations of the auxiliary features, and the global information of the complete features, respectively. Through weighted ensemble prediction, multi-dimensional knowledge complementation is achieved, avoiding the information loss associated with traditional feature selection while reducing the high computational cost of variable feature space methods. Ultimately, this method gets rid of the strong dependence on real labels and only requires a small amount of labeled data to autonomously adapt to feature drift through online EM and sliding window mechanisms, significantly reducing the labeling cost in practical applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments.

[0049] Figure 1 This is a flowchart of a data stream dynamic feature selection method disclosed in an embodiment of the present invention.

[0050] Figure 2 This is a schematic diagram of the framework of a data stream dynamic feature selection method disclosed in an embodiment of the present invention. DETAILED DESCRIPTION

[0051] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0052] According to an embodiment of the present invention, it should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the following production method, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0053] like Figure 1 、 Figure 2 As shown, the present invention provides a data stream dynamic feature selection method, comprising the following steps:

[0054] Step S100: receiving a currently arriving instance in a data stream, and filling missing feature values ​​in the currently arriving instance using a Gaussian Copula model to obtain processed instance data;

[0055] During data stream learning, data instances may have missing feature values ​​due to environmental instability or sensor issues. To ensure that subsequent feature selection and classifier training are based on complete feature data, this step uses a Gaussian Copula model to fill in missing feature values. This approach restores data integrity and preserves correlation information even in the presence of missing features.

[0056] Specifically, the operations of this step can be divided into the following sub-steps:

[0057] In the data flow scenario, the instance Arrives one by one in time order. Assume that each instance in the data stream Include Features, namely , some feature values ​​may be missing, and the missing features will affect the accuracy of subsequent feature selection and model training.

[0058] In order to deal with the missing value problem, this paper processes these data through Gaussian Copula modeling technology. Gaussian Copula is an effective multivariate distribution modeling tool. In Gaussian Copula modeling, the mixed data (including instances with missing eigenvalues) are first mapped to the potential standard normal distribution space. The mapping process is achieved through a monotonic function , the normal distribution in the latent space Data mapped to observation space The specific mapping relationship is:

[0059] ,

[0060] in, represents the observed feature data vector, is the underlying normally distributed vector, is the correlation matrix, which describes the correlation between features; represents a d-dimensional multivariate normal distribution.

[0061] Gaussian Copula modeling is based on the correlation matrix , which describes the dependencies between different features. In the latent space, the correlations between features are modeled using the covariance matrix, which captures the correlation structure between features. This allows us to infer the estimated values ​​of missing features using the known eigenvalues ​​and the correlation matrix when some eigenvalues ​​are missing.

[0062] By modeling the data with Gaussian Copula, the missing eigenvalues ​​can be estimated in the latent space through the correlation matrix and the observed eigenvalues. Specifically, the online EM algorithm is used to continuously optimize the correlation matrix. , thereby gradually improving the estimation accuracy of missing features. The EM algorithm consists of two steps:

[0063] E-step (expectation step): Calculate the potential representation of missing features The conditional expectation of and at the previous moment The correlation matrix .

[0064]

[0065] in, is the estimated value of the missing feature, It is a combination of two expectation calculations, the first expectation Represents the estimated value of the missing feature given the observed latent representation and correlation matrix, the second expectation Further estimation is performed based on the observed features and the correlation matrix to obtain the final estimate of the missing features, is the observed latent representation, is the potential representation of the absence, is the correlation matrix, describing the correlation between features. are the observed feature values ​​(features available in the data stream), Represents the expected value, which is used to estimate missing values.

[0066] This step infers the possible values ​​of the missing features through conditional expectations.

[0067] M-step (maximization step): Based on the known observation data and potential representation, the correlation matrix is ​​updated by maximizing the likelihood function .

[0068]

[0069] is the likelihood function, defined as:

[0070]

[0071] in, is the updated correlation matrix, is the correlation matrix calculated at the previous moment t-1, which is used as the initial value at the current moment. Represents the observation dataset, which contains the instance set of all features at the current moment; Represents the solution to make the objective function Maximized , that is, find the optimal correlation matrix , is a constant, is the correlation matrix The logarithmic determinant of is the trace of the matrix, representing the correlation matrix The inverse of a matrix The trace of the product of , where the trace is the sum of the diagonal elements of the matrix, is the conditional expectation based on the observed data, is the inverse of the correlation matrix.

[0072] This step optimizes the correlation matrix between features, thereby improving the estimation ability of missing features.

[0073] Once the data is processed through Gaussian Copula modeling, missing eigenvalues ​​are imputed, resulting in complete feature data. At this point, each instance in the data stream is imputed as a complete data vector containing all eigenvalues, ready for feature selection and model training in subsequent steps.

[0074] Step S200: Based on the processed instance data, a target feature subset relevant to the classification task at the current moment is selected through multiple decision trees in the sliding window;

[0075] In this step, multiple decision trees within a sliding window are used to select the target feature subset relevant to the classification task at the current moment. The core purpose of this step is to dynamically select the most relevant feature subset based on instances in the data stream, thereby avoiding the negative impact of redundant features on model performance. To this end, this paper proposes a feature selection method based on a sliding window of decision trees. It can efficiently perform online feature selection in a data stream environment and continuously update the feature selection results as the data stream continues to arrive.

[0076] Specifically, the operations of this step can be divided into the following sub-steps:

[0077] To adapt to the dynamic nature of data streams, a decision tree cluster consisting of multiple decision trees was first designed. The core units of the decision tree cluster are multiple decision trees, which process and make decisions on data within the same timeframe. Each decision tree segments and models the data by selecting a different subset of features, resulting in a high degree of diversity. In this invention, the Hoeffding tree is used as the base model, enabling continuous model updates through incremental learning without storing large amounts of data.

[0078] To ensure the model always stays current on the latest concepts in the data stream, a sliding window mechanism is employed. The sliding window has a fixed size; each time a new instance arrives, the data within the window is updated, and the oldest sample is removed from the window. The data within the window is used to train the decision tree. This mechanism ensures the model can continuously adapt to feature drift in the data stream, preventing over-reliance on historical data and maintaining high sensitivity to current data.

[0079] During the training process of a decision tree cluster, each decision tree is constructed based on different features of the data sample, and the splitting feature is selected by calculating information gain. Information gain measures the contribution of a feature to the classification task, and the decision tree splits the data by selecting the feature with the largest gain. For each decision tree, the splitting node is selected based on the feature distribution of the training sample at that node and the gain value.

[0080] Specifically, each time a feature is split, the information gain of all features is calculated, and then the feature with the largest information gain and the most significant gain difference is selected for splitting. If the gain difference of a feature exceeds the Hoffding limit ϵ, then the feature is selected as the feature of the current split node. The calculation formula for the Hoffding limit is as follows:

[0081]

[0082] in, For the Hoffding boundary, is the possible range of attribute information gain (usually the log number of categories), is the confidence level (usually set to 0.05 or 0.1), Indicates the number of samples collected during the time period;

[0083] Through the above methods, the Hoffding tree can ensure that features with high confidence can be selected even with a limited number of samples.

[0084] In a decision tree cluster, each decision tree independently evaluates feature importance and performs feature selection. During training, multiple decision trees make different selections. Based on the selection results of each tree, a voting mechanism is used to determine which features are most important for the current classification task. This voting mechanism effectively reduces the overfitting that can occur in a single decision tree.

[0085] In the sliding window, all decision trees participate in feature selection together, and the voting results are used to select the target feature subset that is most relevant to the classification task at the current moment. These feature subsets not only represent the features that have the greatest impact on the classification task in the current data stream, but also enhance the stability of the feature selection results by integrating multiple decision trees. Finally, based on the voting decision results, a target feature subset is selected. For use in subsequent steps.

[0086] While dealing with feature drift and missing features, this feature selection method can also enhance the diversity and stability of the model by integrating the selection results of multiple decision trees, avoiding the potential bias that may be caused by a single feature selection strategy.

[0087] Step S300: Divide the instance into a core feature subset, an auxiliary feature subset, and a complete feature subset according to the target feature subset, assign weighted training weights to different feature subsets based on feature values, and generate corresponding training samples respectively;

[0088] Based on the target feature subset selected in step S200, each instance is divided into three different feature subsets: core feature subset, auxiliary feature subset, and complete feature subset. The purpose is to assign appropriate weighted training weights to each feature subset based on the evaluation of feature value and generate corresponding training samples so that the subsequent classifier can better handle different types of feature data. This step is to solve the problem of redundant features in feature selection and ensure that the classification model can highlight the features that are most valuable to the classification task during training. Specifically:

[0089] In step S200, the most relevant feature subset at the current moment has been selected through the sliding window mechanism of the decision tree. In this step, each instance in the data stream is divided into the following three parts based on the target feature subset:

[0090] Core feature subset : Core features are those features that are judged to have the greatest influence on the classification task during the feature selection process, through which the classifier can make decisions efficiently.

[0091] Auxiliary feature subset : Features that have low relevance to the classification task but may still provide auxiliary information for the model. The addition of auxiliary features can help the model handle more situations and avoid overfitting problems caused by too much information.

[0092] Full feature subset : A complete dataset containing all features, including core features and auxiliary features. The complete feature set contains all the information about the instance and can provide all available input data for the classification model.

[0093] In feature selection, different feature subsets have varying importance. To enable the classifier to effectively learn based on the value of the features, a weighted training weight is assigned to each feature subset. Weighting is based on the value of the feature, that is, the contribution of each feature to the subset. The value of features is assessed based on their impact on the classification task. For example, core features are assigned higher weights because they directly determine classification accuracy, while auxiliary features are assigned lower weights. Specifically, the feature value assessment method includes the following aspects:

[0094] Information gain: Evaluates the information gain brought by a feature in the classification task. The larger the information gain, the greater the contribution of the feature to the classification result, and the higher the weight should be given.

[0095] Correlation metric: measures the correlation between features and target variables (i.e., class labels). Features with higher correlation are considered more important.

[0096] Predictive ability: Evaluate the predictive ability of features in the existing model. Features with higher importance can significantly improve the prediction accuracy of the classification model.

[0097] Based on the above evaluation results, the core features will be given higher weights and the auxiliary features will be given lower weights, thereby forming different training samples for subsequent classifier training.

[0098] After dividing the instances in the data stream into different feature subsets, each subset will generate corresponding training samples. These training samples will be used to train different classifiers:

[0099] Core feature training samples : The core feature subset contains the features most relevant to the classification task. The training samples constructed based on these features will be used to train the core feature classifier .

[0100] Auxiliary feature training samples : The auxiliary feature subset contains those features that have relatively little impact on the classification task, but may still provide useful information. Therefore, the auxiliary feature classifier Training will be performed based on these features to help the model handle other complex situations.

[0101] Complete feature training samples : The complete feature subset contains all features and is used to generate a complete feature classifier ,The complete feature classifier will utilize all feature information to provide the most comprehensive classification capability.

[0102] During the training sample generation process, each training sample is assigned a weighted training weight based on the importance assessment results of the feature subset. The specific weight setting can be as follows: For the core feature subset, higher training weights are assigned so that the classifier can focus more on the features that have the greatest impact on the classification task. For the auxiliary feature subset, lower training weights are assigned to reflect their relatively minor role in the classification task. For the complete feature subset, the weight may be moderate to ensure that important core information is not lost when considering all feature information.

[0103]

[0104] in, 、 、 Represent samples of three feature subsets respectively, Indicates the current moment, For the moment The target feature subset is filtered by the decision tree sliding window. is the base of natural logarithms; Representing a dataset Medium The feature subset other than Represents a dataset.

[0105] Through the above steps, instances in the data stream are rationally divided into core features, auxiliary features, and complete feature subsets, and appropriate weighted training weights are assigned based on the value of the features. The goal of this process is to optimize the model's learning process, ensuring that the classifier can be efficiently trained based on the actual importance of features, while avoiding interference from redundant or irrelevant features, thereby improving the accuracy and robustness of the classification model.

[0106] Step S400: Initialize the core feature classifier, the auxiliary feature classifier, and the complete feature classifier, respectively use the corresponding training samples as training data for training, and assign initial prediction weights to each classifier to obtain each trained classifier;

[0107] Based on the three feature subsets (core features, auxiliary features, and complete features) divided in step S300, three classifiers are initialized. During classifier initialization, the three classifiers are constructed based on their corresponding feature subsets and training samples, and a different training strategy is specified for each classifier. After the classifiers are initialized, each classifier is trained using the training samples of the different feature subsets generated in step S300. During training, each classifier learns classification rules based on the data of its feature subset. These rules will determine how the classifier makes predictions when faced with new instances. In this way, each classifier can effectively optimize for the characteristics of its feature subset.

[0108] After initializing and training each classifier, an initial prediction weight is assigned to each classifier. The weight reflects the relative importance of each classifier in the ensemble learning framework. The initial prediction weight is set according to the ability and importance of the classifier:

[0109] Core feature classifier The weight of is higher because it is trained based on the most important features and can have the greatest impact on the classification results.

[0110] Auxiliary feature classifier has a lower weight because it is trained based on the most important features and can have the greatest impact on the classification results.

[0111] Full Feature Classifier The weight of is moderate, and since it uses all features for training, its predictive ability is stronger, but not necessarily stronger than the core feature classifier.

[0112] Specifically, weights can be adjusted based on classifier performance. For example, the core feature classifier might be assigned a larger initial weight (e.g., based on information gain or classification accuracy), the auxiliary feature classifier might be assigned a smaller initial weight, and the full feature classifier might be assigned a medium weight. This weight distribution provides the foundation for subsequent ensemble learning frameworks.

[0113] After the training process, the three classifiers (core feature classifier, auxiliary feature classifier, and complete feature classifier) ​​will complete their training and obtain training results. These trained classifiers will be able to classify according to different types of features and output corresponding prediction results. Ultimately, these classifiers will be incorporated into the ensemble learning framework and weighted predictions will be made within the framework based on their predictive power.

[0114] Step S500: adding each trained classifier to the ensemble learning framework, and dynamically adjusting the ensemble prediction weight according to the classifier type;

[0115] The three trained classifiers were incorporated into the ensemble learning framework E3C to form an integrated classification system. By combining the predictions of multiple classifiers, the ensemble learning framework can reduce the risk of overfitting and underfitting on a larger scale and improve the model's generalization ability. Specifically, each classifier is responsible for making predictions for the input instance, but they use different subsets of data features. By incorporating these three classifiers into the framework, each type of feature is appropriately weighted in the final classification decision.

[0116] In the context of ensemble learning, "dynamic adjustment" means adjusting the prediction weights of each classifier based on its actual prediction performance as data continues to arrive. This process ensures that in the ensemble learning framework, better-performing classifiers receive more weight, while poorer-performing classifiers are given less weight.

[0117] Weight adjustment is based on the classifier's prediction accuracy, information gain, or other performance metrics. During each classification, the classifier makes a prediction based on the characteristics and context of the current data. The ensemble learning framework then takes a weighted average of the predictions of each classifier, according to their weights, to produce a single, comprehensive prediction.

[0118] In the ensemble learning framework, the ensemble prediction weight determines the contribution of each classifier to the final prediction result. For each newly arrived instance, the ensemble framework uses three classifiers to make predictions separately and then performs a weighted fusion of these predictions according to their weights. The weighted fusion formula is:

[0119]

[0120] in, It is an integrated learning framework For input samples The prediction results, represents the ensemble learning framework, Represents three classifiers, is the input instance to be predicted, Represents the sample index in the core feature classifier, represents the sample index in the auxiliary feature classifier, represents the sample index in the complete feature classifier, 、 、 Corresponding to the prediction weights of the three classifiers respectively; 、 、 Represent samples of three feature subsets respectively, Indicates the Core feature classifiers for core feature subsets The prediction results, is a set of core feature classifiers, including classifiers, Indicates the Auxiliary feature classifiers for auxiliary feature subsets The prediction results, is a set of auxiliary feature classifiers, including classifiers; Indicates the Full feature classifier for the complete feature subset The prediction results, is a complete feature classifier set, including classifiers.

[0121] Through this weighted fusion approach, the ensemble learning framework comprehensively considers the predictions of all classifiers, preventing a single classifier from overfitting or biasing the final decision. Furthermore, this weighting mechanism allows the framework to dynamically adjust the weights of each classifier during the prediction process based on their performance, thereby improving the model's adaptability and accuracy.

[0122] To achieve dynamic adjustment of prediction weights, the ensemble framework periodically updates the weights of each classifier based on their prediction performance. The weight adjustment strategy is based on the following factors:

[0123] If a classifier has a high prediction accuracy in a certain time period, its weight will increase, indicating that the classifier performs better in the current data stream.

[0124] If the features used by a classifier provide more information gain in the classification task, its prediction results will be given a higher weight.

[0125] If a classifier can demonstrate stable prediction capabilities across multiple data stream instances, its weight will be increased accordingly. Conversely, if the classifier's prediction results fluctuate significantly, its weight may be reduced.

[0126] The weight update method can adopt a gradual adjustment strategy. Whenever a classifier correctly predicts a labeled instance, its weight will increase proportionally, while for the classifier with incorrect prediction, its weight will decrease appropriately. This dynamic adjustment mechanism ensures that the ensemble learning framework always prioritizes the best performing classifier.

[0127] By dynamically adjusting weights, the ensemble learning framework can continuously adapt to feature drift, feature loss, and changes in sample distribution in a data stream environment. As the data stream continues to change, some classifiers may become more effective, while others may gradually lose their predictive power. By continuously adjusting the classifier's prediction weights, the ensemble framework ensures that the most appropriate classifier is used for prediction at each moment, thereby ensuring the accuracy and robustness of the final prediction results.

[0128] Furthermore, ensemble learning frameworks can effectively integrate multiple types of features (such as core features, auxiliary features, and complete features), fully utilizing all types of information in the data stream and avoiding the information loss that can occur with single feature selection methods. This allows ensemble learning frameworks to provide more stable and accurate classification predictions when processing complex and dynamically changing data streams.

[0129] Step S600: In the ensemble learning framework, when the classifier correctly predicts a labeled sample, its ensemble prediction weight is dynamically increased according to a preset step size;

[0130] In an ensemble learning framework, each classifier makes a prediction for an unlabeled sample and compares it to the actual label. The correctness of the prediction is determined by calculating the consistency between the classifier's output and the actual label. If a classifier correctly predicts the sample's label, it is considered effective at that point in time. Conversely, an incorrect prediction results in a decrease in its weight.

[0131] This criterion ensures that the ensemble learning framework accurately evaluates the performance of each classifier’s prediction. Classifiers that make correct predictions are rewarded (with increased weights), while classifiers that make incorrect predictions are penalized (with reduced weights).

[0132] Each time a classifier makes a correct prediction, the ensemble learning framework dynamically increases the prediction weight of the classifier according to the preset step size. ) controls the magnitude of each weight increase. A smaller step size means a more stable weight increase, while a larger step size may cause a more drastic change in weight. The formula for weight increase can be expressed as:

[0133]

[0134] in, is the current weight of the classifier, is the preset step size, is the updated weight.

[0135] By dynamically adjusting the weights of classifiers, the ensemble learning framework ensures that the best performing classifiers are prioritized when presented with new data. As the data stream changes, the classifier's prediction accuracy may also fluctuate. Therefore, by increasing the weights, the influence of each classifier can be effectively and dynamically balanced. For example, if a classifier makes accurate predictions repeatedly, its weight will continue to increase, indicating that it performs better in the current data stream and should bear more prediction responsibility.

[0136] As the classifier weights are continuously adjusted, top-performing classifiers will occupy a larger proportion of the ensemble learning framework, thereby ensuring the framework's predictive accuracy. As the weights increase, these classifiers will have a greater say in future decisions, further improving the stability and robustness of the ensemble model. For example, when new concepts emerge or features drift in the data stream, some classifiers may become less adaptable to these changes, resulting in reduced predictive accuracy. Through a dynamic incremental weighting mechanism, the ensemble framework can automatically reduce the influence of these classifiers, allocating more resources to better-performing classifiers, allowing them to quickly adapt to new data patterns.

[0137] While increasing weights helps enhance the influence of high-performing classifiers, over-reliance on a single classifier can lead to overfitting of the ensemble model. Therefore, ensemble learning frameworks also need to employ certain constraints or regularization techniques to prevent the excessive expansion of the weights of individual classifiers. For example, setting an upper limit on the weight of each classifier prevents any one classifier from becoming overly powerful, leading to excessive dominance over the final prediction result.

[0138] Step S700: Receive newly arrived unlabeled instances, complete the missing features using the Gaussian Copula model, perform weighted prediction using each classifier in the ensemble learning framework, and output the weighted prediction results as the final classification.

[0139] Through the mapping process of the Gaussian Copula model, missing features are mapped to the underlying normal distribution space. Based on the known features and the correlation matrix, the values ​​of the missing features are inferred. Once the missing features are completed, the complete feature data can be input into the ensemble learning framework. The ensemble learning framework includes multiple classifiers (such as core feature classifiers, auxiliary feature classifiers, and complete feature classifiers). Each classifier will make predictions for newly arrived instances based on the subset of features used during its training process. Each classifier generates a prediction result.

[0140] Once all classifiers have completed their predictions, the ensemble learning framework combines the predictions of each classifier and performs a weighted fusion. This process ensures that the contributions of different classifiers are combined according to their prediction weights. Through weighted fusion, the ensemble framework can combine the advantages of each classifier, reduce the potential bias of a particular classifier, and make the final prediction more stable and accurate.

[0141] After completing the weighted prediction, the integrated learning framework will output the final classification result, which is a combination of the weighted prediction results of all classifiers. The final prediction category is generated through weighted voting, weighted averaging or other appropriate strategies. The final output prediction category is the classification result of the new instance.

[0142] In this way, the ensemble learning framework can not only utilize the predictive power of all classifiers, but also dynamically adjust the weights of each classifier to ensure that the framework can make the best classification decisions when processing different types of feature data.

[0143] In this embodiment, the weight design has an important influence on the performance of the algorithm proposed in the present invention. The classifier in E3C mainly has two weights. The first is the integrated prediction weight of the classifier, and the second is the weighted training weight based on different sample types when training the classifier. When designing the integrated prediction weight of the classifier, on the one hand, it is necessary to assign appropriate weights according to the properties of different classifiers. For example, a classifier trained based on core features usually has a higher classification ability than a classifier trained based on auxiliary features; on the other hand, the weights need to be dynamically adjusted when the algorithm is running, giving higher weights to classifiers that contribute significantly to the performance, so as to improve the overall prediction ability of the integrated model. Whenever the classifier is initialized, the initial weight settings of different types of classifiers are as follows:

[0144]

[0145] in, is the number of features in the target feature subset, Indicates the current moment, For the moment The target feature subset is filtered by the decision tree sliding window. is the base of natural logarithms.

[0146] To balance the prediction weights of different classifiers, an upper limit is set for the ensemble prediction weight of classifiers containing core features. Furthermore, each time a classifier correctly predicts a labeled instance, its weight is increased proportionally by (1 + s), where s is the pre-set weight adjustment step size. This mechanism ensures that classifiers with superior prediction performance receive more prediction weight.

[0147] Training weights are crucial to an algorithm's learning performance. Some ensemble learning methods enhance model diversity by increasing sample training weights and increasing weight diversity. In the training weight design of this invention, samples containing a subset of core features are assigned higher training weights, while samples containing auxiliary features are assigned lower weights. The specific settings are as follows:

[0148]

[0149] As can be seen from the above, the present invention solves the problems of feature drift, feature missing and knowledge retention through multi-dimensional collaborative innovation. First, the mixed data modeling based on Gaussian Copula dynamically estimates the correlation matrix of the latent normal space through the online EM algorithm, and reconstructs the missing values ​​by using the dependency relationship between features, thus overcoming the limitations of the traditional interpolation method on the assumption of feature independence, and effectively handling mixed data types and missing problems. Secondly, the decision tree cluster sliding window feature selection mechanism improves diversity while ensuring the rigor of feature selection by constructing multiple Hoeffding trees in parallel and adopting a sliding window update strategy: the tree cluster structure covers more candidate features through multiple splits, and the sliding window dynamically replaces the old model to ensure adaptation to the latest data distribution, avoiding the knowledge fragmentation caused by static feature subsets in traditional feature selection methods. In addition, the feature value-driven instance weighting framework innovatively divides the feature space into three subsets: core, auxiliary, and complete. Classifiers are trained for each subset and assigned dynamic weights: the core feature classifier amplifies the contribution of important features by squared weights, the auxiliary classifier retains potential correlation information, and the complete feature classifier captures global dependencies. The three achieve a synergistic effect by adaptively adjusting the integrated prediction weights (such as incrementing by the step size when a correct prediction is made). This not only alleviates the knowledge loss caused by training with fixed feature subsets, but also avoids the high computational cost of variable feature space methods. At the same time, the unsupervised feature drift detection mechanism evaluates changes in feature importance through online updated decision trees, replacing traditional drift detection methods that rely on true labels, significantly reducing labeling costs. Ultimately, this method achieves a balance between feature dynamics, computational efficiency, and knowledge retention, improving the robustness and accuracy of data stream classification in open environments.

[0150] Taking abnormal behavior recognition in IoT scenarios as an example, we first collect a set of abnormal behavior datasets in IoT scenarios, where each instance contains 30 features.

[0151] Whenever an instance is reached, the instance is first modeled through Gaussian Copula to fill in the missing feature values. Then the decision tree sliding window of the feature importance evaluation is used to evaluate the feature importance and output the important feature subset of the instance. . Based on the important feature subset at the current moment , divide the training data set into different parts. For example, for the sample , and its complete feature space is Using a subset of important features Composition of core feature training samples , using the remaining features Composition of auxiliary feature training samples . Afterwards, the core feature classifier Use core feature training samples Training, auxiliary feature classifier Training samples based on auxiliary features Training, while the full feature classifier Training is then performed based on training samples from the complete feature space. After this, the three classifiers are added to the E3C ensemble learning framework. After the ensemble framework is trained, the E3C ensemble is used to predict unknown instances and identify whether the data instance exhibits abnormal behavior.

[0152] The proposed data stream dynamic feature selection method can be used to identify and classify abnormal behaviors in IoT scenarios, identifying data that exhibits abnormal behaviors. Furthermore, it can be extended to detect network attack data with probability drift.

[0153] According to another aspect of an embodiment of the present application, an electronic device is provided, including a processor and a memory, wherein the processor is configured to implement the steps of the method when executing a computer program stored in the memory.

[0154] In the above embodiments of the present invention, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0155] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only exemplary. For example, the division of the units can be a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0156] In addition, the functional units in the various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0157] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk, etc. Various media that can store program codes.

[0158] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as within the scope of protection of the present invention.

Claims

1. A data stream dynamic feature selection method is used to identify and classify abnormal behaviors in the Internet of Things scenario in data stream analysis problems in an open environment, and to identify data that has abnormal behaviors in the Internet of Things scenario, characterized by: The steps include: Receiving a currently arriving instance in the data stream, and filling missing feature values ​​in the currently arriving instance using a Gaussian Copula model to obtain processed instance data; Based on the processed instance data, multiple decision trees in the sliding window are used to select the target feature subset relevant to the classification task at the current moment; According to the target feature subset, the instance is divided into a core feature subset, an auxiliary feature subset and a complete feature subset, and weighted training weights are assigned to different feature subsets based on feature values, and corresponding training samples are generated respectively; Initialize the core feature classifier, auxiliary feature classifier and complete feature classifier, use the corresponding training samples as training data for training, and assign initial prediction weights to each classifier to obtain the trained classifiers; Add the trained classifiers to the ensemble learning framework and dynamically adjust the ensemble prediction weights according to the classifier type; In the ensemble learning framework, when a classifier correctly predicts a labeled sample, its ensemble prediction weight is dynamically increased according to a preset step size; Receive newly arrived unlabeled instances, fill in missing features using the Gaussian Copula model, use the classifiers in the ensemble learning framework to make weighted predictions, and output the weighted prediction results as the final classification. The method of screening the target feature subset related to the classification task at the current moment by using multiple decision trees in the sliding window includes: Construct multiple decision trees based on Hoeffding trees. Each Hoeffding tree continuously collects statistical information of data samples within a time period and calculates the information gain of each feature. When the difference between the two features with the maximum information gain exceeds the Hoeffding bound, a split node is generated using the feature with the maximum information gain. The calculation formula of the Hoeffding bound is: in, For the Hoffding boundary, is the possible range of attribute information gain, is the confidence level, Indicates the number of samples collected during the time period; Integrate the feature selection results of multiple decision trees to determine the target feature subset at the current moment; The updating of the decision tree cluster is achieved through a sliding window: when a newly generated decision tree cluster is added to the sliding window, the earliest generated decision tree cluster in the window is replaced, and each decision tree cluster is generated based on training samples in different time periods.

2. The data stream dynamic feature selection method according to claim 1, characterized in that: The method for filling the missing eigenvalues ​​in the currently arrived instance by using the Gaussian Copula model is to iteratively optimize the correlation matrix through an online EM algorithm to reconstruct the missing eigenvalues.

3. The data stream dynamic feature selection method according to claim 2, characterized in that: The online EM algorithm includes: Buffer newly arrived instances into a fixed-size data window; Estimating a monotonic function to map mixed observation data to the underlying normal distribution space; In step E, the conditional expected value of the latent representation of the missing features is calculated; Update the correlation matrix by maximizing the log-likelihood function in the M-step; Repeat the above steps until the correlation matrix converges; Reconstruct missing eigenvalues ​​based on the converged correlation matrix.

4. The data stream dynamic feature selection method according to claim 1, characterized in that: The distribution rule of the weighted training weight is: The weight of the core feature subset sample is: ; The weight of the auxiliary feature subset sample is 1; The weights of the complete feature subset samples are: ; in, is the number of features in the target feature subset, Indicates the current moment, For the moment The target feature subset is filtered by the decision tree sliding window. is the base of natural logarithms.

5. The data stream dynamic feature selection method according to claim 4, characterized in that: The allocation rule of the initial prediction weight is: Initial prediction weights for the core feature classifier: The initial prediction weights of the auxiliary feature classifier are fixed as: Initial prediction weights for the full feature classifier: in, is the base of natural logarithms.

6. The data stream dynamic feature selection method according to claim 1, characterized in that: Methods for weighted prediction using classifiers in an ensemble learning framework include: in, It is an integrated learning framework For input samples The prediction results, represents the ensemble learning framework, Represents three classifiers, is the input instance to be predicted, Represents the sample index in the core feature classifier, represents the sample index in the auxiliary feature classifier, represents the sample index in the complete feature classifier, 、 、 Corresponding to the prediction weights of the three classifiers respectively; 、 、 Represent samples of three feature subsets respectively, Indicates the Core feature classifiers for core feature subsets The prediction results, is a set of core feature classifiers, including classifiers, Indicates the Auxiliary feature classifiers for auxiliary feature subsets The prediction results, is a set of auxiliary feature classifiers, including classifiers; Indicates the Full feature classifier for the complete feature subset The prediction results, is a complete feature classifier set, including classifiers.

7. An electronic device comprising a memory and a processor, characterized in that: The memory is used to store a program that supports the processor to execute the data stream dynamic feature selection method according to any one of claims 1 to 6, and the processor is configured to execute the program stored in the memory.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the data stream dynamic feature selection method according to any one of claims 1 to 6 are executed.

Images