Integrated learning classification method and system with concept drift detection function

Abstract

The invention discloses an integrated learning classification method and system with a concept drift detection function, and relates to the technical field of machine learning and data stream processing, and the method comprises the steps: obtaining high-dimensional data of an extensible sliding window, carrying out the dimension reduction of the high-dimensional data of the extensible sliding window through PCA, and carrying out the extraction to obtain key feature data; performing real-time monitoring on the key feature data based on an adaptive window early warning detection method, automatically expanding an adaptive window during drifting early warning, and obtaining original data of the adaptive window when concept drifting occurs; and inputting the original data of the adaptive window when the concept drift occurs into a pre-established learning model based on an integrated Hough tree to obtain a trained learning model based on the integrated Hough tree, and performing real-time classification of data streams based on the trained learning model based on the integrated Hough tree. The accuracy of stream data classification can be improved, and the adaptive ability of the model to concept drift is enhanced.

Description

Technical Field

[0001] The present invention relates to the technical field of machine learning and data stream processing, and in particular to an integrated learning classification method and system with a concept drift detection function. Background Art

[0002] In practical applications, network traffic data will continue to change over time, and new changes may occur in attack patterns, causing the original model to become invalid or outdated. This phenomenon is called concept drift, that is, over time, the data distribution has changed significantly, and the model needs to be updated in real time to maintain accuracy. In response to the concept drift problem, traditional static models cannot respond in a timely manner, and new methods are needed to dynamically adjust the model's learning process. The concept drift of network traffic data is particularly obvious, especially in some large-scale network environments, where attackers continue to innovate attack methods, causing the model to frequently fail. Therefore, studying how to effectively detect and respond to concept drift has become a key issue in today's network security field. Summary of the invention

[0003] In order to solve the deficiencies mentioned in the above background technology, the purpose of the present invention is to provide an integrated learning classification method and system with concept drift detection function.

[0004] In a first aspect, the purpose of the present invention can be achieved by the following technical solution: an integrated learning classification method with concept drift detection function, the method comprising the following steps: Obtain high-dimensional data of an expandable sliding window, use principal component analysis (PCA) to reduce the dimension of the high-dimensional data of the expandable sliding window, and extract key feature data; Based on the adaptive window Kolmogorov-Smirnov early warning detection method, the key feature data is monitored in real time to determine whether concept drift occurs. If concept drift occurs, the original data of the adaptive window when the concept drift occurs is obtained, and the adaptive window is automatically expanded. If concept drift does not occur, the adaptive window slides normally. The original data of the adaptive window when concept drift occurs is input into a pre-established learning model based on an integrated Hoeffding tree to obtain a trained learning model based on an integrated Hoeffding tree, and real-time classification of data streams is performed based on the trained learning model based on an integrated Hoeffding tree.

[0005] In combination with the first aspect, in certain implementations of the first aspect, the method also includes: the pre-established learning model based on the integrated Hoeffding tree is integrated through a main tree and multiple subtree architecture, wherein the main tree is trained using a complete indicator set to ensure global feature coverage, and the subtree generates a differentiated classifier by randomly selecting some indicator dimensions, and both the main tree and multiple subtrees dynamically judge node splitting based on the Gini gain difference and the Hoeffding boundary, and update the tree structure to capture changes in feature distribution.

[0006] In combination with the first aspect, in some implementations of the first aspect, the method further includes: an initialization process of the integrated Hoeffding tree: Build a decision tree consisting of k empty trees T1, T2, ..., T k The model consists of T1 and T2, where T1 is the whole tree trained with all indicators, T2~T k The subtree is trained with some indicators, and the complete tree T1 uses the complete indicator set Conduct training; T2~T k Subtree random selection To ensure that each indicator is within the range of T2~T k can be selected at least once, setting the target indicator coverage probability , the minimum number of subtrees required can be derived through the probability formula of indicator coverage , the steps are as follows: The indicator coverage of a single subtree, the probability that a certain indicator is not selected in a single subtree , as shown in formula (1): (1) The coverage of multiple subtrees, with The tree, in The probability that a certain indicator is not selected in a subtree , as shown in formula (2): (2) exist The probability that a certain indicator in a subtree is selected at least once is expressed as formula (3). Substituting it into formula (2), we get formula (4) as follows: (3) (4) Probability of coverage in target indicators When the value is set to the preset value, calculate the value that satisfies a certain indicator between T2 and T k The condition that the subtrees are selected at least once in the tree while minimizing the number of computational resources required .

[0007] In combination with the first aspect, in some implementations of the first aspect, the method further includes: determining the optimal region division method for leaf nodes of each tree in the pre-built learning model based on the integrated Hoeffding tree, the steps are as follows: Traversing indicator variables: Let the current node sample be S, and the indicator variable set be , traverse each indicator variable in turn ,in, ; Generate candidate split points: For each indicator variable , the indicator variable All the index values ​​contained are sorted from small to large, denoted as , the candidate split point set is the median of all adjacent index values, and the candidate split point is expressed as , as shown in formula (5): (5) in Represents indicator variable The first The index value, Represents the indicator variable The number of all indicator values, The index of the candidate split point, ranging from 1 to ; Calculate the partitioning effect corresponding to the split point: for each candidate split point , divide the current node sample S into two sub-areas, as shown in formula (6): (6) in , are the left and right subsets after division, s is a data instance in the sample set S, For sample instance s in the indicator variable The specific value of In combination with the first aspect, in some implementations of the first aspect, the method further includes: an optimal selection process of the candidate segmentation points: The purity of a node is quantified by the Gini coefficient. For a node, its Gini coefficient is defined as shown in formula (7): (7) in is the total number of categories classified, For the category in the node The sample proportion of is the Gini coefficient; Iterate over each indicator variable All candidate split points are calculated, and the Gini gain after splitting at each split point is calculated. The calculation formula is shown in (8): (8) in is the Gini coefficient of the parent node before splitting, indicating the sample purity of the parent node. , is the Gini coefficient of the left and right child nodes after splitting, which respectively represents the sample purity of the left and right child nodes, , is the number of samples of left and right child nodes, is the total number of samples of the parent node; The candidate split point with the largest Gini gain value is selected as the current optimal split point, and the Gini gain value of the suboptimal split point is recorded. The difference between the two Gini gains is calculated to verify the significance of the split, as shown in formula (9): (9) in is the Gini gain of the optimal split point, is the Gini gain of the suboptimal split point, is the Gini gain difference between the two cut points; To ensure the statistical significance of the Gini gain difference, the Hoeffding bound is introduced as a confidence constraint, which can be expressed as (10): (10) in is the number of samples, is the detection threshold, For variables The length of the value range of ; If the Gini gain is poor , then the optimal split point is considered to be significantly better than the suboptimal split point, and the optimal split point is accepted for tree splitting; if the Gini gain is poor , it means that the current sample is not sufficient to prove the significance of the cut-off point, and re-evaluation is needed after data accumulation; For the newly generated child nodes, the optimal split point selection process is repeated until all nodes meet the conditions for stopping splitting.

[0008] In combination with the first aspect, in some implementations of the first aspect, the method further includes: the integrated Hoeffding tree joint decision process: Based on the pre-established integrated Hoeffding tree model, when the data stream is input into the model for prediction, the main tree T1 and the subtrees T2~T k The prediction results are used to generate the final classification results through a dynamic weighted voting mechanism; Decision window initialization: Use sliding window Q to quantize subtree T2~T k The historical decision reliability of the sliding window is set to , the initial state is filled with all 1 values, assuming that subtree T2~T k The initial decision was correct; Window update rule: Whenever a new sample is input, subtree T2~T k Perform a weighted voting prediction on the sample. If the prediction is correct, fill 1 into the latest bit of the sliding window, otherwise fill 0. The subtree T2~T k Correct predictions are marked as 1, and incorrect predictions are marked as 0; Credibility calculation: When the data stream arrives, the subtree T2~T in the statistical window k The sum of making the right decisions , which can be expressed as formula (11): (11) in Indicates the first decision value; Joint decision-making: With preset threshold Compare; like , determine the subtree T2~T k The decision is credible, and the majority voting strategy is used to merge the main tree T1 and the subtrees T2~T k The prediction results; like , only the output of the main tree T1 is used as the final classification result.

[0009] In combination with the first aspect, in some implementations of the first aspect, the method further includes: performing a dimensionality reduction process on the high-dimensional data of the scalable sliding window using principal component analysis PCA: The proposed principal component analysis (PCA) data dimensionality reduction operation requires the creation of two sliding windows: a fixed-capacity historical window L and a dynamically expandable adaptive window R, where the initial capacity of L is C L , the initial capacity of R is C R And set the maximum expansion limit R MAX , perform dimensionality reduction operations on the historical window L and the adaptive window R respectively; Standardized data,First, the window data is standardized, as shown in formula (12): (12) in is the original data, is the mean value of the indicator, is the standard deviation of the indicator, is the standardized data; Calculate the covariance matrix and calculate the standardized data The covariance matrix of , the covariance matrix represents the correlation between the indicators in the data, as shown in formula (13): (13) in, is the number of samples in the window, for The transposed matrix of By solving the eigenvalues ​​and corresponding eigenvectors of the covariance matrix, the direction of the principal component of the data is obtained. The eigenvalue represents the variance of the principal component, and the eigenvector represents the direction of the principal component. The eigenvalues ​​are arranged in descending order, and the first The eigenvector corresponding to the largest eigenvalue, where the principal component with a large eigenvalue indicates that the original information volume of the data accounts for a large proportion, and the variance contribution rate of each principal component is calculated. , as shown in formula (14): (14) in, Is the first after sorting The eigenvalues ​​of the principal components, is the number of principal components selected; Projection data, select The eigenvectors are arranged in columns to form a transformation matrix , the standardized data Project onto the selected principal component, as shown in formula (15): (15) in, It is the data after dimensionality reduction.

[0010] In combination with the first aspect, in some implementations of the first aspect, the method further includes: the use process of the adaptive window Kolmogorov-Smirnov early warning detection method is as follows: For the data after dimensionality reduction , extract the first The Kolmogorov-Smirnov test is performed on the dimensional data, as shown in formula (16): (16) in, is the data matrix after dimension reduction of the historical window, is the data matrix after adaptive window dimensionality reduction, For the history window The data on the principal components, For the adaptive window Data on principal components; The Kolmogorov-Smirnov test process is as follows: Establish the null hypothesis H0: Assume that there is no difference between the two samples and they come from the same distribution; Compute the cumulative distribution function: For each dimension , respectively calculate , The cumulative distribution function of , , the cumulative distribution function describes the probability that a random variable is less than or equal to a preset specific value, that is ,set up Sample Dataset The samples in are independent and random, then the cumulative distribution function It is expressed as formula (17): (17) in, is the indicator function, as shown in formula (18): (18) Calculate the Kolmogorov-Smirnov statistic , expressed as and The maximum vertical distance between the cumulative distribution functions is the maximum difference between the two distributions, as shown in formula (19): (19) Define the significance level and calculate value, the maximum difference between the data distribution Convert to Value, formula (20) is the calculation expression: (20) in is the effective sample size, that is , yes The number of samples in yes The number of samples in ; Weighting Value: Based on the variance contribution of each principal component ,right The values ​​are weighted to obtain the global drift significance score , as shown in formula (21).

[0011] (twenty one) In combination with the first aspect, in some implementations of the first aspect, the method further includes: the process of real-time monitoring of key feature data based on the adaptive window Kolmogorov-Smirnov early warning detection method is as follows: Extraction of key feature data dimensions, and calculate the statistics of the difference in cumulative distribution functions and corresponding Value, and then according to the variance contribution rate of each principal component Weighted global drift significance score , set the significance level As the concept drift determination threshold, and define the warning threshold ; like , it is believed that data concept drift has occurred, the system will clear the historical window data and retrain the integrated Hoeffding tree model using the adaptive window data; like , it is considered that a drift warning has occurred, and the system automatically enters the elastic buffer stage, dynamically expanding the adaptive window to accumulate potential drift samples; like , it is considered that no drift has occurred and the sliding window continues to update the data.

[0012] In a second aspect, in order to achieve the above-mentioned object, the present invention discloses an integrated learning classification system with a concept drift detection function, comprising: The data dimension reduction module is used to obtain the high-dimensional data of the scalable sliding window, and use principal component analysis PCA to reduce the dimension of the high-dimensional data of the scalable sliding window to extract key feature data; The data expansion module is used to monitor the key feature data in real time based on the adaptive window Kolmogorov-Smirnov early warning detection method to determine whether concept drift occurs. If concept drift occurs, the original data of the adaptive window when the concept drift occurs is obtained and the adaptive window is automatically expanded. If concept drift does not occur, the adaptive window slides normally. The data classification module is used to input the original data of the adaptive window when concept drift occurs into a pre-established learning model based on the integrated Hoeffding tree to obtain a trained learning model based on the integrated Hoeffding tree, and perform real-time classification of data streams based on the trained learning model based on the integrated Hoeffding tree.

[0013] In another aspect of the present invention, in order to achieve the above-mentioned purpose, a terminal device is disclosed, including a memory, a processor, and a computer program stored in the memory and capable of running on the processor, wherein the memory stores a computer program capable of running on the processor, and when the processor loads and executes the computer program, an integrated learning classification method with a concept drift detection function as described above is adopted.

[0014] Beneficial effects of the present invention: In terms of data dimensionality reduction and feature extraction, the present invention uses principal component analysis (PCA) to reduce the dimensionality of high-dimensional data in an expandable sliding window and extract key feature data. This method not only effectively reduces computational overhead, but also retains the main information of the data and improves the efficiency and accuracy of data stream classification.

[0015] In terms of concept drift detection, the present invention realizes real-time monitoring of key feature data based on the adaptive window Kolmogorov-Smirnov early warning detection method. When the distribution difference between windows reaches the early warning threshold but has not crossed the drift judgment boundary, the system enters the elastic buffer stage and dynamically expands the adaptive window to accumulate more data for further detection. This mechanism can reduce false alarms caused by short-term fluctuations and improve the robustness of drift detection. At the same time, in order to prevent the problem of computing resource consumption caused by unlimited window growth, the maximum extension length is set to ensure that the window is adaptively adjusted within a reasonable range, and the normal sliding update mechanism is restored after reaching the upper limit.

[0016] In terms of classification model design, the present invention adopts a learning model based on an integrated Hoeffding tree, and uses the original data of the adaptive window to incrementally train the model when concept drift occurs. This method combines the main tree and multi-subtree collaborative learning mechanism to retain key information of global data features and enhance the adaptability to changes in local data distribution. By dynamically adjusting the weight of subtrees participating in prediction and combining the joint voting decision of the main tree and subtrees, the impact of abnormal data or local drift on a single model is reduced, and the stability and accuracy of data stream classification are improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative work. Figure 1 It is a schematic flow chart of the method of the present invention; Figure 2 This is a structural diagram of the adaptive window Kolmogorov-Smirnov early warning detection window in the present invention; Figure 3 A decision structure diagram of the integrated Hoeffding tree in the present invention; Figure 4 It is a schematic diagram of the system structure of the present invention. DETAILED DESCRIPTION

[0018] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0019] Embodiment 1: like Figure 1 As shown, an integrated learning classification method with concept drift detection function includes the following steps: S101: acquiring high-dimensional data of an expandable sliding window, performing dimensionality reduction on the high-dimensional data of the expandable sliding window by using principal component analysis (PCA), and extracting key feature data; like Figure 2 As shown, Figure 2 The adaptive window Kolmogorov-Smirnov early warning detection window structure diagram is used to establish an adaptive window Kolmogorov-Smirnov early warning detection window model, and the data input window learned by the integrated Hoeffding tree is input into the window for dimensionality reduction operation. The method includes the following steps: Specifically, using principal component analysis (PCA) to reduce the dimensionality of high-dimensional data in an expandable sliding window requires creating two sliding windows: a fixed-capacity historical window L and a dynamically expandable adaptive window R, where the initial capacity of L is C L , the window index range is 1, 2, ..., nr; the initial capacity of R is C R , the window index range is expressed as n-r+1,…,n-1,n, and R has a maximum expansion upper limit R MAX The data stream is first stored in L, and when L is full, it is transferred to R. If R fills up its initial capacity C R The dimension reduction and drift detection process is triggered, and R is allowed to add raw data to the adaptive window expansion position in the warning state to accumulate potential drift samples. In this example, n is set to 800, r is set to 300, and the maximum expansion limit of R is R MAX Set to 600; Standardized data,First, the window data is standardized, as shown in formula (12): (12) in is the original data, is the mean value of the indicator, is the standard deviation of the indicator, is the standardized data; Calculate the covariance matrix and calculate the standardized data The covariance matrix of , the covariance matrix represents the correlation between the indicators in the data, as shown in formula (13): (13) in, is the number of samples in the window, for The transposed matrix of By solving the eigenvalues ​​and corresponding eigenvectors of the covariance matrix, the direction of the principal component of the data is obtained. The eigenvalue represents the variance of the principal component, and the eigenvector represents the direction of the principal component. The eigenvalues ​​are arranged in descending order, and the first The eigenvector corresponding to the largest eigenvalue, where the principal component with a large eigenvalue indicates that the original information volume of the data accounts for a large proportion, and the variance contribution rate of each principal component is calculated. , as shown in formula (14): (14) in, Is the first after sorting The eigenvalues ​​of the principal components, is the number of principal components selected. In this example, k is 3; Projection data, select The eigenvectors are arranged in columns to form a transformation matrix , the standardized data Project onto the selected principal component, as shown in formula (15): (15) in, It is the data after dimensionality reduction.

[0020] S102: Based on the adaptive window Kolmogorov-Smirnov early warning detection method, the key feature data is monitored in real time to determine whether concept drift occurs. If concept drift occurs, the original data of the adaptive window when the concept drift occurs is obtained, and the adaptive window is automatically expanded. If concept drift does not occur, the adaptive window slides normally. Specifically, in this embodiment, the drift detection of sample data is implemented by an adaptive window Kolmogorov-Smirnov early warning detection method.

[0021] For the data after dimensionality reduction , extract the first The Kolmogorov-Smirnov test is performed on the dimensional data, as shown in formula (16): (16) in, is the data matrix after dimension reduction of the historical window, is the data matrix after adaptive window dimensionality reduction, For the history window The data on the principal components, For the adaptive window Data on principal components; The Kolmogorov-Smirnov test process is as follows: Establish the null hypothesis H0: Assume that there is no difference between the two samples and they come from the same distribution; Compute the cumulative distribution function: For each dimension , respectively calculate , The cumulative distribution function of , , the cumulative distribution function describes the probability that a random variable is less than or equal to a preset specific value, that is ,set up Sample Dataset The samples in are independent and random, then the cumulative distribution function It is expressed as formula (17): (17) in, is the indicator function, as shown in formula (18): (18) Calculate the Kolmogorov-Smirnov statistic , expressed as and The maximum vertical distance between the cumulative distribution functions is the maximum difference between the two distributions, as shown in formula (19): (19) Define the significance level and calculate value, the maximum difference between the data distribution Convert to Value, formula (20) is the calculation expression: (20) in is the effective sample size, that is , yes The number of samples in yes The number of samples in ; Weighting Value: Based on the variance contribution of each principal component ,right The values ​​are weighted to obtain the global drift significance score , as shown in formula (21).

[0022] (twenty one) In this embodiment, a dual-threshold progressive drift detection mechanism is used to determine whether sample data has drifted, and the significance level is set. As the concept drift determination threshold, and define the warning threshold , in this case, Set to 0.05, Set to 0.1.

[0023] like , it is believed that data concept drift has occurred, the system will clear the historical window data and retrain the integrated Hoeffding tree model using the adaptive window data; like , it is considered that a drift warning has occurred, and the system automatically enters the elastic buffer stage, dynamically expanding the adaptive window to accumulate potential drift samples; like , it is considered that no drift has occurred and the sliding window continues to update the data.

[0024] S103: inputting the original data of the adaptive window when the concept drift occurs into a pre-established learning model based on the integrated Hoeffding tree to obtain a trained learning model based on the integrated Hoeffding tree, and performing real-time classification of the data stream based on the trained learning model based on the integrated Hoeffding tree; The pre-established learning model based on the integrated Hoeffding tree improves classification performance through an integrated learning strategy. Among them, the Hoeffding tree is an incremental learning model suitable for streaming data classification, which can efficiently approximate the optimal decision tree through limited samples without storing historical data. On this basis, the present invention introduces an integrated learning strategy to construct a main tree and multiple subtree architecture to improve the model's adaptability to concept drift. Among them, the main tree is trained with a complete indicator set to ensure global feature coverage, and the subtree generates a differentiated classifier by randomly selecting some indicator dimensions. Both the main tree and multiple subtrees dynamically judge node splitting based on the Gini gain difference and the Hoeffding boundary, and update the tree structure to capture changes in feature distribution; Specifically, in this embodiment, experiments are conducted using an intrusion detection dataset, which contains benign and the latest common attacks, collected from a real network environment. For the intrusion detection dataset, it constructs abstract behaviors of 25 users based on HTTP, HTTPS, FTP, SSH, and email protocols. Data collection ends at a certain time on a certain day. Since the attack patterns in the dataset change over time, multiple concept drifts occur in the dataset. After using the k-means clustering sampling method, 28,303 representative records are used to evaluate the model. The key traffic indicators selected are feature_list [Flow Duration, Total Length of Fwd Packets, Fwd Packet LengthMax, …, Flow IAT Min], and the attribute columns are classification labels for flow data, with values ​​of "normal traffic" or "abnormal traffic". Network traffic data flows In the input integrated Hoeffding tree, the flow data sample is represented as ,in is the key indicator vector of the classification model, It is the classification attribute of the streaming data. The Hoeffding tree classification module is integrated to achieve real-time and efficient classification of streaming data. The steps are as follows: Initialize the integrated Hoeffding tree and build a tree consisting of k empty decision trees T1, T2, ..., T k The model consists of T1 and T2, where T1 is the whole tree trained with all indicators, T2~T k Subtree trained for partial metrics. Full tree T1 uses full set of metrics Conduct training; T2~T k Subtree random selection This design enhances the diversity of the model and reduces the risk of overfitting. To minimize the computational cost, the number of subtrees The following conditions need to be met to ensure that each indicator is between T2~T k In order to minimize the selection of repeated indicators, a target indicator coverage probability needs to be set. , the value is 99%. The required number of subtrees can be derived through the probability formula of indicator coverage, as follows: The indicator coverage of a single subtree, the probability that a certain indicator is not selected in a single subtree , as shown in formula (1): (1) The coverage of multiple subtrees, with The tree, in The probability that a certain indicator is not selected in a subtree , as shown in formula (2): (2) exist The probability that a certain indicator in a subtree is selected at least once is expressed as formula (3). Substituting it into formula (2), we get formula (4) as follows: (3) (4) Probability of coverage in target indicators When the value is 99%, it can be calculated that the index that satisfies a certain indicator is between T2 and T k The condition that the subtrees are selected at least once in the tree while minimizing the number of computational resources required ; As the number of data stream samples increases, the optimal area division method for the leaf nodes of each tree in the pre-built learning model based on the integrated Hoeffding tree is determined. The specific steps are as follows: Traversing indicator variables: Let the current node sample be S, and the indicator variable set be , traverse each indicator variable in turn ,in, ; Generate candidate split points: For each indicator variable , the indicator variable All the index values ​​contained are sorted from small to large, denoted as , the candidate split point set is the median of all adjacent index values, and the candidate split point is expressed as , as shown in formula (5): (5) in Represents indicator variable The first The index value, Represents the indicator variable The number of all indicator values, The index of the candidate split point, ranging from 1 to ; Calculate the partitioning effect corresponding to the split point: for each candidate split point , divide the current node sample S into two sub-areas, as shown in formula (6): (6) in , are the left and right subsets after division, s is a data instance in the sample set S, For sample instance s in the indicator variable The specific value of Select the optimal split point. To make most of the samples in the split node come from the same category and make the sample attributes of the split node less different, the node purity needs to be improved. We use the Gini coefficient to quantify the purity of the node. For a node, its Gini coefficient is defined as shown in formula (7): (7) in is the total number of categories classified, For the category in the node The sample proportion of is the Gini coefficient. The smaller the value, the more concentrated the samples are. The larger the proportion of samples belonging to the same category is, the higher the node purity is. In order to measure the degree of improvement of purity by the split operation, the Gini gain indicator is introduced. The Gini gain reflects the degree of improvement of classification purity after splitting the current node. The larger the Gini gain value, the more significant the improvement of the purity of the child node after the split, and the better the splitting effect. When splitting, the Gini gain after the split is calculated at each candidate split point in each indicator set. The calculation formula is shown in (8): (8) in is the Gini coefficient of the parent node before splitting, indicating the sample purity of the parent node. , is the Gini coefficient of the left and right child nodes after splitting, which respectively represents the sample purity of the left and right child nodes, , is the number of samples of left and right child nodes, is the total number of samples of the parent node; In order to ensure that the currently selected split point is statistically significant and avoid making wrong decisions due to insufficient samples, the Hoeffding boundary provides a way based on sample size and confidence to determine whether the Gini gain difference is large enough to ensure the superiority of the current split point; First, calculate the Gini gain difference, as shown in formula (9): (9) in is the Gini gain of the optimal split point, is the Gini gain of the suboptimal split point, is the Gini gain difference between the two cut points; The Hoeffding boundary formula can be expressed as (10): (10) in is the number of samples, is the detection threshold, For variables The length of the value range of ; If the Gini gain is poor , then the current split point is considered to be significantly better than the suboptimal split point, and the candidate split point is accepted Split the tree; if the Gini gain is poor , it means that the current sample is not sufficient to prove the significance of the cut-off point, and re-evaluation is needed after data accumulation; Recursively split the node, and repeat the above traversal index, calculation of candidate split points, Gini gain evaluation and Hoeffding boundary judgment process for the newly generated left and right child nodes; If the child node satisfies the Gini gain difference Or if the sample size of a child node is less than the preset minimum split threshold, the splitting of the child node is stopped and marked as a leaf node; If the child node can continue to split, the above splitting operation is performed recursively until all nodes meet the stop splitting condition; When all nodes of the entire tree reach a state where they cannot be further split, the tree construction process is completed; As new data continues to flow in, repeat the above steps to achieve learning of incremental data; Integrated Hoeffding tree joint decision process, based on the pre-established integrated Hoeffding tree model, when the data stream is input into the model for prediction, the main tree T1 and subtrees T2~T k The prediction results are used to generate the final classification results through a dynamic weighted voting mechanism; Decision window initialization: Use sliding window Q to quantize subtree T2~T k The historical decision reliability of the sliding window is set to , the initial state is filled with all 1 values, assuming that subtree T2~T k The initial decision was correct; Window update rule: Whenever a new sample is input, subtree T2~T k Perform a weighted voting prediction on the sample. If the prediction is correct, fill the latest bit of the sliding window with 1, otherwise fill it with 0. k Correct predictions are marked as 1, and incorrect predictions are marked as 0; Credibility calculation: When the data stream arrives, the subtree T2~T in the statistical window k The sum of making correct decisions can be expressed as formula (11): (11) in Indicates the first decision value; Joint decision-making: With preset threshold Compare, in this case, Set to 0.9, sliding window capacity Set to 40; like , determine the subtree T2~T k The decision is credible, and the majority voting strategy is used to merge the main tree T1 and the subtrees T2~T k The prediction results; like , only the output of the main tree T1 is used as the final classification result.

[0025] like Figure 3 As shown, a dynamic decision method based on an integrated Hoeffding tree includes the following steps: Initialize the whole tree. The whole tree T1 uses all the features of the data set for training to ensure global feature coverage so that key information is not lost; Initialize subtrees, subtree T2, ..., subtree T k Use partial features of the dataset for training to effectively capture data diversity features and enhance adaptability to local concept drift; Construct an integrated model, initialize the whole tree T1 and subtrees T2, ..., subtrees T k Combined together, an integrated model is formed, which combines the different characteristics and advantages of the whole tree and subtrees.

[0026] Joint decision-making, as new data are continuously input into the integrated Hoeffding tree model, during the prediction process, the whole tree T1 and subtrees T2, ..., subtrees T k In the joint decision-making process, they make predictions on the input data based on their own model structures and training results, and finally output the results through joint voting, thereby reducing the impact of abnormal data or local drift on a single model and improving the accuracy of the overall classification.

[0027] Specifically, the present invention is further described below by way of embodiments: After experimental comparison and verification, the ensemble learning classification algorithm with concept drift detection function proposed in this paper has better performance. This paper classifies the traffic data in the data set, aiming to distinguish "normal traffic" from "abnormal traffic" to ensure the correct classification of traffic data. For the classification task, this paper selects the following four evaluation indicators based on the confusion matrix: accuracy, precision, recall and .

[0028] Table 1 is the confusion matrix, where and Represent the total number of samples correctly predicted as normal traffic and abnormal traffic, respectively. and Represents the total number of samples predicted incorrectly in normal traffic and abnormal traffic respectively. Accuracy represents the ratio of the number of correctly predicted samples to the total number of samples, precision represents the ratio of samples correctly predicted as normal traffic to samples predicted as normal traffic, recall represents the ratio of samples correctly predicted as normal traffic to samples actually being normal traffic, and The characteristic of integrating precision and recall is their harmonic mean, which is calculated as follows: (twenty two) (twenty three) (twenty four) (25) Table 1 Confusion matrix Table 2 shows the comparison of the proposed method KS-DB-IOS with ELM, DDM, ADWIN, OCDD and these four methods.

[0029] Table 2 Performance comparison of five algorithms Embodiment 2: In the second aspect, as Figure 4 As shown, in order to achieve the above-mentioned purpose, the present invention discloses an integrated learning classification system with concept drift detection function, comprising: A data dimension reduction module 11 is used to obtain high-dimensional data of an expandable sliding window, and use principal component analysis PCA to reduce the dimension of the high-dimensional data of the expandable sliding window to extract key feature data; The data expansion module 12 is used to monitor the key feature data in real time based on the adaptive window Kolmogorov-Smirnov early warning detection method to determine whether concept drift occurs. If concept drift occurs, the original data of the adaptive window when the concept drift occurs is obtained, and the adaptive window is automatically expanded. If concept drift does not occur, the adaptive window slides normally. The data classification module 13 is used to input the original data of the adaptive window when concept drift occurs into a pre-established learning model based on the integrated Hoeffding tree, obtain a trained learning model based on the integrated Hoeffding tree, and perform real-time classification of data streams based on the trained learning model based on the integrated Hoeffding tree.

[0030] Based on the same inventive concept, the present invention also provides a computer device, which includes: one or more processors, and a memory for storing one or more computer programs; the program includes program instructions, and the processor is used to execute the program instructions stored in the memory. The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. It is the computing core and control core of the terminal, which is used to implement one or more instructions, specifically for loading and executing one or more instructions in a computer storage medium to implement the above method.

[0031] It should be further explained that, based on the same inventive concept, the present invention also provides a computer storage medium, on which a computer program is stored, and the computer program is executed by the processor to execute the above method. The storage medium can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electrical, magnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it.

[0032] In the description of this specification, the description with reference to the terms "one embodiment", "example", "specific example", etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present disclosure. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0033] The above shows and describes the basic principles, main features and advantages of the present disclosure. Those skilled in the art should understand that the present disclosure is not limited by the above embodiments, and the above embodiments and descriptions are only for explaining the principles of the present disclosure. Without departing from the spirit and scope of the present disclosure, the present disclosure may have various changes and improvements, and these changes and improvements fall within the scope of the present disclosure to be protected.

Claims

1. An ensemble learning classification method with concept drift detection function, characterized in that: The method comprises the following steps: Obtain high-dimensional data of an expandable sliding window, use principal component analysis (PCA) to reduce the dimension of the high-dimensional data of the expandable sliding window, and extract key feature data; Based on the adaptive window Kolmogorov-Smirnov early warning detection method, the key feature data is monitored in real time to determine whether concept drift occurs. If concept drift occurs, the original data of the adaptive window when the concept drift occurs is obtained, and the adaptive window is automatically expanded. If concept drift does not occur, the adaptive window slides normally. The original data of the adaptive window when concept drift occurs is input into a pre-established learning model based on an integrated Hoeffding tree to obtain a trained learning model based on an integrated Hoeffding tree, and real-time classification of data streams is performed based on the trained learning model based on an integrated Hoeffding tree.

2. The ensemble learning classification method with concept drift detection function according to claim 1, characterized in that: The pre-established learning model based on the integrated Hoeffding tree is integrated through a main tree and multiple subtree architecture, wherein the main tree is trained with a complete indicator set to ensure global feature coverage, and the subtree generates a differentiated classifier by randomly selecting some indicator dimensions. Both the main tree and the multiple subtrees dynamically judge node splitting based on the Gini gain difference and the Hoeffding boundary, and update the tree structure to capture changes in feature distribution.

3. The ensemble learning classification method with concept drift detection function according to claim 2, characterized in that: Initialization process of the integrated Hoeffding tree: Build a decision tree consisting of k empty trees T1, T2, ..., T k The model consists of T1 and T2, where T1 is the whole tree trained with all indicators, T2~T k The subtree is trained with some indicators, and the complete tree T1 uses the complete indicator set Conduct training; T2~T k Subtree random selection To ensure that each indicator is within the range of T2~T k can be selected at least once, setting the target indicator coverage probability , the minimum number of subtrees required can be derived through the probability formula of indicator coverage , the steps are as follows: The indicator coverage of a single subtree, the probability that a certain indicator is not selected in a single subtree , as shown in formula (1): (1) The coverage of multiple subtrees, with The tree, in The probability that a certain indicator is not selected in a subtree , as shown in formula (2): (2) exist The probability that a certain indicator in a subtree is selected at least once is expressed as formula (3). Substituting it into formula (2), we get formula (4) as follows: (3) (4) Probability of coverage in target indicators When the value is set to the preset value, calculate the value that satisfies a certain indicator between T2 and T k The condition that the subtrees are selected at least once in the tree while minimizing the number of computational resources required .

4. The ensemble learning classification method with concept drift detection function according to claim 3, characterized in that: The pre-built learning model based on the integrated Hoeffding tree, as the data stream samples continue to increase, the leaf nodes of the integrated Hoeffding tree need to determine the candidate region division method, the steps are as follows: Traversing indicator variables: Let the current node sample be S, and the indicator variable set be , traverse each indicator variable in turn ,in, ; Generate candidate split points: For each indicator variable , the indicator variable All the index values ​​contained are sorted from small to large, denoted as , the candidate split point set is the median of all adjacent index values, and the candidate split point is expressed as , as shown in formula (5): (5) in Represents indicator variable The first The index value, Represents the indicator variable The number of all indicator values, The index of the candidate split point, ranging from 1 to ; Calculate the partitioning effect corresponding to the split point: for each candidate split point , divide the current node sample S into two sub-areas, as shown in formula (6): (6) in , are the left and right subsets after division, s is the data instance in the sample set S, For sample instance s in the indicator variable The value on .

5. The ensemble learning classification method with concept drift detection function according to claim 4, characterized in that: The optimal selection process of the candidate split points: The purity of the node is quantified by the Gini coefficient. For a node, its Gini coefficient is defined as shown in the formula: (7) in is the total number of categories classified, For the category in the node The sample proportion of is the Gini coefficient; Iterate over each indicator variable All candidate split points are calculated, and the Gini gain after splitting at each split point is calculated. The calculation formula is shown in (8): (8) in is the Gini coefficient of the parent node before splitting, indicating the sample purity of the parent node. , is the Gini coefficient of the left and right child nodes after splitting, which respectively represents the sample purity of the left and right child nodes, , is the number of samples of left and right child nodes, is the total number of samples of the parent node; The candidate split point with the largest Gini gain value is selected as the current optimal split point, and the Gini gain value of the suboptimal split point is recorded. The difference between the two Gini gains is calculated to verify the significance of the split, as shown in formula (9): (9) in is the Gini gain of the optimal split point, is the Gini gain of the suboptimal split point, is the Gini gain difference between the two cut points; To ensure the statistical significance of the Gini gain difference, the Hoeffding bound is introduced as a confidence constraint, which can be expressed as (10): (10) in is the number of samples, is the detection threshold, For variables The length of the value range of ; If the Gini gain is poor , then the optimal split point is considered to be significantly better than the suboptimal split point, and the optimal split point is accepted for tree splitting; if the Gini gain is poor , it means that the current sample is not sufficient to prove the significance of the cut-off point, and re-evaluation is needed after data accumulation; For the newly generated child nodes, the optimal split point selection process is repeated until all nodes meet the conditions for stopping splitting.

6. The ensemble learning classification method with concept drift detection function according to claim 1, characterized in that: The joint decision process of integrating the Hoeffding tree in the learning model based on the integrated Hoeffding tree is as follows: Based on the pre-established integrated Hoeffding tree model, when the data stream is input into the model for prediction, the main tree T1 and subtrees T2~T k The prediction results are used to generate the final classification results through a dynamic weighted voting mechanism; Decision window initialization: Use sliding window Q to quantize subtree T2~T k The historical decision reliability of the sliding window is set to , the initial state is filled with all 1 values, let subtree T2~T k The initial decision was correct; Window update rule: Whenever a new sample is input, subtree T2~T k Perform weighted voting prediction on the sample. If the prediction is correct, fill 1 into the latest bit of the sliding window, otherwise fill 0. The subtree T2~T k Correct predictions are marked as 1, and incorrect predictions are marked as 0; Credibility calculation: When the data stream arrives, the subtree T2~T in the statistical window k The sum of making the right decisions , which can be expressed as formula (11): (11) in Indicates the first decision value; Joint decision-making: With preset threshold Compare; like , determine the subtree T2~T k The decision is credible, and the majority voting strategy is used to merge the main tree T1 and the subtrees T2~T k The prediction results; like , only the output of the main tree T1 is used as the final classification result.

7. The ensemble learning classification method with concept drift detection function according to claim 1, characterized in that: The principal component analysis PCA is used to reduce the dimension of the high-dimensional data of the expandable sliding window: The proposed principal component analysis (PCA) data dimensionality reduction operation requires the creation of two sliding windows: a fixed-capacity historical window L and a dynamically expandable adaptive window R, where the initial capacity of L is C L , the initial capacity of R is C R And set the maximum expansion limit R MAX , perform dimensionality reduction operations on the historical window L and the adaptive window R respectively; Standardized data,First, the window data is standardized, as shown in formula (12): (12) in is the original data, is the mean value of the indicator, is the standard deviation of the indicator, is the standardized data; Calculate the covariance matrix and calculate the standardized data The covariance matrix of , the covariance matrix represents the correlation between the indicators in the data, as shown in formula (13): (13) in, is the number of samples in the window, for The transposed matrix of By solving the eigenvalues ​​and corresponding eigenvectors of the covariance matrix, the direction of the principal component of the data is obtained. The eigenvalue represents the variance of the principal component, and the eigenvector represents the direction of the principal component. The eigenvalues ​​are arranged in descending order, and the first The eigenvector corresponding to the largest eigenvalue, where the principal component with a large eigenvalue indicates that the original information volume of the data accounts for a large proportion, and the variance contribution rate of each principal component is calculated. , as shown in formula (14): (14) in, Is the first after sorting The eigenvalues ​​of the principal components, is the number of principal components selected; Projection data, select The eigenvectors are arranged in columns to form a transformation matrix , the standardized data Project onto the selected principal component, as shown in formula (15): (15) in, It is the data after dimensionality reduction.

8. The ensemble learning classification method with concept drift detection function according to claim 7, characterized in that: The use process of the adaptive window Kolmogorov-Smirnov early warning detection method is as follows: For the data after dimensionality reduction , extract the first The Kolmogorov-Smirnov test is performed on the dimensional data, as shown in formula (16): (16) in, is the data matrix after dimension reduction of the historical window, is the data matrix after adaptive window dimensionality reduction, For the history window The data on the principal components, For the adaptive window Data on principal components; The Kolmogorov-Smirnov test process is as follows: Establish the null hypothesis H0: Assume that there is no difference between the two samples and they come from the same distribution; Compute the cumulative distribution function: For each dimension , respectively calculated , The cumulative distribution function of , , the cumulative distribution function describes the probability that a random variable is less than or equal to a preset specific value, that is ,set up Sample Dataset The samples in are independent and random, then the cumulative distribution function It is expressed as formula (17): (17) in, is the indicator function, as shown in formula (18): (18) Calculate the Kolmogorov-Smirnov statistic , expressed as and The maximum vertical distance between the cumulative distribution functions is the maximum difference between the two distributions, as shown in formula (19): (19) Define the significance level and calculate value, the maximum difference between the data distribution Convert to Value, formula (20) is the calculation expression: (20) in is the effective sample size, that is , yes The number of samples in yes The number of samples in ; Weighting Value: Based on the variance contribution of each principal component ,right The values ​​are weighted to obtain the global drift significance score , as shown in formula (21) (21)。 9. The ensemble learning classification method with concept drift detection function according to claim 1, characterized in that: The process of performing real-time concept drift detection on key feature data based on the adaptive window Kolmogorov-Smirnov early warning detection method is as follows: Extraction of key feature data dimensions, and calculate the statistics of the difference in cumulative distribution functions and corresponding Value, and then according to the variance contribution rate of each principal component Weighted global drift significance score , set the significance level As the concept drift determination threshold, and define the warning threshold ; like , it is believed that data concept drift has occurred, the system will clear the historical window data and retrain the integrated Hoeffding tree model using the adaptive window data; like , it is considered that a drift warning has occurred, and the system automatically enters the elastic buffer stage, dynamically expanding the adaptive window to accumulate potential drift samples; like , it is considered that no drift has occurred and the sliding window continues to update the data.

10. An integrated learning classification system with concept drift detection function, using an integrated learning classification method with concept drift detection function as claimed in any one of claims 1 to 9, characterized in that: include: The data dimension reduction module is used to obtain the high-dimensional data of the scalable sliding window, and use principal component analysis PCA to reduce the dimension of the high-dimensional data of the scalable sliding window to extract key feature data; The data expansion module is used to monitor the key feature data in real time based on the adaptive window Kolmogorov-Smirnov early warning detection method to determine whether concept drift occurs. If concept drift occurs, the original data of the adaptive window when the concept drift occurs is obtained and the adaptive window is automatically expanded. If concept drift does not occur, the adaptive window slides normally. The data classification module is used to input the original data of the adaptive window when concept drift occurs into a pre-established learning model based on the integrated Hoeffding tree to obtain a trained learning model based on the integrated Hoeffding tree, and perform real-time classification of data streams based on the trained learning model based on the integrated Hoeffding tree.

11. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and capable of running on the processor, characterized in that: The memory stores a computer program that can be run on the processor. When the processor loads and executes the computer program, an integrated learning classification method with a concept drift detection function as described in any one of claims 1 to 9 is adopted.

Images