Novel active learning method for classifying unbalanced flow data
Through the multi-window knowledge base and dynamic k-NN classifier combined with affinity matrix and CNN strategy, the data changes and new label detection problems in unbalanced flow data are solved, and high accuracy and adaptability in dynamic environments are achieved.
Patent Information
- Application Number
- CN202410986637.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-23
- Publication Date
- 2024-11-26
AI Technical Summary
Traditional machine learning algorithms are difficult to deal with imbalanced data flowing at high speed, especially in the case of data characteristics changes and new labels, which makes it difficult to achieve model updates and classification accuracy.
Using a multi-window knowledge base and a lazy learning framework based on nearest neighbors, combining affinity matrix and clustering algorithm to identify new categories, and online updates and label detection are performed through CNN strategy and attenuation window model, dynamically adjusting k-NN classifiers and weight allocation .
Effectively adapt to the dynamic changes of data flow, improve classification accuracy and adaptability, optimize the use of limited label resources, and ensure efficient and highly accurate predictions in a dynamic environment.
Smart Images

Figure CN119025875A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of machine learning, and in particular relates to a novel active learning method for classifying unbalanced stream data. Background Art
[0002] Advances in data collection, processing, and analysis in recent years have revealed significant challenges in machine learning, especially for traditional algorithms designed for static data sets. In contrast, modern data sources continue to flow large amounts of information at high speeds, and traditional methods are unable to cope with their limitations in speed, flexibility, and resilience.
[0003] A data stream is a continuous flow of data that should ideally be analyzed as it arrives, unlike traditional data mining which focuses on static subsets of data. In an environment where data is continuously generated, real-time analysis is essential, enabling immediate insights and timely decisions, which is critical in dynamic situations. However, data streams require algorithms to process quickly to prevent delays and data loss, and are typically only able to process the data once, and due to its potentially unlimited nature, these algorithms operate on limited memory.
[0004] Furthermore, data streams are characterized by their variability. Traditional machine learning assumes that a model trained on a representative subset of data will remain valid in the future because future data will be similar to the past. However, data streams can change unpredictably, altering data characteristics and decision boundaries. This phenomenon, known as “concept drift,” refers to changes in the statistical properties of the target variable, affecting a variety of practical applications and requiring adjustments to models over time. For example, in the financial industry, fraud prevention models must constantly evolve as fraudsters change strategies. In the retail industry, customer preference models must adapt to changing trends and external influences.
[0005] In addition, significant challenges arise from “concept evolution,” which refers to the emergence of completely unobserved new patterns or labels. This requires detecting and adapting to entirely new concepts. For example, on social media platforms such as Twitter and Instagram, new trends can emerge rapidly, while in healthcare, detecting new pathogens such as the COVID-19 virus requires adjustments to diagnostic models.
[0006] These challenges are further exacerbated by class imbalance in data streams, where the distribution of classes is uneven and dynamically changing. In such environments, not only do class definitions evolve, but class proportions and class roles also change over time. Additionally, the scarcity of labels in data streams poses significant challenges, as labeled data can be difficult and expensive to obtain in real time. This scarcity complicates training and updating models, making it difficult to achieve accurate and reliable predictions in a constantly changing data environment. Summary of the invention
[0007] The present invention aims to provide a novel active learning method for classifying unbalanced stream data to solve the above-mentioned technical problems.
[0008] In order to solve the above technical problems, the specific technical solution of a new active learning method for classifying unbalanced stream data of the present invention is as follows:
[0009] A novel active learning method for classifying unbalanced streaming data includes the following steps:
[0010] S1: Building a knowledge base; Proposing a new strategy using a multi-window knowledge base to establish an independent, fixed-size window for each data category;
[0011] S2: Propose a classification strategy based on the nearest neighbor principle; cope with the dynamics of data streams by using an instance-based lazy learning framework integration;
[0012] S3: Propose a new label detection method; a new category identification method that measures the relationship dynamics between instances by using an affinity matrix and subsequently identifies anomalous clusters that represent potential new categories;
[0013] S4: Online update updates the knowledge base; identifies and classifies newly arrived data instances. If enough new class instances are found, a separate window is created for the new class and inserted into the knowledge base. Instances that fit into existing categories are classified according to the method in S2.
[0014] Furthermore, the knowledge base S1 includes a separate window for each category, represented by KB = {W1, W2, ... W C}, where C represents the total number of categories identified so far. In each window, the data sample is stored as a four-tuple: (x, Label, ω, ω ca ), where x is a data sample, Label represents the true or predicted category, or remains undefined if unknown, and ω is the weight assigned to the sample.
[0015] Furthermore, the ensemble S2 includes multiple k-nearest neighbor k-NN classifiers, and the diversity within the ensemble is controlled by a parameter θ, which determines the number of k-NN classifiers:
[0016] θ = 1: Use a single k-NN classifier, 1-NN or 3-NN.
[0017] θ = 2: includes two different k-NN classifiers, namely 1-NN and 3-NN.
[0018] θ = 3: four different k-NN classifiers are involved, namely 1-NN, 3-NN, 5-NN and 7-NN;
[0019] At each decision point, the incoming instance x is i is evaluated by each classifier in the ensemble, and the selection of the best classifier is guided by evaluating past performance metrics,
[0020] The performance indicator, denoted as Π, is used to i Evaluate the average classification accuracy of each k-NN classifier, calculated as follows:
[0021]
[0022] where m represents the number of the most recent labeled examples used for this evaluation, indicating that function I outputs 1 if the classifier’s prediction is correct, consistent with the true label y i Match, otherwise output 0;
[0023] Maximize the performance weight Π of all classifiers:
[0024] best k = argmax k (Π k-NN )
[0025] The classifier that exhibits the highest weight is best k is selected to predict the test instance x i The class labels:
[0026] p_label(x i )=best k -NN(x i ).
[0027] Furthermore, the method described in S3 includes the following key stages:
[0028] S3.1 construct affinity matrix;
[0029] The affinity between each pair of instances is calculated using a Gaussian kernel function, defined as follows:
[0030]
[0031] where ‖x i -x j ‖ represents the Euclidean distance between the feature vectors of instance i and j, σ is a scaling parameter that will be adaptively determined based on the data;
[0032] S3.2: Adaptive determination of σ;
[0033] Initially, σ is set using a heuristic. Once the affinity matrix W is calculated, σ is updated to the median of all non-zero entries in W, ensuring that σ reflects the typical distances in the dataset. The affinity matrix is then recalculated using this updated σ.
[0034] S3.3: Modification and normalization of affinity;
[0035] Modify each affinity based on a local density metric or other relevant feature:
[0036]
[0037] Here, γ is a regularization parameter that controls the degree of modification;
[0038] S3.4: Dissimilarity matrix calculation and summation;
[0039] From the affinity matrix W, a dissimilarity matrix D is derived, where each entry is:
[0040]
[0041] For each instance, the dissimilarity score is calculated by summing the values in its row:
[0042]
[0043] With high S i Instances with values show weak affinity to most other instances, indicating that they may be part of a new category;
[0044] S3.5: Clustering of high dissimilarity instances;
[0045] Instances with high dissimilarity scores are considered candidates for forming new categories. These instances are processed by a clustering algorithm to determine whether they cluster into independent groups separate from known categories. The clustering parameters are selected based on the dissimilarity score distribution and the size of D.
[0046] S3.6: New category identification;
[0047] The statistical and feature space properties of each cluster are compared with the known categories, and if a cluster’s properties are significantly different from those of existing categories, it will be labeled as a new category.
[0048] Further, the S4 includes:
[0049] S4.1: Active Learning:
[0050] We introduce a strategy called C-nearest neighbor CNN, where C represents the total number of categories. t When it arrives, the CNN strategy identifies the nearest labeled point in the knowledge base, namely x′1,x′2,x′3,,…,x′ C , these points vote to determine the uncertainty of each new, unlabeled sample, and the voting mechanism is described by the following formula:
[0051]
[0052] where y j Represents a marked neighbor The class label of is an indicator function, if y j If it is equal to c, it returns 1, otherwise it returns 0;
[0053]
[0054] If two or more categories received the highest and equal number of votes, the sample was considered inconclusive and further analysis was performed using the following equation:
[0055]
[0056] If this condition is met, it indicates that there is a majority consensus among neighbors on the sample class. If the vote is split, the sample will be marked for further review, possibly querying the user's actual label to avoid errors introduced by incorrect pseudo-labeling.
[0057] Furthermore, a concept adaptation strategy is introduced:
[0058] The imbalance index for class y, denoted as imb_y, is determined by the following formula:
[0059]
[0060] Where count(y) is the number of instances of category y, sizelab is the size of the label window, and C is the number of categories;
[0061] Combining the random selection strategy with the uncertainty method, the random strategy indiscriminately selects instances from the data stream for labeling, covering all regions of the instance space.
[0062] Furthermore, the S4 includes S4.2: knowledge base data management:
[0063] Use a decaying window model, in which each new instance enters the window with a weight of ω=1. As new data arrives, the weight of each existing instance in the knowledge base is multiplied by a constant decay factor δ∈[0,1]. This gradual reduction in weight allows the model to pay more attention to recent data, helping it dynamically adapt to changes in data trends.
[0064] A query-based rejuvenation mechanism is adopted to re-evaluate the relevance of the instances used as nearest neighbor predictions, and if these instances contribute to accurate predictions, their weights are restored to full strength. If the predictions are incorrect, their weights are halved.
[0065] Pruning instances: When a window reaches its capacity, we manage the insertion of new data by removing instances whose weights decrease to close to zero, thus making room for the new data. If no such instance exists, we merge the two closest instances by averaging their data points.
[0066] For data insertion, instances with known labels are directly inserted in the tuple format (x, Label = c, ω = 1, ω ca =1) are added to the corresponding category window. For instances without known labels, they are placed in the nearest category window and labeled with the undefined tuple (x, label = φ, ω = 1, ω ca =1).
[0067] The novel active learning method for classifying unbalanced stream data of the present invention has the following advantages:
[0068] The method of the present invention effectively adapts to the dynamic challenges of real-time data streams. At the core of the method is an advanced data management system that creates compact data summaries using complex structures, which is critical to overcoming memory limitations. The method uses a multi-window knowledge base with separate fixed-size windows for each data class to maintain unique structures and effectively address imbalance issues. In addition, it includes a set of k-nearest neighbor (k-NN) classifiers that dynamically adjust to adapt to data variability. The active learning part optimizes the use of limited label resources by focusing on underrepresented categories through a mixture of uncertainty and random sampling. This comprehensive strategy not only addresses the issues of data variability, new labels, and class imbalance, but also improves the efficiency of the labeling process, ensuring high accuracy and adaptability in a dynamic data environment. BRIEF DESCRIPTION OF THE DRAWINGS
[0069] Figure 1 is a schematic flow chart of a new active learning method for classifying imbalanced streaming data. DETAILED DESCRIPTION
[0070] In order to better understand the purpose, structure and function of the present invention, a novel active learning method for classifying unbalanced stream data of the present invention is further described in detail below with reference to the accompanying drawings.
[0071] This paper proposes a novel active learning method designed for classifying unbalanced streaming data, which effectively adapts to the dynamic challenges posed by real-time data streams. In data stream processing, memory constraints limit the ability to store all incoming data, so complex techniques and advanced data structures are needed to create compact data summaries. In order to improve the ability to classify and identify new labels, a novel multi-window knowledge strategy is adopted. Unlike traditional methods, this method uses multiple fixed-size windows, each dedicated to a specific data category. This setting retains the unique structure of each category and effectively solves the data imbalance problem. In addition, the method includes a dynamically adjusted set of k-nearest neighbor (k-NN) classifiers to adapt to the variability and dynamics of incoming data. When new data arrives, all classifiers are evaluated, and the classifier with the best recent performance is selected for classification, ensuring that the system accurately adapts over time.
[0072] The active learning part is crucial to effectively manage limited labeled resources, and it uses a hybrid approach of uncertainty and random sampling to focus on underrepresented categories. This proposed approach dynamically adapts to concept drift using an error-driven approach, assigning higher training weights to data points that are likely to experience drift, thereby prioritizing them for further analysis. If these data points belong to the minority class, their importance is further increased to address the class imbalance problem.
[0073] In addition, a novel label detector based on a knowledge base is introduced. The detection process starts with building an affinity matrix using a Gaussian kernel function to measure the similarities between instances. These affinities are then modified and normalized to highlight significant differences, converting them into dissimilarities. Instances with high dissimilarity scores are identified as potential new class candidates, and a clustering algorithm is used to determine whether they form independent groups.
[0074] This comprehensive approach not only addresses the challenges of data variability and class imbalance, but also improves the efficiency of the labeling process throughout the entire data flow, maintaining high accuracy and adaptability in dynamic environments.
[0075] Specifically, Figure 1 As shown, a novel active learning method for classifying unbalanced stream data of the present invention comprises the following steps:
[0076] S1: Build knowledge base;
[0077] In the field of data stream processing, memory limitations make it impossible to store all incoming streaming data, requiring a sophisticated approach to effectively manage and analyze this data. A fundamental aspect of data stream mining is the use of high-level data structures that are able to maintain compact summaries of the continuously arriving data. This approach is critical because it addresses the limitations of available memory while ensuring that the data remains accessible for analysis.
[0078] In a streaming environment, recent data is often more valuable because it can highlight changes in data distribution or mark the emergence of new trends. Therefore, it is critical to focus on the latest data. To achieve this, various windowing techniques have been developed to prioritize and manage data based on its recency. One of these techniques is the sliding window model, which maintains a fixed size and continuously traverses the data stream, integrating new data and discarding old data. This ensures that the analysis always reflects the latest data segment.
[0079] Another important model is the damped window, which applies an exponential decay formula to assign weights based on the recency of data points. This approach enhances the importance of new data while gradually reducing the influence of old data, thereby providing a balanced perspective that captures emerging trends and the residual influence of past data. In addition, the tumbling window model operates on fixed-size, non-overlapping segments, which is well suited for scenarios that require analysis of clear, discrete periods without overlap.
[0080] In order to enhance the classification and detection of new categories in streaming data, a new strategy using multi-window knowledge base is proposed. This approach is significantly different from the traditional method that usually uses a single window. By establishing a separate, fixed-size window for each data category, this strategy not only maintains a concise summary of each category, but also retains the unique structure of each category, effectively solving the problems associated with data imbalance.
[0081] The structure of the knowledge base is designed to manage and classify data efficiently. It consists of a separate window for each category, denoted as KB = {W1, W2, …W C}, where C represents the total number of categories identified so far. In each window, the data sample is stored as a four-tuple: (x, Label, ω, ω ca ), where x is a data sample, Label represents the true or predicted category (if unknown, it remains undefined), and ω is the weight assigned to the sample. This structured approach ensures dynamic and orderly management of streaming data, facilitating rapid and accurate data analysis for changing conditions.
[0082] S2: Proposed classification strategy based on the nearest neighbor principle;
[0083] We propose a classification strategy that copes with the dynamic nature of data streams by using an instance-based lazy learning framework ensemble. The ensemble consists of multiple k-nearest neighbor (k-NN) classifiers. The diversity within the ensemble is controlled by a parameter θ, which determines the number of k-NN classifiers:
[0084] θ = 1: Use a single k-NN classifier, such as 1-NN or 3-NN.
[0085] θ = 2: includes two different k-NN classifiers, namely 1-NN and 3-NN.
[0086] θ = 3: four different k-NN classifiers are involved, namely 1-NN, 3-NN, 5-NN and 7-NN.
[0087] At each decision point, the incoming instance x is i is evaluated by each classifier in the ensemble. The selection of the best classifier is guided by evaluating past performance metrics, with the goal of adapting to possible changes in the underlying data distribution.
[0088] The performance indicator, denoted as Π, is used to i Evaluate the average classification accuracy of each k-NN classifier, calculated as follows:
[0089]
[0090] Where m represents the number of the most recent labeled examples used for this evaluation. The indicator function I outputs 1 if the classifier’s prediction is correct and matches the true label y i Matches, otherwise output 0.
[0091] To determine the best classifier at any given time step, we maximize the performance weight Π of all classifiers:
[0092] best k = argmax k (Π k-NN )
[0093] The classifier that exhibits the highest weight is best k is selected to predict the test instance x i The class labels:
[0094] p_label(x i )=best k -NN(x i )
[0095] S3: Propose a new label detection method;
[0096] This section outlines a new category identification method that uses an affinity matrix to measure the relationship dynamics between instances and subsequently identify anomalous clusters that represent potential new categories. The method consists of several key stages: constructing an affinity matrix, modifying and normalizing these affinities, converting them into dissimilarity, and applying a clustering algorithm to detect new groups. Each step is detailed below:
[0097] S3.1: Constructing the affinity matrix
[0098] The initial step in detecting new categories is to construct an affinity matrix to capture the similarities between instances in the knowledge base. The affinity between each pair of instances is calculated using a Gaussian kernel function, defined as follows:
[0099]
[0100] where ‖x i -x j ‖ represents the Euclidean distance between the feature vectors of instance i and j, and σ is a scaling parameter. This parameter will be adaptively determined based on the data.
[0101] S3.2: Adaptive determination of σ
[0102] The parameter σ is crucial because it affects the sensitivity of the Gaussian kernel to the distances between instances. Initially, σ is set using a heuristic such as the median or standard deviation of all pairwise distances. Once the affinity matrix W is computed, σ is updated to the median of all non-zero entries in W, ensuring that σ reflects the typical distances in the dataset. The affinity matrix is then recomputed using this updated σ.
[0103] S3.3: Modification and normalization of affinity
[0104] After the affinity matrix is constructed, the affinities are adjusted to enhance their discriminatory power. This involves an adjustment step that modifies each affinity based on a local density metric or other relevant feature:
[0105]
[0106] Here, γ acts as a regularization parameter controlling the degree of modification, aiming to reduce the influence of highly connected node pairs, which may mask the existence of new categories.
[0107] S3.4: Dissimilarity Matrix Calculation and Summation
[0108] From the affinity matrix W, a dissimilarity matrix D is derived, where each entry is:
[0109]
[0110] This transformation emphasizes the differences between instances. For each instance, the dissimilarity score is calculated by summing the values in its row:
[0111]
[0112] With high S i Instances with values of show weak affinity to most other instances, indicating that they may be part of a new class.
[0113] S3.5: Clustering of Highly Dissimilar Instances
[0114] Instances with high dissimilarity scores are considered candidates for forming new categories. These instances are then subjected to a clustering algorithm, such as DBSCAN, to determine whether they cluster into independent groups separate from the known categories. Clustering parameters, such as the minimum number of points and epsilon, are selected based on the dissimilarity score distribution and the size of D.
[0115] S3.6: New Category Identification
[0116] The clusters formed will be analyzed to confirm their novelty. The statistical and feature space properties of each cluster are compared with known categories. If the properties of a cluster differ significantly from those of existing categories, it will be labeled as a new category. Further domain-specific validation may be performed to confirm the novelty of these categories, involving expert evaluation or additional external data sources.
[0117] S4: Update knowledge base online;
[0118] In a streaming environment, the relevance of data typically decreases over time, making recent data particularly valuable. This is because recent data can reveal changes in the data distribution or the emergence of new trends. To take advantage of this, it is critical to prioritize and manage data in the knowledge base based on its recency. This is achieved by identifying and classifying newly arrived data instances. If enough new class instances are found, a separate window is created for the new class and inserted into the knowledge base, i.e. KB = KBUW C+1 , and C is updated to C = C + 1. Instances that fit into the existing categories are classified according to the method explained in S2.
[0119] The process of updating the knowledge base is further detailed below:
[0120] S4.1: Active Learning:
[0121] Active learning is crucial in situations where label resources are limited. It involves strategically selecting data samples to label, which is critical for cost-effectiveness and efficiency. In the context of imbalanced data streams, active learning specifically focuses on underrepresented classes. Traditional strategies include nondeterministic, random, and hybrid methods. Nondeterministic strategies select instances based on the model's prediction uncertainty. Random strategies select instances indiscriminately from the data stream, while hybrid strategies combine elements of both.
[0122] We introduce a strategy called C-Nearest Neighbors (CNN), where C represents the total number of classes. This approach provides two main benefits. First, it eliminates the need for externally set thresholds by utilizing majority voting to determine the uncertainty of unlabeled samples, ensuring objective decisions. Second, the CNN approach is more resistant to noise than traditional pseudo-labeling methods. Instead of selecting one nearest neighbor for each class, it selects C nearest neighbors from all available data samples, significantly reducing the impact of noisy data and outliers. While the CNN strategy is similar in basic premise to the well-known k-Nearest Neighbors (kNN) technique, it differs in one key respect: it determines the number of nearest neighbors to consider based on the data itself rather than on user-defined parameters.
[0123] When the new data batch B t When it arrives, the CNN strategy identifies the nearest labeled point in the knowledge base, namely x′1,x′2,x′3,,…,x′ C These points vote to determine the uncertainty of each new, unlabeled example. The voting mechanism is described by the following formula:
[0124]
[0125] where y j Represents a marked neighbor The class label of is an indicator function, if y j Returns 1 if equal to c, otherwise returns 0.
[0126]
[0127] If two or more categories received the highest and equal number of votes, the sample was considered inconclusive. Further analysis was performed using the following equation:
[0128]
[0129] If this condition is met, it indicates that there is a majority consensus among neighbors on the sample class. If the vote is split, the sample will be marked for further review, possibly querying the user's actual label to avoid errors introduced by incorrect pseudo-labeling.
[0130] Building on our uncertainty selection approach, we introduce a concept adaptation strategy. This strategy aims to dynamically respond to potential concept drift in specific regions of the dataset, using an error-driven approach. Essentially, when a labeled data point in the knowledge base shows a misclassification rate exceeding a pre-determined threshold, it indicates that concept drift may have occurred in the domain of that data point. To combat this, we assign higher training weights to these data points and their corresponding regions, increasing the likelihood that they will be selected for further analysis.
[0131] In particular, if the weight factor ω_ca of concept adaptation exceeds the baseline threshold ω_θ, this indicates that the instance is more consistent with the emerging concept, prompting us to select it for labeling. In addition, if the data point belongs to the minority class, we further increase its ω_ca value to prioritize it in future selection rounds to address class imbalance. The imbalance indicator for class y, denoted as imb_y, is determined by the following formula:
[0132]
[0133] where count(y) is the number of instances of category y, sizelab is the size of the label window, and C is the number of categories.
[0134] One limitation of our proposed uncertainty strategy is that it cannot select data drift instances that do not occur near the decision boundary. To address this issue, we combine a random selection strategy with the uncertainty method. The random strategy indiscriminately selects instances from the data stream for labeling, covering all regions of the instance space. This enhances the accuracy and effectiveness of the uncertainty strategy in detecting potential concept drift.
[0135] Additionally, the random strategy helps obtain a class distribution that closely reflects the current class distribution at any given time. We use a fixed-size label sliding window to store the actual labels obtained by the random strategy. This window helps estimate the dynamic class imbalance ratio.
[0136] It is worth noting that we evenly distribute our label budget to queries of random, uncertain, and concept adaptation samples. If the uncertainty and concept adaptation strategies consume less than their specified budget, the remainder will be allocated to further random queries. This balanced approach ensures comprehensive data coverage and efficient use of resources.
[0137] S4.2: Knowledge Base Data Management:
[0138] In the field of data stream processing, where memory limitations are a significant challenge, we took a strategic approach by maintaining a fixed-size window for each data category present in the stream. This approach is particularly effective because the latest data usually has a higher value, which may indicate a change in the data distribution or the emergence of a new trend.
[0139] To cope with concept drift, we use a decaying window model. In this model, each new instance enters the window with a weight of ω=1. As new data arrives, the weight of each existing instance in the knowledge base is multiplied by a constant decay factor δ∈[0,1]. This gradual reduction in weight allows the model to attach more importance to recent data, helping it dynamically adapt to changes in data trends. Unlike traditional sliding windows that simply discard the oldest data, our approach uses a more sophisticated strategy to manage the relevance of data instances.
[0140] We employ a query-based rejuvenation mechanism that re-evaluates the relevance of instances used as nearest neighbor predictions. If these instances contribute to accurate predictions, their weights are restored to full strength. If the predictions are incorrect, their weights are halved. This approach ensures that valuable instances retain their influence in the model. In cases where no actual labels are available for incoming instances, the weights of instances used as nearest neighbor queries are maintained to ensure the stability of model predictions.
[0141] Pruning instances: When a window reaches its capacity, we manage the insertion of new data by removing instances whose weights decrease to near zero, thus making room for new data. If no such instance exists, we merge the two closest instances by averaging their data points. This technique not only preserves diversity within the window, but also manages space efficiently.
[0142] For data insertion, instances with known labels are directly inserted in the tuple format (x, Label = c, ω = 1, ω ca =1) are added to the corresponding category window. For instances without known labels, they are placed in the nearest category window and labeled with the undefined tuple (x, label = φ, ω = 1, ω ca =1).
[0143] The decay and accuracy-based rejuvenation mechanisms are designed to quickly reduce the weights of older and less effective instances. This pruning process allows the model to quickly remove outdated or irrelevant data, which helps the model adapt to concept drift more quickly. In addition, pruning helps reduce the overall size of the window, making the data management process more efficient.
[0144] It is to be understood that the present invention is described by some embodiments, and it is known to those skilled in the art that various changes or equivalent substitutions may be made to these features and embodiments without departing from the spirit and scope of the present invention. In addition, under the teachings of the present invention, these features and embodiments may be modified to adapt to specific circumstances and materials without departing from the spirit and scope of the present invention. Therefore, the present invention is not limited by the specific embodiments disclosed herein, and all embodiments falling within the scope of the claims of this application are within the scope of protection of the present invention.
Claims
1. A novel active learning method for classifying unbalanced streaming data, characterized in that The steps include: S1: Building a knowledge base; Proposing a new strategy using a multi-window knowledge base to establish an independent, fixed-size window for each data category; S2: Propose a classification strategy based on the nearest neighbor principle; cope with the dynamics of data streams by using an instance-based lazy learning framework integration; S3: Propose a new label detection method; a new category identification method that measures the relationship dynamics between instances by using an affinity matrix and subsequently identifies anomalous clusters that represent potential new categories; S4: online update to update the knowledge base; Newly arriving data instances are identified and classified. If enough instances of a new class are found, a separate window is created for the new class and inserted into the knowledge base. Instances that fit into existing categories are classified according to the method in S2.
2. The novel active learning method for classifying unbalanced stream data according to claim 1, characterized in that: S1 The knowledge base includes a separate window for each category, denoted as KB = {W1, W2, ... W C }, where C represents the total number of categories identified so far. In each window, the data sample is stored as a four-tuple: (x, Label, ω, ω ca ), where x is a data sample, Label represents the true or predicted category, or remains undefined if unknown, and ω is the weight assigned to the sample.
3. The novel active learning method for classifying unbalanced stream data according to claim 1, characterized in that: The ensemble S2 includes multiple k-nearest neighbor k-NN classifiers. The diversity within the ensemble is controlled by the parameter θ, which determines the number of k-NN classifiers: θ = 1: use a single k-NN classifier, 1-NN or 3-NN; θ = 2: includes two different k-NN classifiers, namely 1-NN and 3-NN; θ = 3: four different k-NN classifiers are involved, namely 1-NN, 3-NN, 5-NN and 7-NN; At each decision point, the incoming instance x is i is evaluated by each classifier in the ensemble, and the selection of the best classifier is guided by evaluating past performance metrics, The performance indicator, denoted as Π, is used to i Evaluate the average classification accuracy of each k-NN classifier, calculated as follows: where m represents the number of the most recent labeled examples used for this evaluation, indicating that function I outputs 1 if the classifier’s prediction is correct, consistent with the true label y i Match, otherwise output 0; Maximize the performance weight Π of all classifiers: best k =argmax k (Π k-NN ) The classifier that exhibits the highest weight is best k is selected to predict the test instance x i The class labels: p_label(x i )=best k -NN(x i )。 4. The novel active learning method for classifying unbalanced stream data according to claim 1, characterized in that: The method described in S3 includes the following key stages: S3.1 construct affinity matrix; The affinity between each pair of instances is calculated using a Gaussian kernel function, defined as follows: where ‖x i -x j ‖ represents the Euclidean distance between the feature vectors of instance i and j, σ is a scaling parameter that will be adaptively determined based on the data; S3.2: Adaptive determination of σ; Initially, σ is set using a heuristic. Once the affinity matrix W is calculated, σ is updated to the median of all non-zero entries in W, ensuring that σ reflects the typical distances in the dataset. The affinity matrix is then recalculated using this updated σ. S3.3: Modification and normalization of affinity.
5. Modify each affinity based on a local density metric or other relevant feature: Here, γ is a regularization parameter that controls the degree of modification; S3.4: Dissimilarity matrix calculation and summation; From the affinity matrix W, a dissimilarity matrix D is derived, where each entry is: For each instance, the dissimilarity score is calculated by summing the values in its row: With high S i Instances with values show weak affinity to most other instances, indicating that they may be part of a new category; S3.5: Clustering of high dissimilarity instances; Instances with high dissimilarity scores are considered candidates for forming new categories. These instances are processed by a clustering algorithm to determine whether they cluster into independent groups separate from known categories. The clustering parameters are selected based on the dissimilarity score distribution and the size of D. S3.6: New category identification; The statistical and feature space properties of each cluster are compared with the known categories, and if a cluster’s properties are significantly different from those of existing categories, it will be labeled as a new category.
6. The novel active learning method for classifying unbalanced stream data according to claim 1, characterized in that: The S4 includes: S4.1: Active Learning: Introducing a strategy called C-nearest neighbor CNN, where C represents the total number of categories, When the new data batch B t When it arrives, the CNN strategy identifies the nearest labeled point in the knowledge base, i.e. These points vote to determine the uncertainty of each new, unlabeled example, and the voting mechanism is described by the following formula: where y j Represents a marked neighbor The class label of is an indicator function, if y j If it is equal to c, it returns 1, otherwise it returns 0; If two or more categories received the highest and equal number of votes, the sample was considered inconclusive and further analysis was performed using the following equation: If this condition is met, it indicates that there is a majority consensus among neighbors on the sample class. If the vote is split, the sample will be marked for further review, possibly querying the user's actual label to avoid errors introduced by incorrect pseudo-labeling.
7. The novel active learning method for classifying unbalanced stream data according to claim 6, characterized in that: Introducing a concept adaptation strategy: The imbalance index for class y, denoted as imb_y, is determined by the following formula: Where count(y) is the number of instances of category y, sizelab is the size of the label window, and C is the number of categories; Combining the random selection strategy with the uncertainty method, the random strategy indiscriminately selects instances from the data stream for labeling, covering all regions of the instance space.
8. The novel active learning method for classifying unbalanced stream data according to claim 6, characterized in that: The S4 includes S4.2: Knowledge base data management: Use a decaying window model, in which each new instance enters the window with a weight of ω=1. As new data arrives, the weight of each existing instance in the knowledge base is multiplied by a constant decay factor δ∈[0,1]. This gradual reduction in weight allows the model to pay more attention to recent data, helping it dynamically adapt to changes in data trends. A query-based rejuvenation mechanism is adopted to re-evaluate the relevance of the instances used as nearest neighbor predictions, and if these instances contribute to accurate predictions, their weights are restored to full strength. If the predictions are incorrect, their weights are halved. Pruning instances: When a window reaches its capacity, we manage the insertion of new data by removing instances whose weights decrease to close to zero, thus making room for the new data. If no such instance exists, we merge the two closest instances by averaging their data points. For data insertion, instances with known labels are directly inserted in the tuple format (x, Label = c, ω = 1, ω ca =1) are added to the corresponding category window. For instances without known labels, they are placed in the nearest category window and labeled with the undefined tuple (x, label = φ, ω = 1, ω ca =1).
