Position-aware minority class oversampling and unbalanced multi-classification method and position-aware minority class oversampling and unbalanced multi-classification system
Through the position-aware oversampling method of minority classes, the sample weight is calculated and new samples are synthesized, which solves the problem of classification boundaries tending toward majority classes in the uneven multi-class problem, improves classification accuracy and avoids noise and class overlap problems.
Patent Information
- Application Number
- CN202510010975.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-01-03
- Publication Date
- 2025-04-11
AI Technical Summary
When the prior art deals with the problem of unbalanced multi-classification, it is easy to cause the classification boundaries to tend to be more common classes, and the probability of classification errors in minority classes is greater, and data layer methods such as oversampling and undersampling do not effectively consider class overlap and noise problems.
A position-aware minority classes oversampling method is proposed. By calculating the WOS and WON weights of the samples, selecting the appropriate original samples for synthesis, ensuring that the synthetic samples are interpolated at the class boundary and away from the class overlap area, thereby balancing the data distribution and training the weak classifier.
Through the position perception strategy, the problem of noise and class overlap is avoided, and the classification accuracy of the model for unbalanced multi-classification data is improved, and the advantages of wide applicability and high classification accuracy are provided.
Smart Images
Figure CN119807930A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of machine learning and data mining, and specifically relates to a location-aware minority class oversampling and unbalanced multi-classification method and a system thereof. Background Art
[0002] The imbalanced multi-classification problem is a common challenge in machine learning and data mining. It refers to the situation where the number of samples representing one or some categories in the training dataset is far less than that of other categories. The category with fewer samples is called the minority class, and the rest is called the majority class. Directly applying the classification algorithm to the imbalanced dataset will cause the classification boundary to tend towards the majority class, and the probability of misclassification of the minority class is higher. In actual scenarios, minority class samples generally have a larger misclassification cost, so it is crucial to solve the imbalanced classification task.
[0003] The existing data layer method balances the data distribution by undersampling the majority class and oversampling the minority class samples. In order to further improve the classification accuracy, the data layer method is often combined with ensemble learning. However, imbalanced multi-classification data is often accompanied by many problems, such as noise, small extracts, and class overlap. AdaBoost.M2 is an ensemble learning algorithm for multi-classification. Its core idea is to iteratively train a series of weak classifiers and combine these weak classifiers into a strong classifier to improve the overall classification performance. Specifically, at the beginning of training, each sample is given the same weight, and a new weak classifier is trained using the current sample weight in each round of iteration. The AdaBoost.M2 algorithm will increase the weight of the misclassified samples according to the results of the current weak classifier. This allows subsequent weak classifiers to pay more attention to those samples that were previously misclassified. Finally, all trained weak classifiers are combined into a strong classifier. Since the idea of AdaBoost strengthens the learning weight of misclassified samples, samples that are prone to errors, such as samples in the class overlap area, will receive more attention, which will affect the accuracy of the classifier classification. Moreover, data-level methods such as oversampling, which randomly selects samples, do not consider issues such as class overlap and noise. Summary of the invention
[0004] In view of the shortcomings of the prior art, the present invention proposes a location-aware minority class oversampling and unbalanced multi-classification method. First, the data is read and preprocessed, and then the new oversampling method is used to balance the data distribution to train the weak classifier. The effectiveness of synthetic sample interpolation is ensured based on the designed position selection strategy, and finally T classifier results are integrated to improve the accuracy of the model for unbalanced multi-classification data classification. The specific steps of location-aware minority class oversampling are as follows:
[0005] Step 1: Obtain an unbalanced sample data set and digitize and normalize its multiple feature attributes.
[0006] Step 2: Position-aware minority class oversampling method In order to improve the negative impact of unbalanced data distribution on the classifier, the impact of synthesizing new minority class samples on the class boundary is considered. If the newly synthesized samples are interpolated inside the class, the impact on the classification boundary is small; if the newly synthesized samples are interpolated at the class boundary, it may aggravate the overlap problem of the class boundary or generate noise. Therefore, the interpolation position of the newly synthesized samples should be close to the class boundary and away from the class overlap area. The specific steps are as follows:
[0007] Step 2.1: Calculate the minority class sample x i WOS weights:
[0008] WOS(x i )=safety(x i )*SC(x i )
[0009] Among them, safety(x i ) represents the minority class sample x i The security level of neighbor categories, SC(x i ) represents the minority class sample x i Safety level:
[0010]
[0011] Among them, J is the total number of sample categories in the imbalanced dataset, is the sample x i The number of categories of neighbor samples. is the neighbor sample and sample x i The same number of categories. k1 is a hyperparameter that determines the number of minority class samples x i Neighbor range.
[0012] The WOS weight of the minority class sample is used as the probability of being selected to select the original sample used for synthesis.
[0013] Step 2.2: The original sample is selected in the safe area within the class. In order to make the synthetic sample biased towards the class boundary, x i The neighbors of should be selected close to the class boundary, so that the interpolated position of the synthesized sample is between the two samples.
[0014] For the original sample x selected above i , calculate its neighbor sample x k WON weight WON(x k ), determines the interpolation range of the synthetic samples:
[0015] WON(x k )=overlap(xk )*SC′(x k )
[0016] Among them, overlap(x k ) represents the sample x k The multi-class overlap degree of neighbor samples, SC′(x k ) represents the sample x k The ratio of the number of categories of neighbor samples:
[0017]
[0018] in, Represents x k The neighbor samples of x k The number of samples belonging to the same category. k2 is the number that determines the sample x k Neighborhood range hyperparameter.
[0019] Step 2.3: Select the sample x with the largest WON weight from the neighbor samples k , through the oversampling method SMOTE algorithm and the original sample x i Synthesize new samples x in feature space gen :
[0020] x gen =x i +gap*(x k -x i )
[0021] Here, gap is a random value in the range of 0-1.
[0022] Repeat steps 2.1 to 2.3 until the number of samples in each minority class is equal to the number of samples in the majority class, and a balanced data set is obtained.
[0023] A location-aware minority class oversampling and imbalanced multi-classification method, the specific steps are as follows:
[0024] Step 1: Use the preprocessed imbalanced data set as the original data set and initialize the weight of each sample.
[0025] Step 2: Use the above-mentioned position-aware minority class oversampling method to oversample the minority class samples in the dataset, synthesize a new balanced dataset S1, and use the original sample x i The initial weight of the synthetic sample x gen The weight of .
[0026] Step 3: Train classifier h1:
[0027]
[0028] Wherein, D'1 represents the weight set of the balanced data set S1.
[0029] Step 4: According to the classification results of the classifier, the original sample weights are updated and the weak classifier weights are calculated.
[0030] Step 5: Repeat steps 2, 3, and 4, integrate the classifiers obtained from multiple trainings, and use the integrated classifier H to perform multi-classification on the dataset with unbalanced sample distribution.
[0031] A location-aware minority class oversampling and imbalanced multi-classification system includes a data acquisition and preprocessing module, a balanced data distribution reconstruction module, a weak classifier training module, a sample and weak classifier weight update module, and an integrated learning model construction and testing module.
[0032] Data acquisition and preprocessing module. It is used to obtain unbalanced sample data sets and digitize and normalize multiple feature attributes.
[0033] Reconstruct balanced data distribution module. The samples preprocessed by the data acquisition and preprocessing module are used as the original data set, and the location-aware minority class oversampling method is used to balance the data distribution and reconstruct a new balanced data set.
[0034] Train weak classifier module. Use the balanced data set reconstructed by the Reconstruct Balanced Data Distribution module to train weak classifiers according to the sample weights given by the Update Sample and Weak Classifier Weights module.
[0035] Update sample and weak classifier weight module. It is used to initialize the samples of the original data set, and update the original sample weights and calculate the weak classifier weights according to the classification results of the classifier in the training weak classifier module.
[0036] The module for building and testing the integrated learning model integrates the classifiers obtained by multiple training of the weak classifier training module, and uses the integrated classifier H to perform multi-classification on the data set with unbalanced sample distribution.
[0037] The present invention has the following beneficial effects:
[0038] 1. For the problem of unbalanced multi-class classification, the data-level method is combined with the ensemble method, and appropriate minority class samples are selected for synthesis through the weight of safety (WOS) to avoid noise. The present invention can directly process multiple minority classes and balance the training data set in each iteration through the synthesized minority class samples.
[0039] 2. In order to avoid aggravating the class overlap problem during the synthesis process, the present invention proposes a location-aware strategy, which uses multi-class neighbor weights (WON) to dynamically expand the feature space of the minority class, which can avoid generating noise and synthesizing samples in the class overlap area, and improve the problem of excessive interpolation in the local area.
[0040] 3. Compared with the existing technology, it has the advantages of wide applicability and high classification accuracy. BRIEF DESCRIPTION OF THE DRAWINGS
[0041] Figure 1 The process of imbalanced sample classification method for position-aware oversampling of minority class samples;
[0042] Figure 2 Schematic diagram of error interpolation of synthetic samples;
[0043] Figure 3 This is a schematic diagram of the interpolation position of the synthetic sample in the embodiment; DETAILED DESCRIPTION
[0044] The present invention will be further explained below with reference to the accompanying drawings;
[0045] like Figure 1 As shown in the figure, the multi-classification method based on position-aware minority class oversampling and imbalanced data synthesizes minority class samples in each iteration process for the original imbalanced multi-classification training set to balance the sample distribution, train a weak classifier, and update the original sample weights according to the classification results. Finally, multiple trained weak classifiers are integrated for multi-classification of imbalanced data sets. The specific steps are as follows:
[0046] Step 1: For the original training set x i and i Represent the features and true category labels of the i-th sample, respectively, and J represents the number of sample categories in the original training set S. The initial weight D1(i) of the i-th sample is:
[0047]
[0048] Where n is the number of samples in the original training set S.
[0049] Step 2: Set the category with the largest number of samples in the original training set S as the majority class, and the rest as the minority class. Use the position-aware minority class oversampling method to synthesize samples for each minority class in turn until the number of samples for all categories is consistent. The specific steps are as follows:
[0050] Step 2.1: Figure 2As shown in , if the selected sample is located in a small disjunction or class boundary, it is easy to generate noise samples. Therefore, the selected sample should be located in a safe position inside the minority class. For the minority class sample x i , calculate its WOS weight:
[0051] WOS(x i )=safety(x i )*SC(x i )
[0052] Among them, safety(x i ) represents the minority class sample x i The security level of neighbor categories, SC(x i ) represents the minority class sample x i Safety level:
[0053]
[0054]
[0055] in, is a minority class sample x i The number of categories of neighbor samples. is the neighbor sample with sample x i The same number of categories. k1 is a hyperparameter that determines the number of minority class samples x i The neighbor range is set to 5 in this embodiment.
[0056] The WOS weight of the minority class sample is used as the probability of being selected, and the safe samples within the class are selected as the original samples for synthesizing new samples.
[0057] Step 2.2: If the neighbor sample closest to the class boundary or the class overlap area is selected to be synthesized with the original sample, the sample interpolation position is as follows: Figure 3 As shown, it can have a positive impact on the classification boundaries.
[0058] For the original sample x selected in step 2.1 i , calculate the x of its neighbor samples respectively k WON weight WON(x k ):
[0059] WON(x k )=overlap(x k )*SC′(x k )
[0060] Among them, overlap(x k ) represents the sample x k The multi-class overlap degree of neighbor samples, SC′(x k) represents the sample x k The ratio of the number of categories of neighbor samples:
[0061]
[0062] Among them, k2 is the number that determines the sample x k The hyperparameter of the neighbor range is set to 5 in this example. Represents x k The neighbor samples of x k The number of samples belonging to the same category. The WON weight determines the interpolation direction and range of the synthetic samples.
[0063] Step 2.3: In this embodiment, from the original sample x i Select the neighbor sample x with the largest WON weight value from the k1 neighbors of k , through the oversampling method SMOTE algorithm and the original sample x i Synthesize new samples x in feature space gen :
[0064] x gen =x i +gap*(x k -x i )
[0065] Where gap is a random value between 0 and 1. i The initial weight D1 is used as the new synthetic sample x gen The weight of .
[0066] Step 2.4: Repeat steps 2.1 to 2.3 until the number of samples in each category is equal to the number of samples in the majority category, and a balanced training data set S1 and weight D'1 are obtained.
[0067] Step 3: Use the synthetic balanced training data set S1 and weight D'1 to train a classifier h1:
[0068]
[0069] Step 4: Update the original sample weights according to the classification results of classifier h1. Repeat the sample synthesis and training methods in steps 2 and 3 on the original training set S after the updated weights. After T iterations, T weak classifiers h1,…,h T , t=1,2,…T.
[0070] After the tth iteration, the weight of the i-th original sample is updated to D t+1 (i)
[0071]
[0072] in, Indicates the corresponding weight when sample i is predicted as category y in round t. h t (x i ,y) represents the weak classifier h of the tth iteration t For the i-th sample, it belongs to the true category label y i The predicted probability of different labels y. h t (x i ,y i ) represents the weak classifier h t For the i-th sample, it belongs to the true category label y i If a sample is misclassified, that is, h t (x i ,y i ) is smaller, then The larger the value, the weight of the sample will increase in the next iteration. t Indicates the weak classifier h in the tth iteration t The corresponding weight is determined by the weak classifier h t The error rate ε t Decide:
[0073]
[0074] Step 5: Build an integrated classifier H
[0075] The classifiers trained T times using the balanced data set are combined into the final classifier H, which is used to classify the unbalanced data set:
[0076]
[0077] A location-aware minority class oversampling and imbalanced multi-classification system includes a data acquisition and preprocessing module, a balanced data distribution reconstruction module, a weak classifier training module, a sample and weak classifier weight update module, and an integrated learning model construction and testing module.
[0078] The data acquisition and preprocessing module is used to acquire an unbalanced sample data set and perform numerical and normalization processing on multiple feature attributes thereof.
[0079] The module for reconstructing balanced data distribution uses the samples preprocessed by the data acquisition and preprocessing module as the original data set, and uses a location-aware minority class oversampling method to balance data distribution and reconstruct a new balanced data set.
[0080] The training weak classifier module uses the balanced data set reconstructed by the balanced data distribution reconstruction module to train the weak classifier according to the sample weights given by the update sample and weak classifier weight module.
[0081] The updating sample and weak classifier weight module is used to initialize the samples of the original data set, and update the original sample weights according to the classification results of the classifiers in the training weak classifier module, and calculate the weak classifier weights.
[0082] The construction and testing module of the ensemble learning model integrates the classifiers obtained by multiple training of the training weak classifier module, and uses the ensemble classifier H to perform multi-classification on the data set with unbalanced sample distribution.
[0083] This example selects 25 commonly used unbalanced multi-classification public data sets to evaluate the performance of the method, and compares it with other advanced methods for directly dealing with multi-classification imbalance problems, including ROS, SMOTE, Static-SMOTE (SSE), AdaBoost.NC (ANC), SMOTE+AdaBoost (SAB), selects CART as the classifier, and uses Gini impurity as the criterion. Set the minimum number of sample splits to 2 and the minimum number of sample leaf nodes to 1. Use MFM (Macro F1Measure) as the performance evaluation indicator:
[0084]
[0085] The selected 25 datasets are shown in Table 1:
[0086] Table 1
[0087]
[0088] During the evaluation process, each data set was cross-validated five times, and each method was independently tested 10 times. The average value was taken as the final result, as shown in Table 2:
[0089] Table 2
[0090]
[0091] According to the results in Table 2, it can be seen that this method achieves the best results on 25 data sets, with an average value of 75.14 and an average ranking of first, which has certain advantages over other existing methods.
Claims
1. A location-aware minority class oversampling method, characterized by: The specific steps are as follows: Step 1: Calculate the minority class sample x i WOS weights: WOS(x i )=sofety(x i )*SC(x i ) Among them, safety(x i ) represents the minority class sample x i The security level of neighbor categories, SC(x i ) represents the minority class sample x i Safety level: Among them, J is the total number of sample categories in the imbalanced dataset, is the sample x i The number of categories of neighbor samples; is the neighbor sample and sample x i The same number of categories; k1 is a hyperparameter that determines the minority class samples x i Neighborhood range; The WOS weight of the minority class sample is used as the probability of being selected to select the original sample used for synthesis; Step 2: For the original sample x selected above i , calculate its neighbor sample x k WON weight WON(x k ): WON(x k )=overlap(x k )*SC′(x k ) Among them, overLap(x k ) represents the sample x k The multi-class overlap degree of neighbor samples, SC′(x k ) represents the sample x k The ratio of the number of categories of neighbor samples: in, Represents x k The neighbor samples of x k The number of samples belonging to the same category; k2 is the number that determines the sample x k Neighborhood range hyperparameters; Step 3: From the original sample x i Select the sample x with the largest WON weight from the neighbor samples k , through the oversampling method SMOTE algorithm and the original sample x i Synthesize new samples x in feature space gen ; Step 4: Repeat steps 1 to 3 until the number of samples in each minority class is equal to the number of samples in the majority class, completing the position-aware oversampling of minority class samples and obtaining a balanced data set.
2. The location-aware minority class oversampling method according to claim 1, characterized in that: x gen =x i +gap*(x k -x i ) Here, gap is a random value in the range of 0-1.
3. The location-aware minority class oversampling method according to claim 1, wherein: Set the value of hyperparameters k1 and k2 to 5.
4. Location-aware minority class oversampling and unbalanced multi-classification method, characterized by: The specific steps are as follows: Step 1: Initialize the weight of each sample for the original data set; Step 2: Use the location-aware minority class oversampling method as described in any one of claims 1 to 3 to oversample the minority class samples in the original data set to synthesize a new balanced data set S1, and use the original sample x i The weight of the synthetic sample x gen Weights, train classifier h1: Where D'1 represents the weight set of the balanced data set S1; Step 3: Update the sample weights in the original data set according to the classification results of the classifier; repeat step 2, integrate multiple weak classifiers obtained through multiple trainings, and use the integrated classifier H to perform multi-classification on the data set with unbalanced sample distribution.
5. The location-aware minority class oversampling and unbalanced multi-classification method according to claim 4, characterized in that: In the original data set, set the initial weight of the i-th sample to Here, n is the number of samples in the original training set.
6. The location-aware minority class oversampling and unbalanced multi-classification method according to claim 5, characterized in that: After the tth iteration, the weight of the i-th original sample is updated to D t+1 (i) in, represents the weight corresponding to when sample i is predicted as category y in round t; h t (x i ,y) represents the weak classifier h of the tth iteration t For the i-th sample, it belongs to the true category label y i The predicted probability of different labels y; h t (x i ,y i ) represents the weak classifier h t For the i-th sample, it belongs to the true category label y i The predicted probability of t Indicates the weak classifier h in the tth iteration t The corresponding weights are: ε t Represents the weak classifier h t Error rate:
7. The location-aware minority class oversampling and unbalanced multi-classification method according to claim 4, characterized in that: The integrated classifier is: Among them, T represents the number of training times, α t represents the weight of the weak classifier obtained from the tth training; h t represents the weak classifier obtained in the tth iteration, t=1,2,…T.
8. Location-aware minority class oversampling and imbalanced multi-classification system, characterized by: Used to realize the multi-classification of unbalanced data as claimed in claim 4, comprising a data acquisition and preprocessing module, a balanced data distribution reconstruction module, a weak classifier training module, a sample and weak classifier weight update module and an integrated learning model construction and testing module; The data acquisition and preprocessing module is used to acquire an unbalanced sample data set and perform numerical and normalization processing on multiple feature attributes thereof; The reconstructing balanced data distribution module uses the samples preprocessed by the data acquisition and preprocessing module as the original data set, and uses the location-aware minority class oversampling method to balance the data distribution and reconstruct a new balanced data set; The training weak classifier module uses the balanced data set reconstructed by the reconstructing balanced data distribution module to train the weak classifier according to the sample weights given by the updating sample and weak classifier weight module; The update sample and weak classifier weight module is used to initialize the samples of the original data set, and update the original sample weights according to the classification results of the classifier in the training weak classifier module, and calculate the weak classifier weights; The construction and testing module of the ensemble learning model integrates the classifiers obtained by multiple training of the training weak classifier module, and uses the ensemble classifier H to perform multi-classification on the data set with unbalanced sample distribution.