Cost-sensitive ensemble classification method based on iterative undersampling

Abstract

The invention relates to an iteration-based undersampling cost-sensitive ensemble classification method, belongs to the technical field of medical data analysis, and is particularly suitable for processing a disease prediction model in an unbalanced data set. According to the method, the sampling probability of the majority class samples is dynamically adjusted through an iterative undersampling strategy, it is ensured that the features of the minority class samples are focused on in the training process, and meanwhile important information of the majority class samples is prevented from being lost. In combination with cost-sensitive learning and ensemble learning, the invention provides a classification method capable of remarkably improving the minority class sample prediction performance while keeping the overall accuracy of a classifier. The method has good adaptability, is suitable for various unbalanced data set scenes, and shows an excellent application effect in medical data analysis and disease early warning.

Description

Technical Field

[0001] The present invention belongs to the field of medical data analysis technology, and in particular, relates to a classification method for processing unbalanced data sets, which is applicable to disease prediction models in medical data sets. A cost-sensitive ensemble classification method based on iterative undersampling is mainly aimed at the current situation that existing methods cannot simultaneously guarantee the prediction accuracy and recall rate of minority class samples. Background Art

[0002] In the current research field of medical big data, disease prediction models have always been the focus of researchers. The core is to predict the category of unknown samples by learning existing data samples, which is highly similar to the logic of medical diagnosis. Traditional classification algorithms train large amounts of data to calculate the probability that the sample to be tested belongs to a certain category, thereby providing a basis for decision makers to make reasonable judgments.

[0003] However, such algorithms usually assume that the training data set is balanced. Under this premise, traditional classification algorithms can achieve relatively ideal performance. However, in actual medical data, there is often an imbalance in sample distribution, that is, the number of positive samples (representing disease status) is much lower than that of negative samples (representing health status), and the ratio difference between majority class samples and minority class samples may exceed 10:1. Due to this imbalance, traditional classification algorithms often tend to predict majority class samples during the training process, resulting in a significant decrease in the prediction ability of minority class samples. This makes the model perform poorly on minority class samples, although it can obtain a high overall prediction accuracy in disease prediction. However, in medical disease prediction models, the prediction accuracy of minority class samples is the key factor in the performance of medical models. For example, the number of samples in the diagnostic data set of stroke patients is 1,000, of which 10 samples are diagnosed as having a stroke, and the diagnosis results of the remaining samples are not stroke. Then the model classification method only needs to predict all samples as not having a stroke to obtain a prediction accuracy of 99%. But obviously, this model is completely useless for the intended goal of identifying stroke patients. Therefore, building effective disease early warning models in imbalanced medical data has become an important research challenge.

[0004] At present, when dealing with unbalanced data sets, the use of sampling algorithms can improve the performance of classifiers to a certain extent. Therefore, sampling algorithms are a feasible solution to the imbalance problem in medical samples. The current mainstream sampling methods mainly include oversampling and undersampling methods. The basic principle of oversampling is to balance the overall structure of the data set by copying or generating new minority class samples. Although this method can improve model performance under certain circumstances, especially when dealing with continuous data, since medical data is usually discrete, the samples generated by oversampling may lose authenticity and are difficult to meet medical interpretation requirements. On the other hand, undersampling balances the data set by reducing the number of majority class samples. Although this method is often used to deal with unbalanced data, it also brings some problems. Undersampling may cause important features of majority class samples to be discarded, making it easy to misjudge majority class samples as minority class samples during prediction. Although it can improve the recall rate of minority class samples, it often sacrifices the overall prediction accuracy of the model.

[0005] Other methods to solve the problem of sample imbalance include cost-sensitive learning and ensemble learning. Cost-sensitive learning assigns different weights to different samples, so that the model pays more attention to minority samples during training, thereby alleviating the problem that the number of minority samples is much lower than that of majority samples at the algorithm level. Ensemble learning builds multiple base classifiers, each of which handles the same problem, and then integrates the prediction results of these base classifiers to reduce the impact of sample imbalance, thereby improving the overall performance of the model. Although these two methods can significantly improve the performance of classifiers when dealing with unbalanced data, they also have certain limitations. For example, cost-sensitive learning and ensemble learning sometimes over-adjust the model, causing it to mistakenly classify majority samples as minority samples while paying attention to minority samples, thereby affecting the overall prediction accuracy of the model, especially reducing the recognition ability of majority samples.

[0006] Therefore, in view of the imbalance of medical data, the present invention proposes a cost-sensitive ensemble classification method based on iterative undersampling. In the data preprocessing stage, through the improved undersampling algorithm, while ensuring that the minority class samples are fully learned, the majority class samples are covered as much as possible, and more attention is paid to the misclassified majority class samples. At the classification algorithm level, a method combining cost-sensitive learning and ensemble learning is adopted, which not only ensures the recall rate of minority class samples, but also improves the overall prediction accuracy of the model. This method overcomes the limitations of traditional classification algorithms on unbalanced data sets, and is particularly suitable for disease prediction models in medical data. Summary of the invention

[0007] Technical problem: The purpose of the present invention is to provide a classification method based on iterative undersampling and cost-sensitive learning combined with ensemble learning, which can improve the prediction performance of minority class samples when processing unbalanced data sets while ensuring the overall accuracy of the classifier.

[0008] Technical solution: This invention proposes a novel iterative undersampling cost-sensitive ensemble classification method, which combines the advantages of undersampling, cost-sensitive learning and ensemble learning, and optimizes the overall performance of the classification model by dynamically adjusting the sampling probability of majority class samples. The specific solution is as follows:

[0009] A cost-sensitive ensemble classification method based on iterative undersampling comprises the following steps:

[0010] Step 1, Sampling method analysis: The sampling method is used to select samples of the same number as the minority class samples from the majority class samples, and construct a data set with a relatively balanced ratio of positive and negative samples, so as to ensure that the base classifier can achieve the ideal classification effect; secondly, it is used to screen the data set participating in each training by sampling probability, and try to avoid the majority class samples that have been correctly classified in the previous iteration from entering the current iteration, while the majority class samples that were misclassified in the previous iteration have a higher probability of being selected into this round of iteration, so that the model pays more attention to the samples that are not correctly classified;

[0011] Step 2: Initial sampling probability setting: assign an initial sampling probability to each majority class sample so that the under-sampled data set maintains a balance in the number of positive and negative samples;

[0012] Step 3, iterative sampling update: By evaluating the prediction effect of the current classifier on the majority class samples, the sampling probability of the majority class samples is dynamically adjusted to ensure that more attention is paid to the misclassified majority class samples in each iteration;

[0013] Step 4: Cost-sensitive learning: assign classification error costs according to different sample categories, increase attention to minority class samples, and improve their prediction accuracy;

[0014] Step 5: Ensemble learning and cost voting strategy: Build multiple base classifiers, perform weighted integration of classification results through cost-sensitive ensemble method, and combine cost voting strategy to improve the recognition accuracy of minority class samples;

[0015] Step 6: Model convergence: When the classifier performance improvement is lower than the set threshold, stop the iteration and output the target classifier;

[0016] Step 7: In the actual application of medical data analysis, for disease prediction models with unbalanced data sets, verify the generalization and accuracy of the model.

[0017] Furthermore, the step of dynamically adjusting the sampling probability of majority class samples includes: if a majority class sample is misclassified in the previous round of iteration, increasing its sampling probability in the next round of iteration; if a majority class sample is correctly classified, reducing its sampling probability in the next round of iteration.

[0018] Furthermore, the cost-sensitive learning step includes: first, setting a higher classification error cost for minority class samples so that the model pays more attention to minority class samples and improves their recall rate; second, adjusting the sample classification weights based on the cost-sensitive mechanism so that minority class samples receive more attention during the training process.

[0019] Furthermore, the ensemble learning process includes: using multiple base classifiers to learn data in an iterative process to form multiple classifiers; setting cost weights based on the classification effect of each classifier, and weighted integrating the prediction results of the base classifiers to obtain a comprehensive prediction result.

[0020] Furthermore, the cost weight of the base classifier is calculated in the following manner: based on the prediction accuracy of each classifier on the training set and the recall rate of the minority class samples, a higher weight is given to the efficient classifier.

[0021] Furthermore, the iterative stopping condition of the method is: when the overall prediction performance of the integrated classifier is improved by less than a set threshold in two consecutive iterations, the iteration is stopped and the final target classifier is output.

[0022] Furthermore, the cost voting strategy includes: calculating the cost voting scores of multiple base classifiers for each sample, and selecting the category with the highest score as the final prediction category, thereby improving the recognition accuracy of minority class samples.

[0023] Furthermore, the cost voting strategy dynamically adjusts the voting weight of each base classifier according to its misclassification rate during training, so as to ensure that a more efficient base classifier occupies a larger weight in the final prediction result.

[0024] Furthermore, the method achieves a higher training coverage of majority class samples, reduces the misjudgment rate, and optimizes the prediction performance of majority class samples through multiple sampling and integration processes.

[0025] Furthermore, the method is applicable to disease prediction models in medical data analysis, and is particularly suitable for application scenarios that process unbalanced data sets, and can effectively improve the prediction accuracy of minority disease samples and the stability of the overall model.

[0026] Beneficial effects:

[0027] (1) Improve the prediction performance of minority class samples: The present invention adopts an iterative undersampling strategy, which enables the classification model to focus on minority class samples more effectively during the training process, significantly improving the prediction accuracy and recall rate of minority class samples. This is particularly important for disease prediction models in the medical field, because minority class samples often represent key disease states.

[0028] (2) Enhance the overall accuracy of the model: Combining cost-sensitive learning with ensemble learning not only ensures a high recognition rate for minority class samples, but also improves the model's prediction performance for majority class samples, ensuring that the accuracy of the overall classifier is effectively maintained. By dynamically adjusting the sampling probability of majority class samples, the loss of important features of majority class samples is avoided.

[0029] (3) Strong adaptability: This method has good adaptability and can dynamically adjust the sampling strategy according to the characteristics of the data set and the feedback of the model, so that the classification model is continuously optimized in multiple iterations. This flexibility makes the present invention suitable for a variety of unbalanced data set scenarios, especially in medical data analysis.

[0030] (4) Reduce the misclassification rate: By paying more attention to the misclassified majority class samples, the probability of misclassifying majority class samples as minority class samples is reduced, thereby reducing the misclassification rate of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 Schematic diagram of the process flow of the iterative undersampling cost-sensitive ensemble classification method.

[0032] Figure 2 It is a cost-sensitive ensemble classification method based on iterative undersampling. DETAILED DESCRIPTION

[0033] The technical solution of the present invention is further described in detail below in conjunction with the accompanying drawings.

[0034] See also Figure 1 , Figure 2 The present invention provides a cost-sensitive ensemble classification method based on iterative undersampling, and the detailed steps are as follows:

[0035] Step 1: Sampling method analysis. Figure 1 , 2As shown in the figure, the sampling method plays two key roles: first, it selects samples of the same number as the minority class samples from the majority class samples to construct a data set with a relatively balanced ratio of positive and negative samples, so as to ensure that the base classifier can achieve the ideal classification effect; second, it selects the data set participating in each training by sampling probability, and tries to avoid the majority class samples that have been correctly classified in the previous iteration from entering the current iteration, while the majority class samples that were misclassified in the previous iteration have a higher probability of being selected into this round of iteration, so that the model pays more attention to the samples that are not correctly classified. While saving computing resources, this strategy effectively improves the overall accuracy of the model.

[0036] Step 2: Initialize sampling probability.

[0037] At the beginning of the iteration, the sampling probability of each majority class sample is unknown. This method sets the same initial sampling probability for each majority class sample. In order to ensure the balance of the number of samples in the recombinant samples obtained after sampling, this method assigns each majority class sample an initial sampling probability of Where |B + | is the number of minority class samples, |B - | is the number of majority class samples. The sum of the sampling probabilities of majority class samples is |B + |, and when the sum of the sampling probabilities of the majority class samples is |B + |, it can be guaranteed that the expectation after sampling the majority class samples is |B + |. The proof is as follows: ∑ i E(θ i )=E(∑ i θ i )=|B + |. Therefore, under the initial sampling probability, the expected number of samples drawn from the majority class samples is |B + |, which can ensure that the number of majority class samples and minority class samples in the reorganized samples after sampling is balanced.

[0038] Step 3: Update sampling probability.

[0039] When updating the sampling probability, compare the prediction probability of the majority class samples by the best classifier output in this iteration with the prediction probability of the majority class samples by the best classifier generated in the previous iteration, and then judge the prediction effect of the samples at this time, and update the sampling probability of the majority class samples based on this.

[0040] The output of the classifier F at the tth iteration t (x) For the majority class sample B i The prediction effect of P t,i To represent, let y i is the majority class sample B iThe true label of , when the true label is positive, set y i = +1; when the true label is negative, set y i =-1.

[0041] For each sample B i , classifier F t (x) will output a predicted probability P t,i To represent the classifier's current prediction B i is the probability of being a positive sample. Therefore, for the minority class positive samples, P t,i The larger the value, the better the classifier F t (x) The greater the probability that it belongs to the positive sample, the better the classification effect of the classifier is. For the majority of negative samples, P t,i The larger the value, the better the classifier F t (x) The smaller the probability that it is a negative sample, the worse the classification effect of the classifier is.

[0042] Sampling probability update principle of majority class samples: For any majority class sample B i , if the best classifier F output by this round of iteration t (x) Its predicted probability is better than the best classifier F output in the previous iteration t-1 (x) The predicted probability of it should be small, that is, p t,i -p t-1,i <0, then reduce B before the next iteration i The sampling probability of i According to this principle, before the next iteration, the sampling probability distribution of the majority class samples is updated to D t+1 =(θ t+1,1 ,…,θ t+1,i ,…,θ t+1,|B-| ),i=1,2,…,|B - |, where B i The sampling probability θ in the t+1th iteration t+1,i It is expressed as: Note: For a majority class sample B i , when p t,i -p t-1,i >0, according to the predicted probability P t,i Explanation: At this time, the classifier F output by this round of iteration t (x) for B i The prediction effect is better than the classifier F output of the previous iteration t-1 (x) for B i The prediction effect is poor. Thus increasing B i The sampling probability oft,i -p t-1,i <0, the classifier F output by this iteration t (x) for B i The prediction effect is better than the classifier F output of the previous iteration t-1 (x) for B i The prediction effect is good. Thus reducing B i To ensure that the number of majority class samples drawn is still equal to the number of minority class samples after the sampling probability is updated, use Z t For θ t+1,i To scale:

[0043] Prove that after scaling, the number of majority class samples extracted is still equal to the number of minority class samples. The proof is as follows:

[0044]

[0045] It is concluded that after the sampling probability of the majority class samples is updated, this method can still ensure that the number of majority class samples and the number of minority class samples extracted in the next iteration are balanced.

[0046] Step 4: Cost voting method.

[0047] First, calculate the cost weights of multiple base classifiers, and then for each sample, calculate the cost voting choices of multiple base classifiers for it, and then give the choice with the largest cost votes.

[0048]

[0049] The weight of each base classifier is calculated according to the prediction effect of the base classifier on the training set. Assume that the prediction effect of the i-th classifier on the training set is P i , then the weight is calculated as:

[0050]

[0051] Step 5: A complete overview and process of the iterative undersampling algorithm.

[0052] Input: training set B of unbalanced samples = {(x 1 ,y 1 ),(x 2 ,y 2 ),…,(x N ,y N )},in Output: target classifier F(x).

[0053] (1) Initialize the sampling probability of the majority class samples in B, |B - | is the number of majority class samples, |B + | is the number of minority class samples.

[0054] D 1 =(θ 1,1 ,…,θ 1 ,i,…,θ 1,|B_| ),i=1,2,…,|B - |

[0055]

[0056] (2) Let t be the number of iterations. For t = 2, …, T

[0057] (a) Using the abstract probability distribution D t-1 The majority class samples in the training set are sampled M times with replacement, and then combined with the minority class samples to obtain M reorganized balanced data sets: {B t1 ,…,B tj ,…,B tM},j=1,2,…,M

[0058] (b) Using the base classifier to perform classification learning on the M balanced data sets in (a), we can obtain M learned base classifiers: {f t1 (x),…,f tj (x),…,f tM (x)},j=1,2,…,M

[0059] (c) Calculate the classification effect of M base classifiers on the training data set B, using P as an indicator:

[0060] (d) Integrate these M base classifiers into the classifier output by the t-1 iteration in a weighted manner to form M new integrated classifiers:

[0061] {F t1 (x),...,F tj (x),...,F tM (x)},j=1,2,...,M

[0062] F tj (x) = F t-1 (x)+w tj f tj (x)

[0063]

[0064] (e) Calculate the improvement effect of the M integrated classifiers relative to the classifier output in the previous round:

[0065] {Δβ t1 ,…,Δβ tj ,…,Δβ tM},j=1,2,…,M

[0066]

[0067]

[0068] (f) Select the one with the greatest improvement effect from the M ensemble classifiers as the output of this iteration: F t (x) = argmax j ({Δβ t1 ,…,Δβ tj ,…,Δβ tM}), j = 1, 2, ..., M

[0069] (g) If the improvement effect of the ensemble classifier outputted by this iteration is less than the improvement threshold, it means that the model has converged within the improvement threshold. The iteration is stopped and the classifier F(x) outputted by this iteration is used as the target classifier output: F(x) = F t (x). Otherwise, through F t (x) Update the sampling probability of each majority class sample based on the predicted probability of the majority class sample, and then enter the next iteration: D t+1 =(θ t+1,1 ,…,θ t+1,i ,…,θ t+1,|B-| ),i=1,2,…,|B - |

[0070] Step 6: Compare and analyze various sampling methods with this method

[0071] The data set uses a stroke imbalance data set provided on Kaggle

[0072] Sampling method Recall Precision F1 IU-CI 0.8018 0.8243 0.7872 RENN 0.4734 0.8432 0.6064 ENN 0.5042 0.6724 0.5763 CC 0.6345 0.6821 0.6574 NM 0.4251 0.8173 0.5592 TL 0.6132 0.7324 0.6675 RU 0.4834 0.8012 0.6031

[0073] Description: IU-CI (Iterative Undersampling-Cost-sensitive integration), REEE (Repeated Edited Nearest Neighbours), ENN (Edited-Nearest-Neighbours), CC (Cluster-Centroids), NM (Near Miss), TL (Tomek Links)

[0074] Analysis: Other undersampling methods are characterized by high recall but low precision. This is because these methods discard some information about majority class samples when constructing a balanced dataset, while retaining minority class samples completely, so that the distribution of minority class samples in the reorganized dataset remains unchanged, while the distribution of majority class samples changes significantly. As a result, the model's classification effect on minority class samples improves, but some majority class samples are misclassified as minority classes, resulting in a decrease in overall precision.

[0075] On the basis of retaining the information of minority class samples, this method not only extracts multiple groups of majority class samples and minority class samples, but also increases the sampling probability of misclassified majority class samples through iteration, ensuring the training coverage of majority class samples and focusing on samples with poor classification effect. Therefore, the IU-CI method effectively improves the overall precision while ensuring a high recall rate.

Claims

1. A cost-sensitive ensemble classification method based on iterative undersampling, characterized by: The following steps are involved: Step 1, Sampling method analysis: The sampling method is used to select samples of the same number as the minority class samples from the majority class samples, and construct a data set with a relatively balanced ratio of positive and negative samples, so as to ensure that the base classifier can achieve the ideal classification effect; secondly, it is used to screen the data set participating in each training by sampling probability, and try to avoid the majority class samples that have been correctly classified in the previous iteration from entering the current iteration, while the majority class samples that were misclassified in the previous iteration have a higher probability of being selected into this round of iteration, so that the model pays more attention to the samples that are not correctly classified; Step 2: Initial sampling probability setting: assign an initial sampling probability to each majority class sample so that the under-sampled data set maintains a balance in the number of positive and negative samples; Step 3, iterative sampling update: By evaluating the prediction effect of the current classifier on the majority class samples, the sampling probability of the majority class samples is dynamically adjusted to ensure that more attention is paid to the misclassified majority class samples in each iteration; Step 4: Cost-sensitive learning: assign classification error costs according to different sample categories, increase attention to minority class samples, and improve their prediction accuracy; Step 5: Ensemble learning and cost voting strategy: Build multiple base classifiers, perform weighted integration of classification results through cost-sensitive ensemble method, and combine cost voting strategy to improve the recognition accuracy of minority class samples; Step 6: Model convergence: When the classifier performance improvement is lower than the set threshold, stop the iteration and output the target classifier; Step 7: In the actual application of medical data analysis, for disease prediction models with unbalanced data sets, verify the generalization and accuracy of the model.

2. The cost-sensitive ensemble classification method based on iterative undersampling according to claim 1, characterized in that: In step three, the step of dynamically adjusting the sampling probability of majority class samples includes: if a majority class sample is misclassified in the previous round of iteration, then increasing its sampling probability in the next round of iteration; if a majority class sample is correctly classified, then reducing its sampling probability in the next round of iteration.

3. The cost-sensitive ensemble classification method based on iterative undersampling according to claim 2, characterized in that: In the step 4, the cost-sensitive learning step includes: first, setting a higher classification error cost for minority class samples so that the model pays more attention to minority class samples and improves their recall rate; second, adjusting the sample classification weights based on the cost-sensitive mechanism so that minority class samples receive more attention during the training process.

4. The cost-sensitive ensemble classification method based on iterative undersampling according to claim 3, characterized in that: In the step five, the integrated learning process includes: using multiple base classifiers to learn the data in an iterative process to form multiple classifiers; setting cost weights based on the classification effect of each classifier, and weighted integrating the prediction results of the base classifiers to obtain a comprehensive prediction result.

5. The cost-sensitive ensemble classification method based on iterative undersampling according to claim 4, characterized in that: The cost weight of the base classifier is calculated in the following way: based on the prediction accuracy of each classifier on the training set and the recall rate of the minority class samples, a higher weight is given to the efficient classifier.

6. The cost-sensitive ensemble classification method based on iterative undersampling according to claim 5, characterized in that: In step six, the iteration stopping condition is: when the overall prediction performance of the integrated classifier is improved by less than a set threshold in two consecutive iterations, the iteration is stopped and the final target classifier is output.

7. The cost-sensitive ensemble classification method based on iterative undersampling according to claim 1, characterized in that: In the step 5, the cost voting strategy includes: calculating the cost voting scores of multiple base classifiers for each sample, and selecting the category with the highest score as the final prediction category, thereby improving the recognition accuracy of minority class samples.

8. The cost-sensitive ensemble classification method based on iterative undersampling according to claim 7, characterized in that: The cost voting strategy dynamically adjusts the voting weight of each base classifier according to its misclassification rate during training to ensure that a more efficient base classifier has a greater weight in the final prediction result.

9. The cost-sensitive ensemble classification method based on iterative undersampling according to claim 1, characterized in that: The method is applicable to disease prediction models in medical data analysis, and is particularly suitable for application scenarios that process unbalanced data sets. It can effectively improve the prediction accuracy of minority disease samples and the stability of the overall model.

10. The cost-sensitive ensemble classification method based on iterative undersampling according to claim 1, characterized in that: The specific steps are: (1) Input the training set B of unbalanced samples = {( x1 ,y1),(x2,y2),…,(x N ,y N )},in y i ∈γ={-1,+1}; Output: target classifier F(x); (2) Initialize the sampling probability of the majority class samples in the training set B, |B _ | is the number of majority class samples, |B + | is the number of minority class samples; the sampling probability distribution is: in, (3) Let t be the number of iterations, for t = 2,…,T; (a) Using the abstract probability distribution D t-1 The majority class samples in the training set B are sampled M times with replacement, and then combined with the minority class samples to obtain M reorganized balanced data sets: {B t1 ,...,B tj ,...,B tM },j=1,2,...,M (b) Use the base classifier to perform classification learning on the M balanced data sets in (a) to obtain M learned base classifiers: {f t1 (x),...,f tj (x),...,f tM (x)},j=1,2,...,M (c) Calculate the classification effect of M base classifiers on the training data set B, using P as an indicator: {a t1 ,...,a tj ,...,a tM }, j=1, 2,..., M α tj =P score (f tj (x),B) (d) Integrate these M base classifiers into the classifier output by the t-1 iteration in a weighted manner to form M new integrated classifiers: {F t1 (x),...,F tj (x),...,F tM (x)},j=1,2,...,M F tj (x)=F t-1 (x)+w tj f tj (x) (e) Calculate the improvement effect of these M integrated classifiers relative to the classifier output in the previous round: {Db} t1 ,...,Dv tj ,...,Dv tM }, j=1, 2,..., M (f) Select the classifier with the greatest improvement effect from the M ensemble classifiers as the output of this iteration: F t (x)=arg max j ({Δβ t1 ,...,Dv tj ,...,Dv tM }), j=1, 2,..., M (g) If the improvement effect of the ensemble classifier outputted by this iteration is less than the improvement threshold, it means that the model has converged within the improvement threshold. The iteration is stopped and the classifier F(x) outputted by this iteration is used as the target classifier output: F(x) = F t (x); otherwise, through F t (x) Update the sampling probability of each majority class sample based on the predicted probability of the majority class sample, and then enter the next iteration:

Images