Hybrid model-based unbalanced data classification method and device, equipment and medium

Abstract

The invention relates to an unbalanced data classification method based on a hybrid model. The method comprises the steps of obtaining a to-be-classified unbalanced data set; calling a twin support vector machine algorithm to form a hybrid model combination algorithm, and performing inner layer iteration; a classification node of the cascade model receives the sample set, a hybrid model combinatorial algorithm allocates an initial weight, and a pre-training set is generated according to the initial weight for training to obtain a component classifier; calculating the AUC value of the component classifier on the pre-selected training set, updating the weight distribution of the sample by taking the AUC value as a reference until the training is completed, and ending the inner layer iteration to obtain the component classifier; integrating all component classifiers into a node classifier to obtain a next sample set, transmitting the next sample set to a next classification node, and performing circulation to obtain the node classifier until the classification node circulation of the cascade model is finished; and integrating the node classifiers into a final classifier, and classifying the to-be-classified unbalanced data set to obtain a classification result. The unbalanced data classification method provided by the invention is high in calculation precision and stable in performance.

Description

Technical Field

[0001] The present invention relates to the technical field of data classification, and in particular to a method, device, equipment and medium for unbalanced data classification based on a hybrid model. Background Art

[0002] In an unbalanced data set with two categories of classification, the category with a relatively small number of samples is called a rare class, and the category with a relatively large number of samples is called a common class. The classification problem of unbalanced data sets exists in large numbers in real life, and improving the classification accuracy has important application value. For example, in the massive data of bridge operation monitoring and maintenance, how to quickly and accurately identify abnormal data and accurately classify it, and determine the real-time status of various structural parts of the bridge by analyzing and identifying the internal correlation of each data is very important.

[0003] Traditional data classification methods can achieve good classification results under the premise that the category distribution of a given classification data set is basically balanced. However, when dealing with unbalanced data sets, there are often the following problems: the performance of the classifier is greatly reduced, that is, the calculation accuracy of the entire data set is not high; the classifier has a large bias, that is, the recognition rate of rare classes is very low; when facing large-scale data, the classifier is inefficient and the calculation complexity is high. That is, the traditional data classification method has a high recognition rate for common classes, but a very low recognition rate for rare classes. Summary of the invention

[0004] The present application aims to solve the technical problem that the recognition rate of the related technical data classification method for common classes is high, but the recognition rate for rare classes is very low.

[0005] The present invention provides a method for classifying imbalanced data based on a hybrid model, comprising the following steps:

[0006] Obtain an unbalanced data set to be classified as the initial classification node of the cascade model, wherein the unbalanced data set includes a rare class sample set P of m samples and a majority class sample set N of n samples, and m≤n;

[0007] Calling the twin support vector machine algorithm Twin-SVM as a weak classifier of the AdaBoost algorithm to form a hybrid model combination algorithm Boost-TwinSVM, and performing K inner layer iterations of each classification node in the cascade model;

[0008] The i-th (i>1) classification node of the cascade model i Received from the previous node Node i-1 The sample set W passed in i, the hybrid model combination algorithm Boost-TwinSVM is the sample set W i Each sample in the According to the initial weight Select the corresponding samples to generate a pre-training set, train the pre-training set, and obtain the corresponding component classifier h (i,1) ;

[0009] Based on the component classifier h (i,1) , calculate the component classifier h (i,1) The AUC value on the pre-selected training set is used to update the weight distribution of the sample based on the AUC value until all samples are trained, and the K inner layer iterations are completed to obtain a series of component classifiers h (i,1) ,…,h (i,k) ;

[0010] All component classifiers are integrated into the node classifier H i , get the next sample set W i+1 And pass it to the next classification node Node i+1 , the above process is repeated to obtain a series of node classifiers H 1 ,H 2 …,H I , until the classification node loop of the cascade model CascadeModel ends;

[0011] A series of node classifiers H 1 ,H 2 …,H I The final classifier H(X) is integrated, and the unbalanced data set to be classified is classified according to the final classifier H(X) to obtain a classification result.

[0012] In one embodiment, the hybrid model combination algorithm Boost-TwinSVM is the sample set W i Each sample in the Based on the hybrid model combination algorithm Boost-TwinSVM, set

[0013] In one embodiment, the component classifier h (i,1) The calculation formula of the AUC value on the pre-selected training set is:

[0014]

[0015] Among them, m is the number of samples in the rare class sample set, n is the number of samples in the majority class sample set, q is the number of all known misclassified samples, and p is the number of misclassified samples in the majority class.

[0016] In one embodiment, the weight distribution of the sample is updated based on the AUC value.

[0017] The calculation formula is:

[0018] Among them, a (i,k) is the AUC value of the kth iteration cycle in the i-th classification node, Z k+1 is the normal factor, x i is the feature vector of the i-th sample, y i is the label or target value of the i-th sample.

[0019] In one embodiment, the integration of all component classifiers into a node classifier H i , get the next sample set W i+1 And pass it to the next classification node Node i+1 include:

[0020] A series of component classifiers h (i,1) ,…,h (i,k) Ensembled as node classifier H i ,

[0021] The node classifier H i The calculation formula is:

[0022] Among them, a (i,k) is the AUC value of the kth iteration cycle in the i-th classification node;

[0023] Based on the node classifier, the sample set W i The majority class sample set is trained, and the majority class samples that have been correctly classified in the majority class sample set are excluded according to the preset false alarm rate to obtain the next sample set W i+1 , passed to the next classification node Node of the cascade model i+1 .

[0024] In one implementation, the classification node loop termination condition of the cascade model is: the number of samples in the majority class sample set is not greater than the number of samples in the rare class sample set or a preset false alarm rate is reached.

[0025] In one embodiment, the series of node classifiers H 1 ,H 2 …,H I The final classifier H(X) is integrated.

[0026] The calculation formula of the final classifier H(X) is:

[0027] The embodiment of the present application further provides an unbalanced data classification device based on a hybrid model, and the unbalanced data classification device based on a hybrid model includes:

[0028] The data set acquisition module is configured to: acquire an unbalanced data set to be classified as an initial classification node of the cascade model CascadeModel, wherein the unbalanced data set includes a rare class sample set P of m samples and a majority class sample set N of n samples, and m≤n;

[0029] The first computing module is configured to: call the twin support vector machine algorithm Twin-SVM as a weak classifier of the AdaBoost algorithm to form a hybrid model combination algorithm Boost-TwinSVM, and perform K inner layer iterations of each classification node in the cascade model;

[0030] The second computing module is configured as: the i-th (i>1) classification node Node of the cascade model i Received from the previous node Node i-1 The sample set W passed in i , the hybrid model combination algorithm Boost-TwinSVM is the sample set W i Each sample in the According to the initial weight Select the corresponding samples to generate a pre-training set, train the pre-training set, and obtain the corresponding component classifier h (i,1) ;

[0031] The third computing module is configured to: (i,1) , calculate the component classifier h (i,1) The AUC value on the pre-selected training set is used to update the weight distribution of the sample based on the AUC value until all samples are trained, and the K inner layer iterations are completed to obtain a series of component classifiers h (i,1) ,…,h (i,k) ;

[0032] The fourth computing module is configured to: integrate all component classifiers into a node classifier H i , get the next sample set W i+1 And pass it to the next classification node Node i+1 , the above process is repeated to obtain a series of node classifiers H 1 ,H 2 …,H I, until the classification node cycle of the cascade model Cascade Model ends;

[0033] The final classification module is configured as follows: a series of node classifiers H 1 ,H 2 …,H I The final classifier H(X) is integrated, and the unbalanced data set to be classified is classified according to the final classifier H(X) to obtain a classification result.

[0034] An embodiment of the present application also provides an unbalanced data classification device based on a hybrid model, the unbalanced data classification device based on a hybrid model comprising: a memory, a processor, and an unbalanced data classification program stored in the memory and executable on the processor, the unbalanced data classification program being configured to implement the steps of the unbalanced data classification method based on the hybrid model as described in any one of the above items.

[0035] An embodiment of the present application further provides a storage medium, on which an imbalanced data classification program is stored. When the imbalanced data classification program is executed by a processor, the steps of the imbalanced data classification method based on the hybrid model as described in any one of the above are implemented.

[0036] The beneficial effects brought by the technical solution provided in the embodiments of the present application include:

[0037] The present application provides a method for classifying unbalanced data based on a hybrid model, combining the advantages of the boosting algorithm (AdaBoost), the twin support vector machine (Twin-SVM) and the cascade model (Cascade Model) to form a CBT method (Cascade-Boost-TwinSVM). Specifically, a double-layer iteration is adopted, the outer layer is the cascade model iteration, and the inner layer introduces the twin support vector machine (Twin-SVM) with better classification performance as the weak classifier of the AdaBoost algorithm to form the Boost-TwinSVM algorithm, and then the Boost-TwinSVM algorithm is embedded in each layer of the cascade model and trained alternately to update the corresponding sample weights and obtain the corresponding node classifier, and then the node classifier is used to gradually exclude the majority class nodes that have been correctly classified in the data set, retain the majority class samples that are misclassified and all rare class samples, and gradually iterate to form a component classifier, and use the AUC value of each component classifier on the pre-selected training set as the benchmark for updating the sample weight in the next cycle, until the node classifier on the last node is obtained, and integrated into the final classifier, and the final classification result is obtained by classification according to the final classifier. The classification method provided in the embodiment of the present application has high calculation accuracy, low calculation complexity, stable performance and is suitable for processing large-scale unbalanced data. It can effectively improve the recognition rate of rare classes while ensuring that the overall classification effect is improved or not reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0039] Figure 1 The figure is a flow chart of an imbalanced data classification method based on a hybrid model in one embodiment of the present invention.

[0040] Figure 2 The figure is a schematic diagram of the device structure of the hardware operating environment in one embodiment of the present invention.

[0041] Figure 3 4 is a comparison chart of the precision rates of three algorithms in one embodiment of the present invention.

[0042] Figure 4 4 is a comparison chart of the recall rates of three algorithms in one embodiment of the present invention.

[0043] Figure 5 This is a comparison chart of the AUC values ​​of three algorithms in one embodiment of the present invention. DETAILED DESCRIPTION

[0044] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0045] like Figure 1 As shown, Figure 1 The figure is a flow chart of an imbalanced data classification method based on a hybrid model in one embodiment of the present invention.

[0046] This embodiment provides an imbalanced data classification method based on a hybrid model, comprising the following steps:

[0047] Step S1, obtaining an unbalanced data set to be classified as the initial classification node of the cascade model, the unbalanced data set includes a rare class sample set P of m samples and a majority class sample set N of n samples, and m≤n;

[0048] Step S2, calling the twin support vector machine algorithm Twin-SVM as the weak classifier of the AdaBoost algorithm to form a hybrid model combination algorithm Boost-TwinSVM, and performing K inner layer iterations of each classification node in the cascade model;

[0049] Step S3, Cascade Model Cascade Model i (i>1) classification node Node i Received from the previous node Node i-1 The sample set W passed in i , the hybrid model combination algorithm Boost-TwinSVM is the sample set W i Each sample in the According to the initial weight Select the corresponding samples to generate a pre-training set, train the pre-training set, and obtain the corresponding component classifier h (i,1) ;

[0050] Step S4: Based on the component classifier h (i,1) , calculate the component classifier h (i,1) The AUC value on the pre-selected training set is used as the basis to update the weight distribution of the samples until all samples are trained, ending K inner layer iterations and obtaining a series of component classifiers h (i,1) ,…,h (i,k) ;

[0051] Step S5: integrate all component classifiers into the node classifier H i , get the next sample set W i+1 And pass it to the next classification node Node i+1 , the above process is repeated to obtain a series of node classifiers H 1 ,H 2 …,H I , until the classification node loop of the cascade model CascadeModel ends;

[0052] Step S6: a series of node classifiers H 1 ,H 2 …,H I The final classifier H(X) is integrated and the imbalanced dataset to be classified is classified according to the final classifier H(X) to obtain the classification result.

[0053] This embodiment provides an unbalanced data classification method based on a hybrid model, combining the advantages of the boosting algorithm (AdaBoost), the twin support vector machine (Twin-SVM) and the cascade model (Cascade Model) to form a CBT method (Cascade-Boost-TwinSVM). Specifically, a double-layer iteration is adopted, the outer layer is the cascade model iteration, and the inner layer introduces the twin support vector machine (Twin-SVM) with better classification performance as the weak classifier of the AdaBoost algorithm to form the Boost-TwinSVM algorithm, and then the Boost-TwinSVM algorithm is embedded in each layer of the cascade model and trained alternately to update the corresponding sample weights and obtain the corresponding node classifier, and then the node classifier is used to gradually exclude the majority class nodes that have been correctly classified in the data set, retain the majority class samples that are misclassified and all rare class samples, and gradually iterate to form a component classifier, and the AUC value of each component classifier on the pre-selected training set is used as the benchmark for updating the sample weight in the next cycle, until the node classifier on the last node is obtained, and integrated into the final classifier, and the final classification result is obtained by classification according to the final classifier. The classification method provided in the embodiment of the present application has high calculation accuracy, low calculation complexity, stable performance and is suitable for processing large-scale unbalanced data. It can effectively improve the recognition rate of rare classes while ensuring that the overall classification effect is improved or not reduced.

[0054] Each step is described and explained in detail below.

[0055] In step S1, the unbalanced data set to be classified is used as the initial classification node of the cascade model.

[0056] The cascade model can be seen as a hierarchical structure consisting of a set of node classifiers. All samples to be classified start from the first node N. 1 At the beginning, each cascade node receives samples of a subset of all rare classes and majority classes passed from the previous node. Generally speaking, when the majority class samples are correctly classified, they will not be passed to the next cascade node for further training, while when the rare class samples are correctly classified, they will be passed to the next cascade node for further training, and all the samples that are misclassified will be passed to the next cascade node for further training. i The training sample set is composed of N i-1 The passed samples include all rare class samples and the majority class samples that are misclassified. At the beginning, it is relatively easy for each node to obtain a relatively ideal classifier through training. As the training progresses, the number of majority class samples decreases, and the last node N n Contains all rare class samples and misclassified majority class samples. This node is called the optimal sample point, and the corresponding component classifier is also regarded as the optimal classifier. When dynamically dividing the data set, each node requires a corresponding detection rate d i and false alarm rate f i In practical applications, i The value of is required to be relatively high, because d i The larger the value of is, the more beneficial it is to the final classification result; and f i It indicates the proportion of the majority class that is misclassified as the rare class. It can effectively control the proportion of majority class samples that are excluded each time. In the cascade model, the nodes generated earlier exclude more common class samples due to the large class imbalance in the training set. This feature enables the cascade model to significantly improve efficiency when solving large-scale data learning problems.

[0057] In step S2, the twin support vector machine algorithm Twin-SVM is called as the weak classifier of the AdaBoost algorithm to form a hybrid model combination algorithm Boost-TwinSVM.

[0058] Twin Support Vector Machine (Twin-SVM) is a machine learning method developed on the basis of support vector machine (SVM). Compared with traditional SVM, Twin-SVM solves the binary classification problem by solving two sets of smaller quadratic programming problems, thereby significantly improving the computational speed and classification performance. The basic idea of ​​Twin-SVM is to find two non-parallel hyperplanes so that each hyperplane is close to the sample points of this class and far away from the sample points of the other class. This method decomposes a single large quadratic programming problem into two smaller quadratic programming problems, thereby improving computational efficiency. The classification speed of Twin-SVM is about 4 times faster than that of standard SVM, and due to its unique optimization method, it shows better generalization ability when dealing with certain problems. If the number of samples of a certain class in the data set is higher than that of samples of other classes, TwinSVM can also set different penalty coefficients for the misclassified samples of these two classes respectively to improve the accuracy of classification training. TwinSVM has obvious advantages over SVM in dealing with the imbalance problem of the accuracy of the minority class, and its prediction accuracy is significantly higher than other current classification algorithms.

[0059] The existing Boost-SVM algorithm gradually increases the weight of the wrongly classified samples and reduces the weight of the correctly classified samples during the iteration process, so that the next iterative classification process pays more attention to those difficult-to-classify samples. However, the disadvantage of the Boost-SVM algorithm is that its stability is not high. When processing unbalanced data or data with high class overlap, its advantages are not obvious; in addition, when the scale of the unbalanced data set is large, the efficiency of the Boost-SVM algorithm will become very low. Compared with the existing Boost-SVM algorithm, the Boost-TwinSVM algorithm provided in the embodiment of the present application overcomes the defects and deficiencies in the above classification process and can significantly improve the classification accuracy of minority classes.

[0060] The hybrid model combination algorithm Boost-TwinSVM performs K inner iterations of each classification node in the cascade model, which can make more trade-offs in the selection of rare classes and majority classes, thereby changing the simple form of the cascade model that selects all rare classes, so that the final training set contains sample points in both rare and majority classes that have a greater impact on classification.

[0061] In one embodiment, in step S3, the hybrid model combination algorithm Boost-TwinSVM is a sample set W i Each sample in the Based on the hybrid model combination algorithm Boost-TwinSVM, set That is, the initial weight of the rare class sample set is 1 / 4m, and the initial weight of the majority class sample set is 1 / 4n.

[0062] Through the above scheme, initial weights are assigned to the new pre-selected training set each time the inner layer iteration is entered. By setting specific initial weights, the weights of samples of different categories (such as rare classes and majority classes) are increased during the training process, thereby increasing the classification discrimination and being able to better identify the differences between different categories, thereby reducing misclassification and improving the accuracy and reliability of classification.

[0063] Furthermore, according to the weight distribution of samples in the pre-selected training set, the optimal sample points are gradually selected as the final training set for TwinSVM training to obtain a component classifier.

[0064] In one embodiment, in step S4, the component classifier h (i,1) The calculation formula for the AUC value on the pre-selected training set is:

[0065]

[0066] Among them, m is the number of samples in the rare class sample set, n is the number of samples in the majority class sample set, q is the number of all known misclassified samples, and p is the number of misclassified samples in the majority class.

[0067] AUC (Area Under the ROC Curve) is an evaluation criterion for classification algorithms. It is insensitive to category distribution and error cost, and is intuitive and understandable. It has obvious advantages in evaluating and comparing cost-sensitive classifiers.

[0068] In one embodiment, in step S4, the weight distribution of the samples is updated based on the AUC value.

[0069] The calculation formula is:

[0070] Among them, a (i,k) is the AUC value of the kth iteration cycle in the i-th classification node, also the component classifier h (i,k) The proportion of Z in the final classifier H(X) k+1 is the normal factor, xi is the eigenvector of the i-th sample, y i is the label or target value of the i-th sample, such as a category label, indicating which category the sample belongs to.

[0071] Specifically, according to the initial weight Select the corresponding samples to generate a pre-training set, train the pre-training set, and obtain the corresponding component classifier h (i,k) , that is, the weak classifier h (i,k);

[0072] Calculate h (i,k) The error rate ε k , the calculation formula is:

[0073] Let a (i,k) =1 / 2ln[(1-ε k ) / ε k ];

[0074] Among them, a (i,k) is the AUC value of the kth iteration cycle in the i-th classification node, also the component classifier h (i,k) The proportion in the final classifier H(X);

[0075] Update the weight distribution of samples based on the AUC value.

[0076] The formula is:

[0077] Among them, Z k+1 is the normal factor, which ensures

[0078] Through the above scheme, the weight distribution of samples is updated based on the AUC value of each component classifier, so that the next component classifier can obtain more appropriate and informative samples as training sets.

[0079] In step S4, based on the component classifier h (i,1) , calculate the component classifier h (i,1) The AUC value on the pre-selected training set is used to update the sample weight distribution based on the AUC value, thereby obtaining a new sample weight distribution In the next cycle, the hybrid model combination algorithm Boost-TwinSVM is based on To select a new training sample set for training and get h (i,2) and The hybrid model combination algorithm Boost-TwinSVM in the node is iterated until a series of component classifiers h are obtained. (i,1) ,…,h (i,k) .

[0080] In one embodiment, in step S5, all component classifiers are integrated into a node classifier H i , get the next sample set W i+1 And pass it to the next classification node Node i+1 include:

[0081] A series of component classifiers h (i,1) ,…,h (i,k) Ensembled as node classifier Hi ,

[0082] Node Classifier H i The calculation formula is:

[0083] Among them, a (i,k) is the AUC value of the kth iteration cycle in the i-th classification node;

[0084] Based on the node classifier, the sample set W i The majority class sample set is trained, and the majority class samples that have been correctly classified in the majority class sample set are excluded according to the preset false alarm rate to obtain the next sample set W i+1 , passed to the next classification node of the cascade model Cascade Model i+1 .

[0085] In one embodiment, the classification node loop termination condition of the cascade model is: the number of samples in the majority class sample set is not greater than the number of samples in the rare class sample set or a preset false alarm rate is reached.

[0086] The condition for the classification node loop of the Cascade Model to end is: the number of samples in the majority class sample set is not greater than the number of samples in the rare class sample set. This condition may be difficult to meet in some special data sets. The number of nodes of the Cascade Model can be effectively controlled by setting an appropriate false alarm rate f.

[0087] Specifically, the Cascade Model uses the parameter false alarm rate f as the end condition of the outer loop, and then controls the number of outer nodes, so that the total number of iterations of the outer and inner layers is the same in each classification process, even if the number of component classifiers obtained for each classification is the same. The false alarm rate f represents the upper limit of the ratio of the majority class sample set that is excluded at the end of classification at each node. Its value has a range. As long as it is within this range, a relatively ideal cascade model can be obtained. Its representation is: f = (nq) / n, where, if a two-category data set containing m rare classes and n majority classes is given, q is the number of all known misclassified samples.

[0088] Through the above scheme, when the cascade model is used in the outer iteration, the node classifier obtained in each iteration of the cascade model is integrated by all the component classifiers at the node, and is only used to predict the majority class sample set, and according to the prediction results, the majority class samples with higher classification accuracy (correctly classified) are gradually excluded from the majority class sample set, and the majority class samples that are difficult to classify (misclassified by the previous node classifier) ​​and all rare class samples are passed to the inner layer hybrid model combination algorithm Boost-TwinSVM as pre-selected training sets. The samples in the pre-selected training set are the samples that have a greater impact on the classification results. As the iteration proceeds (the cascade model gradually excludes the majority class samples with higher classification accuracy), the inner layer hybrid model combination algorithm Boost-TwinSVM obtains a pre-selected training set with increasingly balanced data categories, thereby making it easier to obtain the feature information of the rare class.

[0089] In one embodiment, in step S6, a series of node classifiers H 1 ,H 2 …,H I The final classifier H(X) is integrated.

[0090] The calculation formula of the final classifier H(X) is:

[0091] Through the above scheme, the advantages of the cascade model and the lifting technology are combined to select the most informative sample points as the training set, focusing on training rare classes and the majority class samples that are often misclassified, thereby effectively handling the problem of imbalanced data classification.

[0092] The embodiment of the present application further provides an unbalanced data classification device based on a hybrid model, and the unbalanced data classification device based on a hybrid model includes:

[0093] The data set acquisition module is configured to: acquire an unbalanced data set to be classified as an initial classification node of the cascade model CascadeModel, wherein the unbalanced data set includes a rare class sample set P of m samples and a majority class sample set N of n samples, and m≤n;

[0094] The first computing module is configured to: call the Twin Support Vector Machine algorithm Twin-SVM as a weak classifier of the AdaBoost algorithm to form a hybrid model combination algorithm Boost-TwinSVM, and perform K inner layer iterations of each classification node in the Cascade Model;

[0095] The second computing module is configured as: the i-th (i>1) classification node of the cascade model i Received from the previous node Node i-1The sample set W passed in i , the hybrid model combination algorithm Boost-TwinSVM is the sample set W i Each sample in the According to the initial weight Select the corresponding samples to generate a pre-training set, train the pre-training set, and obtain the corresponding component classifier h (i,1) ;

[0096] The third computing module is configured to: based on the component classifier h (i,1) , calculate the component classifier h (i,1) The AUC value on the pre-selected training set is used as the basis to update the weight distribution of the samples until all samples are trained, ending K inner layer iterations and obtaining a series of component classifiers h (i,1) ,…,h (i,k) ;

[0097] The fourth computing module is configured to: integrate all component classifiers into a node classifier H i , get the next sample set W i+1 And pass it to the next classification node Node i+1 , the above process is repeated to obtain a series of node classifiers H 1 ,H 2 …,H I , until the classification node cycle of the cascade model ends;

[0098] The final classification module is configured as follows: a series of node classifiers H 1 ,H 2 …,H I The final classifier H(X) is integrated and the imbalanced dataset to be classified is classified according to the final classifier H(X) to obtain the classification result.

[0099] The steps implemented by each functional module of the imbalanced data classification device refer to the steps of the imbalanced data classification method provided in the embodiment of the present application, and will not be repeated here.

[0100] An embodiment of the present application also provides an unbalanced data classification device based on a hybrid model. The unbalanced data classification device based on a hybrid model includes: a memory, a processor, and an unbalanced data classification program stored in the memory and executable on the processor. The unbalanced data classification program is configured to implement the steps of the above-mentioned unbalanced data classification method based on the hybrid model.

[0101] Specifically, Figure 2As shown, the device includes: a processor 1001, such as a CPU, a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. The communication bus 1002 is used to realize connection and communication between these components.

[0102] The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a Wi-Fi interface). The memory 1005 may be a high-speed RAM memory or a stable memory (Non-Volatile Memory), such as a disk memory. The memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0103] Those skilled in the art will understand that Figure 2 The device structure shown in the figure does not constitute a limitation of the device, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0104] like Figure 2 As shown, the memory 1005 as a storage medium may include an operating device, a network communication module, a user interface module, and an unbalanced data classification program.

[0105] The device of the present invention calls the imbalanced data classification program stored in the memory 1005 through the processor 1001, and executes each operation of the above-mentioned imbalanced data classification method, which will not be described in detail here.

[0106] An embodiment of the present application further provides a storage medium, on which an unbalanced data classification program is stored. When the unbalanced data classification program is executed by a processor, the steps of the above-mentioned unbalanced data classification method based on the hybrid model are implemented.

[0107] Those skilled in the art will appreciate that all or part of the steps in the above-mentioned implementation method can be accomplished by instructing related hardware through a program, and the program is stored in a storage medium, including a number of instructions for enabling a device (which may be a single-chip microcomputer, chip, etc.) or a processor to execute all or part of the steps of the method described in each implementation method of the present application; and the aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, and other media that can store program codes.

[0108] The embodiment of the present application also uses two other evaluation criteria: recall rate R and precision rate P. The recall rate refers to the ratio of the amount of relevant information retrieved to the total amount of relevant information in the system, and the precision rate refers to the ratio of the amount of relevant information retrieved to the total amount of information retrieved.

[0109] Specifically, if a two-category data set containing m rare classes and n majority classes is given, after all the output results are obtained, it is known that the number of all misclassified samples is q, of which the number of misclassified samples of the majority class is p, then the number of misclassified samples of the rare class is qp, and the calculation formulas for recall and precision are as follows:

[0110] Recall Precision

[0111] A specific embodiment is provided below, and is compared with an existing algorithm to illustrate the technical effect of the embodiment of the present application.

[0112] Take the classification process of the data set on the UCI Machine Learning Database (http: / / archive.ics.uci.edu / ml / ) as an example.

[0113] Input: rare class set P containing m samples, majority class set N containing n samples, and m≤n; false alarm rate f of CascadeModel, Cascade Model node cycle i=1,2…I; number of iterations k=1,2…K of Boost-TwinSVM algorithm.

[0114] Output: Integrate the classifiers of a series of component classifiers formed in the iterative process of the Boost-TwinSVM algorithm into a node classifier, then combine all node classifiers into the final classifier and test the input data set, and output the test results;

[0115] Initialization: Assign initial weights: Let Set the initial parameters of the Boost-TwinSVM algorithm; the number of nodes of CascadeModel i = 0, the number of iterations of the Boost-TwinSVM algorithm k = 0;

[0116] Do While(m <n)

[0117] The number of nodes i = i + 1;

[0118] According to the initial weight distribution rule in the Boost-TwinSVM algorithm, different weights are assigned to the rare class and the majority class to obtain the initial weight distribution

[0119] Number of cycles k = k + 1;

[0120] Boost-TwinSVM algorithm is based on the latest sample weight distribution To select the corresponding samples for training to obtain a component classifier h (i,k) , and then adjust the weight distribution according to the corresponding AUC value to obtain

[0121] When k=K, the loop ends and a series of component classifiers h are obtained. (i,1) ,…,h (i,k) By formula

[0122] Ensembled as node classifier H i ;

[0123] Use H i Classify the majority class sample set N, and gradually exclude the correctly classified samples in N through the false alarm rate f;

[0124] End loop Do;

[0125] Get the number of Cascade Model nodes i generated during the training process;

[0126] Set all node classifiers H 1 ,H 2 …H i According to their respective weights, they are integrated into the final classifier H(X);

[0127] The classifier H(X) is based on the formula Calculate the classification results on the corresponding input data set.

[0128] Eight data sets were selected from the UCI machine learning database, and the classification results of the CBT algorithm provided in the embodiment of the present application were compared with those of the existing Boost-SVM and BalanceCascade algorithms.

[0129] The Boost-SVM algorithm gradually increases the weight of samples that are misclassified and reduces the weight of samples that are correctly classified during the iteration process, so that the next iteration classification process pays more attention to samples that are difficult to classify. However, the disadvantage of the Boost-SVM algorithm is that it is not very stable. When processing unbalanced data or data with high class overlap, its advantages are not obvious; in addition, when the scale of the unbalanced data set is large, the efficiency of the Boost-SVM algorithm will become very low.

[0130] The BalanceCascade algorithm is an algorithm that combines the advantages of decision trees and cascade models. It has a good effect in dealing with imbalanced data classification problems. Its essence is to use samples that have a greater impact on the classification results to dynamically adjust the category distribution of the data set. The BalanceCascade algorithm uses the decision tree as a weak classifier for the cascade model, and only performs one decision tree classification at each node of the cascade model to obtain the node classifier. Since the decision tree treats common and rare class samples equally when classifying, it does not consider the microscopic characteristics of the data set, nor does it adopt a weight distribution strategy to treat common and rare class samples separately. When dealing with imbalanced data classification problems, although the BalanceCascade algorithm improves the performance of the decision tree, this improvement is also quite limited.

[0131] The performance of the three algorithms is compared using the recall rate R, precision rate P and AUC three evaluation criteria. The relevant information of the data set is shown in the first 5 columns of Table 1, and the false alarm rate f of the CBT algorithm is set as shown in the last column of Table 1. This setting makes the number of component classifiers obtained in the three algorithms about 40.

[0132] Table 1 Dataset

[0133] Dataset Number of categories Number of attributes Target class label Rare class: majority class False alarm rate abalone 29 8 7 391:2786 0.7 cmc 3 9 2 333:1140 0.7 haberman 3 3 2 81:225 0.75 ionosphere 2 34 good 126:225 0.85 lr 26 16 A 789:19211 0.4 pima 2 8 1 286:500 0.8 sat 6 36 4 626:5809 0.5 wdbc 2 30 M 212:357 0.85

[0134] The classification results of the three algorithms are converted into precision P, recall R and AUC values, as shown in Table 2:

[0135] Table 2 Recall R, precision P and AUC of three algorithms

[0136]

[0137]

[0138] As shown in Table 2 and Figure 3-5 It can be seen that the CBT algorithm provided in this application has the highest precision, recall and AUC values ​​on all data sets, and is significantly better than the Boost-SVM algorithm and the BalanceCascade algorithm in classification results. It even has a significant improvement in classification performance on specific data sets, which shows that the CBT algorithm has higher performance in overall classification effect and recognition rate of rare classes.

[0139] The reason may be that the initial sample weight setting method and update rules in the CBT algorithm provided in this application enable the classifier to weigh the selection of rare classes and majority classes more, and perform classification training according to the new weight rules to prompt the classification efficiency of the Cascade Model. This shows that the CBT algorithm uses Twin-SVM with better classification effect as the weak classifier of AdaBoost to train samples to improve the overall classification performance. In the classification process, more attention is paid to the selection of rare class samples, which effectively improves the classification efficiency of the Boost-TwinSVM algorithm for rare classes. On this basis, combined with the Cascade Model, the Boost-TwinSVM algorithm is used in each node of the Cascade Model to alternately classify the data set, giving full play to the advantages of the Boost-TwinSVM algorithm and the Cascade Model in processing rare classes, further improving the classification ability of the CBT algorithm for rare classes, significantly improving the overall classification performance of the data set, and having better performance and effect for the overall classification effect of unbalanced data and the recognition rate of rare classes. Make the CBT algorithm superior to the Boost-SVM algorithm and the BalanceCascade algorithm in classification results.

[0140] It should be noted that, in this application, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0141] In the description of the embodiments of the present application, unless otherwise specified, “ / ” means or, for example, A / B can mean A or B; the “and / or” in the text is merely a description of the association relationship of associated objects, indicating that three relationships may exist, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, “multiple” refers to two or more than two.

[0142] The above is only a specific implementation of the present application, so that those skilled in the art can understand or implement the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest range consistent with the principles and novel features applied for herein.

Claims

1. A method for imbalanced data classification based on a hybrid model, characterized in that: The following steps are involved: Obtain an unbalanced data set to be classified as the initial classification node of the cascade model, wherein the unbalanced data set includes a rare class sample set P of m samples and a majority class sample set N of n samples, and m≤n; Calling the twin support vector machine algorithm Twin-SVM as a weak classifier of the AdaBoost algorithm to form a hybrid model combination algorithm Boost-TwinSVM, and performing K inner layer iterations of each classification node in the cascade model; The i-th (i>1) classification node of the cascade model i Received from the previous node Node i-1 The sample set W passed in i , the hybrid model combination algorithm Boost-TwinSVM is the sample set W i Each sample in the i (1) , according to the initial weight W i (1) Select the corresponding samples to generate a pre-training set, train the pre-training set, and obtain the corresponding component classifier h (i,1) ; Based on the component classifier h (i,1) , calculate the component classifier h (i,1) The AUC value on the pre-selected training set is used to update the weight distribution of the sample based on the AUC value until all samples are trained, and the K inner layer iterations are completed to obtain a series of component classifiers h (i,1) ,…,h (i,k) ; All component classifiers are integrated into the node classifier H i , get the next sample set W i+1 And pass it to the next classification node Node i+1 , the above process is repeated to obtain a series of node classifiers H1, H2…, H I , until the classification node loop of the cascade model CascadeModel ends; A series of node classifiers H1, H2…, H I The final classifier H(X) is integrated, and the unbalanced data set to be classified is classified according to the final classifier H(X) to obtain a classification result.

2. The imbalanced data classification method based on the hybrid model according to claim 1, characterized in that: The hybrid model combination algorithm Boost-TwinSVM is the sample set W i Each sample in the i (1) Based on the hybrid model combination algorithm Boost-TwinSVM, set 3. The imbalanced data classification method based on a hybrid model as claimed in claim 1, characterized in that: The component classifier h (i,1) The calculation formula of the AUC value on the pre-selected training set is: Among them, m is the number of samples in the rare class sample set, n is the number of samples in the majority class sample set, q is the number of all known misclassified samples, and p is the number of misclassified samples in the majority class.

4. The imbalanced data classification method based on a hybrid model as claimed in claim 1, characterized in that: The weight distribution of the sample is updated based on the AUC value. The calculation formula is: Among them, a (i,k) is the AUC value of the kth iteration cycle in the i-th classification node, Z k+1 is the normal factor, x i is the feature vector of the i-th sample, y i is the label or target value of the i-th sample.

5. The imbalanced data classification method based on a hybrid model according to claim 1, characterized in that: The node classifier H is integrated with all component classifiers. i , get the next sample set W i+1 And pass it to the next classification node Node i+1 include: A series of component classifiers h (i,1) ,…,h (i,k) Ensembled as node classifier H i , The node classifier H i The calculation formula is: Among them, a (i,k) is the AUC value of the kth iteration cycle in the i-th classification node; Based on the node classifier, the sample set W i The majority class sample set is trained, and the majority class samples that have been correctly classified in the majority class sample set are excluded according to the preset false alarm rate to obtain the next sample set W i+1 , passed to the next classification node Node of the cascade model i+1 .

6. The imbalanced data classification method based on a hybrid model according to claim 1, characterized in that: The classification node cycle termination condition of the cascade model is: the number of samples in the majority class sample set is not greater than the number of samples in the rare class sample set or the preset false alarm rate is reached.

7. The imbalanced data classification method based on a hybrid model according to claim 1, characterized in that: The series of node classifiers H1, H2…, H I The final classifier H(X) is integrated. The calculation formula of the final classifier H(X) is:

8. A device for classifying imbalanced data based on a hybrid model, characterized in that: The imbalanced data classification device based on the hybrid model includes: The data set acquisition module is configured to: acquire an unbalanced data set to be classified as an initial classification node of the cascade model, wherein the unbalanced data set includes a rare class sample set P of m samples and a majority class sample set N of n samples, and m≤n; The first computing module is configured to: call the twin support vector machine algorithm Twin-SVM as a weak classifier of the AdaBoost algorithm to form a hybrid model combination algorithm Boost-TwinSVM, and perform K inner layer iterations of each classification node in the cascade model; The second computing module is configured as: the i-th (i>1) classification node Node of the cascade model i Received from the previous node Node i-1 The sample set W passed in i , the hybrid model combination algorithm Boost-TwinSVM is the sample set W i Each sample in the i (1) , according to the initial weight W i (1) Select the corresponding samples to generate a pre-training set, train the pre-training set, and obtain the corresponding component classifier h (i,1) ; The third computing module is configured to: (i,1) , calculate the component classifier h (i,1) The AUC value on the pre-selected training set is used to update the weight distribution of the sample based on the AUC value until all samples are trained, and the K inner layer iterations are completed to obtain a series of component classifiers h (i,1) ,…,h (i,k) ; The fourth computing module is configured to: integrate all component classifiers into a node classifier H i , get the next sample set W i+1 And pass it to the next classification node Node i+1 , the above process is repeated to obtain a series of node classifiers H1, H2…, H I , until the classification node cycle of the cascade model Cascade Model ends; The final classification module is configured as follows: a series of node classifiers H1, H2…, H I The final classifier H(X) is integrated, and the unbalanced data set to be classified is classified according to the final classifier H(X) to obtain a classification result.

9. A device for classifying imbalanced data based on a hybrid model, characterized in that: The imbalanced data classification device based on the hybrid model includes: a memory, a processor, and an imbalanced data classification program stored in the memory and executable on the processor, wherein the imbalanced data classification program is configured to implement the steps of the imbalanced data classification method based on the hybrid model as described in any one of claims 1 to 7.

10. A storage medium, characterized in that: The storage medium stores an imbalanced data classification program, which, when executed by a processor, implements the steps of the imbalanced data classification method based on a hybrid model as described in any one of claims 1 to 7.

Images