169 results about "Class imbalance" patented technology

The invention discloses a class-imbalance problem classification method based on expansion training data set; the method comprises the following steps: obtaining a true data set needed by a classification task; screening a few class samples from the true data set, and distinguishing samples that are close to and far away from the decision boundary; inputting said samples, running a productive confrontation network, thus obtaining artificial samples similar to the true data; adding certain amount of artificial samples into the true data set, thus obtaining a mixed data set; inputting the mixeddata set, and using a classifier to classify. The method combines a CycleGAN model with the boundary information of an original data set, thus effectively simulating distribution features of the truedata. The method samples small sample data so as to improve the classifier precision, and effectively preventing the class-imbalance problem from affecting the classification task.

The invention discloses an unbalanced text classification method and system combining SVM and semi-supervised clustering. The unbalanced text classification method comprises the steps: carrying out preprocessing on a to-be-processed text, and obtaining text data in a vector format, and enabling the text data to serve as a data set; using the training set to train the SVM classifier to obtain a classification model, and using the classification model to predict the test set to obtain the category and confidence of the test set; clustering the data set by using a semi-supervised clustering algorithm to obtain the category to which the test set belongs and the confidence coefficient of the test set; and fusing the category to which the test set obtained by the SVM classifier and the semi-supervised clustering algorithm belongs and the confidence coefficient of the test set to obtain final output. According to the unbalanced text classification method, different types of methods in the technical field of unbalanced text classification are combined; advantage complementation of the different methods is achieved; vectorization and normalization methods are used; and the defect that whenhigh-dimensional sparse text data are processed, a text classification result is inaccurate due to the fact that labeled texts are too few is overcome. The unbalanced text classification method effectively solves the problem of text class imbalance.

The present invention discloses a multiclass unbalanced genomics data iterative integrated feature selection method and system. Aiming at the characteristic of unbalanced data distribution of multi-labeled genomics data, the present invention provides the iterative feature selection method. On the basis of integrating classifiers in a one-to-many manner, undersampling or oversampling and feature selection are iteratively operated, so that samples of a data set gradually reach a balanced state along with gradual decrease of the number of features. By adopting a classifier obtained after integration in the process, classification identification capability on subclass samples can be obviously improved. A weak classifier based on sub balanced data training is integrated into a strong classifier by adopting an integrated learning technology, so that classification accuracy can be obviously improved.

The invention discloses a software defect prediction method based on two-stage wrapping-type feature selection, and belongs to the field of software quality assurance. The software defect prediction method comprises the following steps: (1) mining the version control system and the defect tracking system of a software project, extracting a program module from the version control system and the defect tracking system, and carrying out type marking and software measurement on the program module to generate a defect prediction data set D; (2) carrying out two-stage wrapping-type feature selection on the defect prediction data set so as to remove redundant features and irrelevant features in the data set D as many as possible, and finally, selecting an optimal feature subset FS' from an original feature set FS; and (3) on the basis of the optimal feature subset FS', preprocessing the data set D, forming a preprocessed data set D', and finally, constructing a defect prediction model in virtue of a decision tree which is a classification method. By use of the software defect prediction method, on one hand, the redundant features and the irrelevant features in the defect prediction data set can be effectively identified and removed, on the other hand, a class imbalance problem in the defect prediction data set can be effectively alleviated, and finally, the performance of the defect prediction model can be effectively improved.

The invention discloses a protein-DNA binding residue prediction method based on sampling and integrated learning. The method comprises the steps of (1) feature extraction and training sample set construction, (2) sampling and model training, (3) model integration, and (4) online prediction. The method is used for solving the shortcomings of low prediction precision caused by the problems of few feature types and class imbalance in protein-DNA binding residue prediction problems and has the advantages of high prediction precision and high generalization ability.

The invention discloses an MRI brain tumor automatic segmentation method based on a double-flow decoding convolutional neural network of edge feature optimization. According to the invention, two optimization strategies based on edges are mainly used to improve the performance of brain tumor segmentation. Firstly, on a network structure, an independent decoding network branch is designed to process edge stream information, and the edge stream information is fused into semantic stream information through feature fusion. Secondly, punishing unmatched pixels of the prediction segmentation mask and the label near the edge by using a regularization loss function to encourage the prediction segmentation mask to be aligned with the label value around the edge; in training, a new edge extraction algorithm is introduced to provide edge tags with higher quality. In addition, an adaptive balance class weight coefficient is added into a cross entropy loss function, so that the problem of serious class imbalance in back propagation of edge extraction is solved. Experiments show that the tumor segmentation precision is effectively improved.

The invention relates to a data equalization method based on a deep learning multi-weight loss function, and the method comprises the steps: firstly obtaining a target image data set in a training process employing a deep learning model, determining the class number C of data samples and the size Ni of each class of samples according to the target data set, determining hyper-parameters [alpha] and [gamma] and a weighting coefficient Ci of the importance of each class of samples, and determining a multi-weight loss function MWLfocal (z, y), carrying out continuous iterative training by using the neural network model, carrying out error calculation by using the multi-weight loss function in the training process, and continuously updating weight parameters of the model by using a back propagation algorithm until network convergence reaches an expected target, thereby finally completing training. By means of the loss function, the problems of sample number imbalance and classification difficulty imbalance of different data classes can be solved at the same time, the detection accuracy of key classes can be further improved, the method can be applied to a data set with the data imbalance problem, and therefore the influence of the class imbalance problem is effectively relieved.

The invention relates to a binary classification method for processing non-small cell lung cancer data with missing values and imbalance, and belongs to the technical field of data classification. Themethod comprises the following steps: preprocessing data, filling missing values with medians, eliminating abnormal values by using a Tukey's method, and normalizing the data by using deviation standardization; secondly, carrying out data balance by adopting an SMOTEENN comprehensive sampling method combining oversampling and undersampling; finally, the balanced data set is used for training a random forest classifier, the classification effect is tested on the test set, and therefore the non-small cell lung cancer survival prediction dichotomy method effectively aiming at the problems of missing values and class imbalance is achieved. Experiments performed on a non-small cell lung cancer data set prove the effectiveness and superiority of the method, the classification precision of the non-small cell lung cancer data with missing values and imbalance is improved, and more accurate medical decisions can be achieved.

The invention relates to the technical field of network traffic classification, and relates to an unbalanced-like network traffic classification method and device and computer equipment. The method comprises the steps of obtaining to-be-classified network traffic data, and extracting features of network traffic; deleting irrelevant features and redundant features by adopting a feature selection algorithm, and performing dimension reduction on the remaining features so as to select an optimal feature subset; and inputting the optimal feature subset into a weight-based multi-classifier, performing network traffic classification training in an incremental learning mode, optimizing classifier performance, and classifying the network traffic. Aiming at the problem of unbalanced distribution ofnetwork traffic samples, irrelevant features and redundant features are deleted, and the recognition rate of small categories is effectively improved on the premise of ensuring the overall classification accuracy; an incremental learning thought is introduced, the flexibility of model updating training is improved, and the model updating period is shortened; and by utilizing multiple classifiers based on weight, the influence caused by concept drift is reduced.

The invention relates to the technical field of pedestrian re-identification in computer vision, in particular to a pedestrian re-identification method based on distance centralization and projectionvector learning. The method includes the following steps: Step 1, dividing a pedestrian training set and test set; Step 2, extracting features of a pedestrian image, including color features and texture features; Step 3, calculating the feature distance of centralization; Step 4, building a pedestrian re-identification model based on iterative projection vector learning; Step 5, using a conjugategradient method to solve the model iteratively; and Step 6, calculating different pedestrian feature distances in the test set to perform pedestrian re-identification, the overfitting situation brought by class imbalance is effectively solved and the identification accuracy of pedestrian re-identification is improved. The pedestrian re-identification method based on distance centralization and projection vector learning can well improve the training speed and has a good suppression effect on the noise. The method has very good robustness to pedestrian posture, illumination variation and shielding.

The invention relates to an evaluation method for performance influence degree of classification models by class imbalance. The evaluation method comprises the following steps of (1) building a classification model base; (2) constructing a new data set; (3) forecasting the new data set by the classification models; (4) evaluating the performance of the classification models; and (5) evaluating an influence degree level. According to the evaluation method, firstly, a typical classification algorithm in machine learning is adopted to build the classification model base; secondly, a class imbalance data set is selected as a reference data set, a group of new data sets with imbalance ratio gradually increased is built on the basis, different classification models are selected to respectively classify and forecast the group of new data sets; and finally, a variable coefficient is adopted to evaluate the performance variation degree of the classification models and also carry out level division, thus, the influence degree of the class imbalance on the performance of different classification models is evaluated, and a guidance significance is played in research on the class imbalance process. With regards to different classification models, the evaluation method for performance influence degree of the classification models by class imbalance, provided by the invention, has high universality.

The invention discloses an advertisement click rate prediction framework and algorithm based on user behaviours. ID characteristics and other characteristics are co-converted in different levels intomeaningful numerical characteristics; due to the characteristics, the characteristic sparsity and redundancy can be reduced; the characteristic expressiveness can be improved; simultaneously, to further improve the characteristic expressiveness, characteristic selection and characteristic combination are carried out by utilization of a GBDT model in the invention; high-dimensional characteristicsare processed by utilization of an LR model; finally, to solve a class imbalance problem, a down-sampling algorithm based on a K_Means model is provided in the invention; in an experimental process, characteristic extraction on original characteristics is carried out at first; then, characteristic classification is carried out by adoption of heuristic thinking; characteristic combination is carried out by inputting perceptual characteristics into the GBDT model; finally, rational characteristics and combination characteristics are input into the LR model with a certain weight, so that advertisement click rate prediction is carried out; and an experimental result shows that the algorithm in the invention is improved both on RMSE and R2 indexes.

The present invention relates to the data mining technology, and especially relates to a method for training a class-imbalanced data classifier, a class-imbalanced data classifier and a method for classifying the class-imbalanced data. According to one embodiment of the method for training the class-imbalanced data classifier, data classified by the class-imbalanced data classifier has a plurality of properties. The method comprises the following steps that the properties are divided into a plurality of property groups, each property group corresponds to one sub-classifier, and each sub-classifier is suitable for classifying the data based on the corresponding property group, so as to obtain an ultimate classification result by the classification results of the sub-classifiers according to pre-set rules; training data samples are divided into multiple test sets; and for each property group, the corresponding sub-classifiers are trained by using different test sets.

The invention discloses an image enhancement method based on deep learning. The method comprises the following steps: collection and processing of an original image, augmentation processing of the original image, sharpening processing of a target image, construction of a training set, construction of a data set and removal of class imbalance in the data set, removal of redundancy in the training set by wavelet transform, and training of the data set and the training set by a convolutional neural network. According to the invention, redundant information in the original image can be effectivelyremoved through gray processing of the original image; the redundancy is removed through wavelet transform, so that the training precision and effectiveness of the original image serving as a training set in the later stage can be comprehensively and effectively improved; in addition, in the process, the image can be denoised in real time, the image with a good visual effect can be obtained, theclass imbalance phenomenon is removed by adopting an interpolation-based SMOTE method, the problem of sample class imbalance can be effectively solved, and the enhancement effect of the image is ensured to be stable.

The invention relates to a support vector machine integrated learning method based on AdaBoost. In order to overcome the defect that an existing support vector machine learning method is low in precision when processing a class imbalance classification problem, the invention provides a support vector machine integrated learning method based on AdaBoost, and a weighted support vector machine (W-SVM) is used. According to the method, sample distribution information can be deeply mined, prediction precision can be remarkably improved, and the method is an effective tool for solving the class imbalance problem.

The invention discloses a job shop real-time scheduling method based on PCA-XGBoost-IRF. The method comprises the steps of 1, constructing a standard data sample; 2, pre-processing the sample data, performing abnormal value processing, class imbalance processing and normalization processing on the sample data, and segmenting a data set to meet the input requirements for decision model construction; 3, carrying out feature engineering processing on a training set, wherein the feature engineering processing comprises feature extraction, feature importance calculation and feature selection; 4, carrying out decision model construction based on an improved random forest, including random forest model construction, improvement of an RF model to obtain an IRF model, and optimization of hyper-parameters of the IRF model based on grid search; 5, performing PCA-XGBoost-IRF decision model training based on the optimal parameters; and 6, realizing the real-time selection and decision-making of a dynamic job shop scheduling rule by using a decision-making model based on PCA-XGBoost-IRF. According to the present invention, the real-time scheduling method which is more reliable and higher in robustness and generalization is provided for the intelligent scheduling research based on data driving.

The invention discloses a product key part state classification method for class imbalance data. The method comprises the steps of: obtaining and preprocessing an auxiliary training set and a source training set; performing N times of undersampling processing on the majority class samples in the source training set to obtain N relatively balanced sub-data sets; training N SVM classifiers in parallel by using the N sub-data sets, and selecting by using a voting method to obtain a final prediction result; taking out a minority of auxiliary data in a final prediction result and adding the minority of auxiliary data to the source training set; constructing a deep learning classification model and supervising training by using the reconstructed source training set; and performing detection processing on to-be-predicted sensor data. According to the method, label data in the source training set and unlabeled data in the auxiliary data set are fully utilized, a weak supervised learning methodis utilized for processing, the imbalance proportion of class imbalance data can be reduced, and the prediction effect of the classification model is improved.

The invention provides a data resampling method based on clustering oversampling and an instance hardness threshold. The method comprises the following steps: firstly, performing clustering processingon a data set by utilizing a Kmeans method, and performing filtering processing and sampling weight distribution on clustering; then, adopting an SMOTE algorithm to carry out oversampling on the dataset to generate new data, so that the number of minority class samples in the data set is equal to that of majority class samples, and the data set becomes class balance; and finally, cleaning the data by adopting an instance hardness threshold algorithm to obtain a final balanced data set with fewer noisy points. According to the method, the class imbalance data set can be processed into the balance data set, and the prediction performance of the classifier for minority class samples is improved.

The invention relates to a few-sample learning classifier construction method based on unbalanced data, and belongs to the technical field of computer data classification. The method comprises the following steps: firstly, designing a twinning parallel full connection network for feature learning of input sample pairs according to primary learning and few sample learning characteristics of a twinning neural network; and then, processing the imbalance problem of the input sample pairs by using a cost sensitive optimizer, designing an expected error classification cost function according to different error classification costs, and integrating the expected error classification cost function into a network parameter optimization algorithm for adjusting class imbalance classification weights.According to the method, a better classification result can be obtained under unbalanced, high-dimensional and limited target data sets, and the classification performance is more stable.