Oversampling method and device based on SMOTE algorithm and electronic equipment

Abstract

The invention provides an oversampling method and device based on an SMOTE algorithm and electronic equipment. The method comprises the following steps: acquiring a historical sample data set, and determining positive and negative samples and corresponding numbers thereof; determining majority class sample data and minority class sample data, and performing data vectorization processing; screeningtarget sample data from the minority class sample data set by using a departure point monitoring method; performing oversampling on the target sample data based on an SMOTE algorithm to generate a specific number of new sample data; and obtaining an amplified minority class sample data set according to the generated new sample data and the original minority class sample data. According to the method, while the sampling method is optimized, the problem of data imbalance is solved, the accuracy of model prediction is improved, and the deviation caused by data imbalance is effectively reduced.

Description

technical field [0001] The present invention relates to the field of computer information processing, in particular to an oversampling method, device and electronic equipment based on the SMOTE algorithm. Background technique [0002] Class imbalance is a typical problem in classification tasks, which is mainly manifested by a large gap in the number of samples between two classes. In reality, there are many unbalanced categories, such as the identification of financial fraud and insurance fraud, the identification of cancer in medicine, and so on. The main difficulty in classifying unbalanced data is that traditional machine learning methods are based on the balance of categories in the training set, and are less sensitive to the distribution of data deviations, resulting in biased prediction results for multi-class data. However, from the perspective of data mining, minority groups often carry more important and useful information, therefore, mining and predicting these f...

AI Technical Summary

Problems solved by technology

Among them, the SMOTE algorithm is used to calculate the distance between samples and synthesize new samples to amplify the data, but the sampling points are linear sampling, and the distribution of sampling is not wide enough.

Since the existing method only samples all samples indiscriminately based on the distance between samples, it does not take into account the data characteristics between samples of the same type, which will lead to blurred or even overlapping sample boundaries after sampling, resulting in reduced prediction accuracy and affecting analysis. result

Therefore, there is still much room for improvement for oversampling methods

Images