Sample equalization method, apparatus and device, and storage medium

Abstract

The invention discloses a sample equalization method and device, equipment and a storage medium. The method comprises the following steps: dividing an obtained initial sample set to obtain a majority class sample set and a minority class sample set; determining the total number of generated samples according to the number of samples contained in the majority class sample set and the number of samples contained in the minority class sample set; based on the spatial distribution condition of the minority class samples in the minority class sample set, determining the identification difficulty of each minority class sample; according to each identification difficulty, in combination with the total sample generation number, determining a sample generation sub-number corresponding to each minority class sample; and a linear interpolation method is adopted to generate a corresponding sample for each minority class sample to generate a number of new sample data so as to realize sample equalization. According to the method, noise is effectively reduced during sample oversampling, more new samples are generated at the fuzzy position of the sample classification boundary, the training effect of the classification model is improved, and the classification model pays more attention to learning at the classification boundary.

Description

technical field [0001] Embodiments of the present invention relate to the technical field of data processing, and in particular, to a sample equalization method, device, device, and storage medium. Background technique [0002] In machine learning classification problems, sample imbalance often affects the classification performance of classification models. In the classification problems in the fields of finance, medicine, intrusion detection, etc., there are very few abnormal data, and the classification model often cannot obtain a good ability to distinguish minority samples through the original data. [0003] Taking the financial field as an example, sample imbalance problems widely exist in data mining practice. For example, most of the obtained historical data on risk control and intrusion detection are normal values, and there are very few abnormal samples. However, these abnormal samples are relatively Normal samples contain more information, so correctly identifyin...

AI Technical Summary

Problems solved by technology

[0028] Although the ADASYN method considers the distribution of the minority class and the majority class, and generates more samples at the classification boundary, when the neighbors of a minority class sample are all majority classes, the ADASYN algorithm will consider the weight of the sample to be the highest. , in this processing method, the influence of noise points will also be amplified

Images