Self-adaptive oversampling method combining local density and position information of sample
By dividing minority class samples into boundary and safe categories and performing adaptive oversampling based on local density and location information to generate new samples, the overfitting and noise problems in imbalanced data classification are solved, and the classification accuracy of minority class samples is improved.
Patent Information
- Application Number
- CN202511410224.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-09-29
- Publication Date
- 2026-01-09
AI Technical Summary
In existing imbalanced data classification, traditional methods are prone to overfitting and generating noisy samples, and the classification accuracy of minority class samples is low, making it impossible to effectively utilize important samples to generate new samples.
Minority class samples are divided into boundary minority class and safe minority class. The safe minority class is given a higher weight, and new samples are generated near the safe samples. Boundary samples are combined with the majority class to generate new samples. Adaptive oversampling is performed through local density and location information.
It improves the classification accuracy of minority class samples, reduces the impact of noise, focuses on the generation of important samples, solves the problem of generating new samples, and improves the overall classification performance.
Smart Images
Figure CN121301923A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of data balancing technology, and more specifically to an adaptive oversampling method that combines local sample density and location information. Background Technology
[0002] In machine learning and data mining, classifying imbalanced data has always been a challenge. Imbalanced data refers to a situation where the distribution of samples across different classes is extremely uneven, typically manifested as a significant increase in the number of samples from one class compared to others. This phenomenon is particularly common in scenarios such as fraud detection, disease diagnosis, text segmentation, image segmentation, security management, and anomaly detection. However, due to this imbalance, traditional classifiers often tend to favor the majority class, resulting in extremely low accuracy for the minority class. Although the overall accuracy of the classifier may be high, the classification results are meaningless because minority class samples are often crucial in these scenarios. For example, in disease diagnosis, the number of patients is usually less than the number of healthy individuals; misdiagnosing a patient as healthy could have serious consequences.
[0003] Currently, a common solution is to use data-level oversampling to balance the data. Oversampling achieves class balance by increasing the number of minority class samples. The sampling mechanism of random oversampling algorithms is relatively simple. This algorithm rebalances the data by randomly copying minority class samples, thereby improving classification accuracy. However, this method may lead to overfitting. The SMOTE algorithm significantly reduces the probability of overfitting. Unlike simply copying minority class samples, the SMOTE algorithm generates synthetic samples by randomly interpolating between minority class samples. However, the SMOTE algorithm is susceptible to noise and may interpolate between minority class samples and noise, thus generating synthetic samples and introducing more noise samples. To address this, an adaptive oversampling method combining local sample density and location information is proposed. Summary of the Invention
[0004] The technical problem this invention aims to solve is: how to determine where and how to synthesize samples. It provides an adaptive oversampling method that combines local sample density and location information for imbalanced data classification tasks. This method first divides minority class samples into boundary minority samples and safe minority class samples. Then, for safe minority class samples, higher weights are assigned to samples considered more important so that more new samples can be generated using these samples in subsequent sample generation processes. In the safe sample synthesis stage, neighboring samples that meet the criteria are selected, and new samples are generated near the safe samples. Finally, for boundary minority samples, a sampling method combining sampling with that of the majority class is used to synthesize them.
[0005] The present invention solves the above technical problems through the following technical solutions. The present invention includes the following steps:
[0006] S1: Input the original sample set D and divide it into a minority sample set Sand a majority sample set S maj ;
[0007] S2: Divide the minority samples into safe minority samples and boundary minority samples, and remove the noise;
[0008] S3: Assign weights to the safe minority samples and determine the number of synthetic samples generated by each safe minority sample according to the weights;
[0009] S4: Oversample the safe minority samples and the boundary minority samples respectively to obtain newly generated synthetic samples;
[0010] S5: Merge the newly generated synthetic samples with the denoised original samples and output a new sample set.
[0011] Furthermore, in the step S2, the specific processing process is as follows:
[0012] S21: For each minority sample x in the original sample set D i , find its k nearest neighbors, where, there are v majority-class nearest neighbors and k - v minority-class nearest neighbors;
[0013] S22: When the number of majority-class nearest neighbors of each sample x min in the minority sample set S i satisfies the condition: k / 2 ≤ ν < k, add the corresponding minority sample to the boundary minority sample set border_m; when the number of majority-class nearest neighbors of each sample x min in the minority sample set S i satisfies the condition: ν = k, regard the corresponding minority sample as a noise sample and add it to the noise sample set noise; when neither of the above conditions is satisfied, regard the corresponding minority sample as a safe minority sample and add it to the safe minority sample set safe_m; and let safe_m = safe_m ∪ noise for subsequent weight assignment;
[0014] S23: Filter the noise in the majority samples. For each sample x maj in the majority sample set S j , use the formula to find the nearest neighbor of this sample. If the nearest neighbor is a boundary minority sample, regard it as noise and delete it, where, m represents the nearest neighbor of this sample.
[0015] Furthermore, in step S3, the specific processing procedure is as follows:
[0016] S31: Input the safe minority class sample set safe_m, initialize the weight list weights to an empty list, and use it to store the final weight of each safe minority class sample;
[0017] S32: For each sample in the safe minority class sample set, calculate the number of majority class neighbors ν among its k nearest neighbors, that is, the number of samples belonging to the majority class among its k nearest neighbors, and determine the number of minority class neighbors as k-ν, that is, the number of samples belonging to the minority class among its k nearest neighbors;
[0018] S33: Select sample x from the safe minority sample set. i The few nearest neighbors of a class, i.e., k-ν, are used to construct a local space set R. i =x i ∪NN1,NN2,…,NN k-ν , where NN1, NN2, ..., NN k-ν x represents i k-ν nearest neighbors of the minority class;
[0019] S34: Based on x i The local space set R i Calculate the relative local density ρ(i) and the proximity of neighboring locations s(i);
[0020] S35: Calculate the sampling weight w(i) by combining the relative local density ρ(i) and the proximity of the neighbor positions s(i), normalize the sampling weight w(i), and add it to the weight list weights;
[0021] S36: Assign the number of synthetic samples to be generated for each safe minority class sample based on the sampling weight.
[0022] Furthermore, in step S34, the formula for calculating the relative local density ρ(i) is as follows:
[0023]
[0024] Where, N safe It is the total number of safe minority class samples, and d(i) is R. i The average distance between samples;
[0025] The formula for calculating the proximity s(i) of neighbor locations is as follows:
[0026]
[0027] Where kv is the current sample x i The number of minority class neighbors.
[0028] Furthermore, R i The formula for calculating the average distance d(i) between samples is as follows:
[0029]
[0030] Here, distance(x,y) represents the Euclidean distance between sample x and sample y.
[0031] Furthermore, in step S35, the formula for calculating the sampling weight w(i) is as follows:
[0032]
[0033] Furthermore, in step S35, if sample x in the safe minority class sample set safe_m... i If none of the k nearest neighbors are minority class neighbors, i.e., kv = 0, then it is determined to be a noise sample and assigned a weight weight(i) = 0.
[0034] Furthermore, in step S36, the specific processing procedure is as follows:
[0035] S361: Calculate the number of synthetic samples n to be generated for each safe minority class sample based on the sampling weights. i :
[0036]
[0037] Where, N maj N is the number of majority class samples. min This represents the number of minority class samples; round(·) indicates rounding the number in parentheses.
[0038] S362: Calculate the total number of synthetic samples G_safe required to generate safe minority class samples.
[0039] G_safe=n i ·N safe ;
[0040] S363: Assign the number of synthetic samples to be generated g_safe(i) to each safe minority class sample based on the sampling weights:
[0041] g_safe(i)=round(G_safe·weight(i)).
[0042] Furthermore, in step S4, the specific processing procedure is as follows:
[0043] S41: For each safe minority class sample, find its k nearest neighbors; if the nearest neighbor NN1 is a minority class sample, then NN1, NN2, ..., NN... k Add samples sequentially to the set `best_index` until a majority class neighbor is encountered, allowing for the inclusion of one majority class neighbor; if the nearest neighbor NN1 is a majority class sample, add NN1 and NN2 to the set `best_index`; perform iterative synthesis using the sample synthesis method in KNNOR; and add minority class samples x... i Starting from the first neighbor in the `best_index` set, generate a synthetic sample between the starting point and the first neighbor using random values of 0-1. Then, using this synthetic point as the starting point, generate a new synthetic sample between the second neighbor of the synthetic point and the original sample point using random values of 0-1. Repeat this process until a set number of synthetic samples are generated. The synthesis formula is as follows: Where α is a random number between 0 and 1. These are newly generated synthetic samples. It is to utilize Generate a synthetic sample; add the generated synthetic sample to the set D(safe);
[0044] S42: For each border minority class sample, use the SWIM method to generate a synthetic sample and add the generated synthetic sample to the set D(border).
[0045] Furthermore, in step S5, the dataset Data(D′) after removing minority class noise samples and majority class noise samples is merged with the safe minority class synthetic sample set D(safe) and the border minority class synthetic sample set D(border) to obtain a new sample set.
[0046] The present invention has the following advantages over the prior art:
[0047] 1. The present invention divides minority class samples into boundary minority class samples and safe minority class samples. Then, for the safe minority class samples, those samples that are determined to be more important are given higher weights so that more new samples can be generated using these samples when generating samples in the future.
[0048] 2. In the safe sample synthesis stage, neighboring samples that meet the conditions are selected, and new samples are generated near the safe samples; for boundary minority class samples, a sampling method that combines with the majority class is used to synthesize samples for them; this invention focuses more on important minority class samples, adaptively selects neighboring samples for them, and solves the problem of determining where and how to generate new samples. Attached Figure Description
[0049] Figure 1This is a schematic diagram illustrating the main steps of the adaptive oversampling method that combines local sample density and location information in an embodiment of the present invention.
[0050] Figure 2 This is a schematic diagram illustrating the implementation process of the adaptive oversampling method that combines local sample density and location information in an embodiment of the present invention.
[0051] Figure 3 This is a comparison chart of the average F-measure ranking of the present invention and nine oversampling methods under the KNN classifier in the embodiments of the present invention, where AOSP represents the method of the present invention;
[0052] Figure 4 This is a comparison chart of the average G-mean ranking of the nine oversampling methods in the KNN classifier in the embodiments of the present invention, where AOSP represents the method of the present invention.
[0053] Figure 5 This is a comparison chart of the average AUC ranking of the present invention with nine oversampling methods under the KNN classifier, where AOSP represents the method of the present invention. Detailed Implementation
[0054] The embodiments of the present invention are described in detail below. These embodiments are implemented based on the technical solution of the present invention, and provide detailed implementation methods and specific operation processes. However, the scope of protection of the present invention is not limited to the following embodiments.
[0055] like Figure 1 , 2 As shown, this embodiment provides a technical solution: an adaptive oversampling method that combines sample local density and location information, including the following steps:
[0056] Step 1: Divide the original dataset
[0057] Divide the original sample set D into a minority class sample set S. min and the majority class sample set S maj ;
[0058] Step 2: Minority class sample segmentation and noise removal
[0059] For each minority class sample x in the original sample set D i Find its k nearest neighbors, which include v majority neighbors and kv minority neighbors;
[0060] To classify minority class samples, if the minority class sample set S min Each sample x in iIf the majority class neighbor number of most samples satisfies the condition: k / 2 ≤ ν < k, then add the minority class sample to the border minority class sample set border_m, that is: border_m = border_m ∪ x i ; If for each sample x min in the minority class sample set S i the majority class neighbor number satisfies the condition: ν = k, then regard the minority class sample as a noise sample and add it to the noise sample set noise, that is noise = noise ∪ x i ; If the above conditions are not satisfied, then regard the minority class sample as a safe minority class sample and add it to the safe minority class sample set safe_m, that is safe_m = safe_m ∪ x i . Since it is necessary to ensure that the number of majority class samples and minority class samples after sample synthesis is the same, and according to the formula, the weight obtained by the noise sample in the subsequent weight assignment is 0, so the noise sample will not be used to generate synthetic samples. Therefore, let safe_m = safe_m ∪ noise for subsequent weight assignment.
[0061] Filter the noise in the majority class samples. For each sample x maj in the majority class sample set S j , use the formula to find their nearest neighbors. If the nearest neighbor is a border minority class sample, regard it as noise and delete it;
[0062] Step 3: Weight assignment and determining the number of synthetic samples generated according to the weights
[0063] Input the safe minority class sample set safe_m, that is, the set of minority class samples determined to be safe, and initialize the weight list weights as an empty list, which is used to store the final weights of each safe minority class sample;
[0064] For each x i ∈ safe_m, where x i represents the i-th safe minority class sample, calculate the number of majority class neighbors ν among its k nearest neighbors. ν represents the number of samples belonging to the majority class among the k nearest neighbors of x i , and determine the number of minority class neighbors as k - ν, that is, the number of samples belonging to the minority class among the k nearest neighbors;
[0065] Select the k - ν nearest neighbors of the sample x i from the safe minority class sample set safe_m to construct the local space set R i = x i ∪ NN1, NN2, …, NN k-ν k-ν x represents i k-ν nearest neighbors of the minority class;
[0066] According to the formula Calculate R i The average distance d(i) between samples is given by distance(x,y), where distance(x,y) represents the Euclidean distance between sample x and sample y. The denominator 2(k-v+1) is used to eliminate redundant calculations and the influence of distances between sets, because set R i There are k-v+1 samples. When calculating the distance between pairs of samples, there will be duplicates. Adding 1 is to avoid the denominator being 0.
[0067] According to the formula Calculate the relative local density ρ(i), N safe ρ(i) represents the total number of samples in safe_m. The numerator is the average distance of the local spaces of all safe minority class samples, which is used to measure the sparsity of the overall samples. The denominator d(i) is the average distance of the local space of sample xi. The larger ρ(i) is, the sparser the local space of the current sample is.
[0068] According to the formula Calculate the proximity of the neighboring locations s(i), where k-ν is the distance of the current sample x. i The number of minority class neighbors, N safe It is the total number of safe minority class samples, and s(i) reflects x i The relative number of minority neighbors; the larger the value, the more minority neighbors there are, and it may be in a minority sample clustering area.
[0069] According to the formula The sampling weight w(i) is calculated by combining the relative local density and the proximity of the neighbor positions. ρ(i) represents the relative local density, and the sparser the local density, the higher the weight of the sample. s(i) represents the proximity of the neighbor positions, and the sample with low local density and high neighbor ratio has a higher weight.
[0070] If sample x in safe_m i If a sample has no minority class neighbors in its k-nearest neighbors (kv = 0), it is considered a noise sample and assigned a weight (weight(i) = 0). This is because noise samples have no minority class neighbors and are unsuitable for synthesizing new samples; therefore, the weight is set to 0. Otherwise, according to the formula... Normalized weights are processed by normalizing the weights w(i) of all safe minority class samples so that the sum of all weights is 1, which facilitates subsequent sampling based on the weights.
[0071] Add weight(i) to the weight list weights;
[0072] Returns the normalized list of weights.
[0073] The sampling weights for the safe minority class samples are defined as weights;
[0074] Calculate the number of synthetic samples (n) needed to be generated for each safe minority class sample based on the sampling weight. i The formula is Where, N maj N is the number of majority class samples. min This represents the number of minority class samples; round(·) indicates rounding the number in parentheses.
[0075] The total number of synthetic samples G_safe required to generate safe minority class samples is calculated using the formula G_safe = n. i ·N safe ;
[0076] The number of synthetic samples to be generated, g_safe(i), is allocated to each safe minority class sample according to the sampling weight, using the formula g_safe(i) = round(G_safe·weight(i)), where G_safe represents the total number of synthetic samples and weight(i) represents the number of safe minority class samples x. i The weights;
[0077] Step 4: Minority class oversampling
[0078] For each safe minority class sample, find its k nearest neighbors; if the nearest neighbor NN1 is a minority class sample, then NN1, NN2, ..., NN... k Add them sequentially to the set best_index until a majority class neighbor is encountered, and allow one majority class neighbor to be included;
[0079] If the nearest neighbor NN1 is a majority class sample, add NN1 and NN2 to the set best_index; iteratively synthesize samples using the sample synthesis method from the paper KNNOR (An oversampling technique for imbalanced datasets); and combine the minority class samples x... i Starting from the first neighbor in the `best_index` set, generate a synthetic sample between the starting point and the first neighbor using random values of 0-1. Then, using this synthetic point as the starting point, generate a new synthetic sample between the synthetic point and the second neighbor of the original sample point using random values of 0-1. Repeat this process until a sufficient number of synthetic samples are generated. The synthesis formula is as follows: Where α is a random number between 0 and 1. These are newly generated synthetic samples. It is to utilize The generated synthetic sample;
[0080] Add the generated synthetic sample to set D(safe);
[0081] For each boundary minority class sample, use the SWIM method to generate synthetic samples; calculate the number of synthetic samples G_border that need to be generated for the boundary minority class samples, using the formula G_border = n. i ·N border ;
[0082] The SWIM method determines the location for generating synthetic samples based on the density of each minority class sample relative to the majority class sample. Its key components include density estimation and migration processes, and the model is built using radial basis functions (RBF) and Mahalanobis distance.
[0083] Add the generated boundary minority class synthetic samples to the set D(border);
[0084] Step 5: Sample Merging
[0085] The dataset Data(D′) after removing noise samples from the minority class and the majority class is merged with the set of safe minority class synthetic samples D(safe) and the set of border minority class synthetic samples D(border) to obtain a new dataset Data(D″).
[0086] Returns a new dataset Data(D″).
[0087] like Figures 3 to 5 As shown, the F-measure index of nine oversampling methods under the KNN classifier ( Figure 3 ), G-mean index ( Figure 4 AUC indicator Figure 5 The average ranking comparison results show that the average ranking of the AOSP method of this invention is the highest.
[0088] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.
Claims
1. An adaptive oversampling method combining local sample density and location information, characterized in that, Includes the following steps: S1: Given the original sample set D, divide it into minority class sample sets S. min and the majority class sample set S maj ; S2: Divide the minority samples into safe minority samples and boundary minority samples, and remove noise; S3: Assign weights to the safe minority class samples, and determine the number of synthetic samples generated for each safe minority class based on the weights; S4: Oversample the safe minority class samples and the boundary minority class samples respectively to obtain newly generated synthetic samples; S5: Merge the newly generated synthetic samples with the denoised original samples to output a new sample set.
2. The adaptive oversampling method combining local sample density and location information according to claim 1, characterized in that, In step S2, the specific processing procedure is as follows: S21: For each minority class sample x in the original sample set D i Find its k nearest neighbors, which include v majority neighbors and kv minority neighbors; S22: When for each sample x in the minority sample set S min the number of majority class neighbors satisfies the condition: k / 2 ≤ ν < k, add the corresponding minority sample to the border minority sample set border_m; when for each sample x in the minority sample set S i the number of majority class neighbors satisfies the condition: ν = k, regard the corresponding minority sample as a noise sample and add it to the noise sample set noise; when neither of the above conditions is satisfied, regard the corresponding minority sample as a safe minority sample and add it to the safe minority sample set safe_m; and let safe_m = safe_m ∪ noise for subsequent weight assignment; S23: Filter noise in the majority class samples, and process the majority class sample set S. maj Each sample x in j Using the formula Find the nearest neighbor of the sample. If the nearest neighbor is a minority class sample on the boundary, it is considered noise and deleted. Here, m represents the nearest neighbor of the sample.
3. The adaptive oversampling method combining local sample density and location information according to claim 2, characterized in that, In step S3, the specific processing procedure is as follows: S31: Input the safe minority class sample set safe_m, initialize the weight list weights to an empty list, and use it to store the final weight of each safe minority class sample; S32: For each sample in the safe minority class sample set, calculate the number of majority class neighbors ν among its k nearest neighbors, that is, the number of samples belonging to the majority class among its k nearest neighbors, and determine the number of minority class neighbors as k-ν, that is, the number of samples belonging to the minority class among its k nearest neighbors; S33: Select sample x from the safe minority sample set. i The few nearest neighbors of a class, i.e., k-ν, are used to construct a local space set R. i =x i ∪NN1,NN2,…,NN k-ν , where NN1, NN2, ..., NN k-ν x represents i k-ν nearest neighbors of the minority class; S34: Based on x i The local space set R i Calculate the relative local density ρ(i) and the proximity of neighboring locations s(i); S35: Calculate the sampling weight w(i) by combining the relative local density ρ(i) and the proximity of the neighbor positions s(i), normalize the sampling weight w(i), and add it to the weight list weights; S36: Assign the number of synthetic samples to be generated for each safe minority class sample based on the sampling weight.
4. The adaptive oversampling method combining local sample density and location information according to claim 3, characterized in that, In step S34, the formula for calculating the relative local density ρ(i) is as follows: Where, N safe It is the total number of safe minority class samples, and d(i) is R. i The average distance between samples; The formula for calculating the proximity s(i) of neighbor locations is as follows: Where k-ν is the current sample x i The number of minority class neighbors.
5. The adaptive oversampling method combining sample local density and location information according to claim 4, characterized in that, R i The formula for calculating the average distance d(i) between samples is as follows: Here, distance(x,y) represents the Euclidean distance between sample x and sample y.
6. The adaptive oversampling method combining sample local density and location information according to claim 5, characterized in that, In step S35, the formula for calculating the sampling weight w(i) is as follows:
7. The adaptive oversampling method combining local sample density and location information according to claim 6, characterized in that, In step S35, if sample x in the safe minority class sample set safe_m... i If none of the k nearest neighbors are minority class neighbors, i.e., kv = 0, then it is determined to be a noise sample and assigned a weight weight(i) = 0.
8. The adaptive oversampling method combining local sample density and location information according to claim 7, characterized in that, In step S36, the specific processing procedure is as follows: S361: Calculate the number of synthetic samples n to be generated for each safe minority class sample based on the sampling weights. i : Where, N maj N is the number of samples in the majority class. min This represents the number of minority class samples; round(·) indicates rounding the number in parentheses. S362: Calculate the total number of synthetic samples G_safe required to generate safe minority class samples. G_safe=n i ·N safe ; S363: Assign the number of synthetic samples to be generated g_safe(i) to each safe minority class sample based on the sampling weights: g_safe(i)=round(G_safe·weight(i)).
9. The adaptive oversampling method combining sample local density and location information according to claim 8, characterized in that, In step S4, the specific processing procedure is as follows: S41: For each safe minority class sample, find its k nearest neighbors; if the nearest neighbor NN1 is a minority class sample, then NN1, NN2, ..., NN... k Add samples sequentially to the set `best_index` until a majority class neighbor is encountered, allowing for the inclusion of one majority class neighbor; if the nearest neighbor NN1 is a majority class sample, add NN1 and NN2 to the set `best_index`; perform iterative synthesis using the sample synthesis method in KNNOR; and add minority class samples x... i Starting from the first neighbor in the `best_index` set, generate a synthetic sample between the starting point and the first neighbor using random values of 0-1. Then, using this synthetic point as the starting point, generate a new synthetic sample between the second neighbor of the synthetic point and the original sample point using random values of 0-1. Repeat this process until a set number of synthetic samples are generated. The synthesis formula is as follows: Where α is a random number between 0 and 1. These are newly generated synthetic samples. It is to utilize Generate a synthetic sample; add the generated synthetic sample to the set D(safe); S42: For each border minority class sample, use the SWIM method to generate a synthetic sample and add the generated synthetic sample to the set D(border).
10. The adaptive oversampling method combining sample local density and location information according to claim 9, characterized in that, In step S5, the dataset Data(D′) after removing minority class noise samples and majority class noise samples is merged with the safe minority class synthetic sample set D(safe) and the border minority class synthetic sample set D(border) to obtain a new sample set.