Customer loss prediction method based on oversampling classification

Abstract

The invention discloses a customer loss prediction method based on oversampling classification, and the method comprises the steps: obtaining a customer loss data set, carrying out the preprocessing, carrying out the data clustering, selecting and marking a cluster with a dominant minority of samples, calculating a sampling weight value, calculating the membership degree of the samples in the cluster with the sampling weight value larger than 0, and carrying out the space division, and selecting a sample according to a region division result, executing linear interpolation oversampling to generate a balanced data set, training a classifier, preferentially constructing a final customer loss prediction model, and outputting a customer loss prediction category and probability. According to the method, the customer loss data set is balanced through an oversampling method of clustering and membership region division, the safest samples are selected for synthesis, the risk of introducing noise and fuzzy boundaries is effectively reduced, data category distribution is balanced by generating high-quality minority-class samples, and the accuracy of data classification is improved. And therefore, the recognition performance of minority class samples can be improved, and the accuracy of customer loss prediction can be improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of customer churn prediction methods, and in particular relates to a customer churn prediction method based on oversampling classification. Background Art

[0002] With increasingly fierce market competition, more and more companies are facing the problem of customer churn. Particularly in the service industry, as companies' products and services become increasingly homogeneous and customer choices become increasingly diverse, customer churn has become a key factor affecting corporate profits and market share. Numerous studies have shown that the cost of losing a customer is often higher than the cost of acquiring a new one. Consequently, companies have gradually come to realize that customer retention is more important than attracting new customers. By effectively predicting customer churn, companies can implement targeted strategies, such as providing customized services or promotional campaigns, to improve customer satisfaction and thereby reduce churn. In this context, research on how to develop effective customer churn prediction models and identify customers at risk of churn has become a key task in corporate marketing and customer management. These predictive models not only help optimize resource allocation but also enable companies to develop more targeted customer retention strategies.

[0003] However, with continued research, scholars have gradually realized that customer churn is a typical binary classification problem with an imbalanced data distribution. Churned customers only account for a small proportion of customer datasets, and the ratio of positive and negative samples is extremely unbalanced. This unbalanced data distribution poses a challenge to training classification models. Traditional classification methods, when faced with data imbalance, tend to predict the majority class, resulting in poor prediction of churned customers. Such models are unable to complete the critical task of identifying customers who are likely to churn. Therefore, data imbalance leads to poor model recognition performance for churned customers (minority class samples). Addressing data class imbalance is one of the key issues in the customer churn prediction task and has important research value.

[0004] Over the past decade, the imbalanced problem in customer churn prediction has attracted extensive research and discussion. Many methods have been proposed to handle imbalanced data, which can be divided into two categories: algorithm-level methods and data-level methods. While algorithm-level methods are typically designed and implemented for specific problems and are highly dependent on classifiers, data-level methods sample datasets without relying on specific classifiers, and therefore have greater versatility. These techniques resample data by removing the majority class or adding instances to the minority class by repeating or generating new samples, and can be categorized as undersampling, oversampling, and mixed sampling methods. However, existing methods often suffer from problems such as overfitting, noise sensitivity, the introduction of additional parameters selected based on experience, and high time complexity. Summary of the Invention

[0005] The purpose of the present invention is to provide a customer churn prediction method based on oversampling classification, which solves the problem of poor accuracy of existing customer churn prediction methods.

[0006] The technical solution adopted by the present invention is: a customer churn prediction method based on oversampling classification, comprising the following steps:

[0007] Step 1: Obtain and preprocess the customer churn dataset;

[0008] Step 2: Use k-means clustering algorithm to cluster the data;

[0009] Step 3: Select and mark the cluster where the minority class samples dominate, and calculate the sampling weight according to the sparsity of the minority class samples in the cluster;

[0010] Step 4: Calculate the membership of the samples in the cluster whose sampling weight is greater than 0, and divide the space into safe area, overlapping area and noise area based on the membership;

[0011] Step 5: Select samples based on the region division results and perform linear interpolation oversampling to generate a balanced data set;

[0012] Step 6: Use the balanced data set to train the classifier, select the optimal classifier to build the final customer churn prediction model, and output the customer churn prediction category and probability.

[0013] The present invention is also characterized in that:

[0014] When determining the number of clusters k for the k-means clustering algorithm in step 2, first use the elbow method to determine the range of k, and then use the silhouette coefficient method to select the optimal k value within the determined range.

[0015] Step 3 specifically includes the following steps:

[0016] Step 3.1: Traverse the cluster labels and select and mark the clusters where the minority class samples dominate;

[0017] Step 3.2: Calculate the sampling weights based on the sparsity of minority class samples within the cluster.

[0018] Step 3.1 specifically includes the following steps:

[0019] Step 3.1.1. Calculate the imbalance rate IR within each cluster:

[0020]

[0021] Among them, majority count(c) represents the number of majority class samples, and minority count(c) represents the number of minority class samples;

[0022] Step 3.1.2: Compare the imbalance rate of each cluster with the imbalance ratio threshold, and mark the clusters whose imbalance rate IR is less than the threshold and the number of minority class samples is greater than 1.

[0023] Step 3.2 specifically includes the following steps:

[0024] Step 3.2.1. Calculate the Euclidean distance matrix of the minority class samples for each labeled cluster. Then, sum all the off-diagonal elements of the Euclidean distance matrix and divide by the number of off-diagonal elements to calculate the average distance of the minority class samples within each labeled cluster.

[0025] Step 3.2.2: Divide the number of minority class samples in each labeled cluster by the average distance raised to the power of d, where d is the density factor, to calculate the density of minority class samples.

[0026] Step 3.2.3: Take the inverse of the density of the minority class samples to get the sparsity, and then calculate the sampling weight by the ratio of the sparsity of each labeled cluster to the total sparsity of all labeled clusters.

[0027] Step 4 specifically includes the following steps:

[0028] Step 4.1: Calculate the membership matrix for the samples in the labeled cluster and calculate the average membership of each category;

[0029] Step 4.2: Use the membership-based partitioning strategy to divide the data space into security areas D safe , overlapping area D ovSafe , noise area D nosiy .

[0030] Step 4.1 specifically includes the following steps:

[0031] Step 4.1.1. Calculate the center of category j as c j :

[0032]

[0033] Among them, n j is the number of samples of category j, x i is the eigenvalue of sample i of category j;

[0034] Step 4.1.2: Calculate the center c of sample i and category j j The distance d ij :

[0035] d ij =|x i -c j |

[0036] Step 4.1.3: Calculate the membership μ of sample i to category jij :

[0037]

[0038] Among them, k is the category other than category j, C is the number of categories, d ik is the distance between sample i and the center of category k, m is any real number greater than 1, corresponding to the degree of fuzziness of the solution; the membership value is between 0 and 1, and the sum of the membership degrees of a sample is 1;

[0039] Step 4.1.4: Calculate the average membership of samples in category k to category j

[0040]

[0041] For each class j, the average membership is:

[0042] Step 4.2 specifically includes the following steps:

[0043] Step 4.2.1: If the sample x i The membership degree μ of its true category j ij Greater than the average membership of category j When , we think that the sample x i In a safe area:

[0044]

[0045] Step 4.2.2: If there is a sample x i Membership μ to other categories k ik Greater than the average membership of other categories k When , we think that the sample x i In a noisy area:

[0046]

[0047] Step 4.2.3, if the sample x i The membership degree μ of its true category j ij Less than the average membership of category j And the sample x i Membership μ to other categories k ik are all smaller than the average membership of other categories k When , we think that the sample x i In the overlapping area, it is represented as the area of the data space where the safe samples and noise samples are removed:

[0048] D ovSafe =DD safe -Dnoisy .

[0049] Step 5 specifically includes the following steps:

[0050] Step 5.1. Calculate the safe area D safe For each sample x i The target number of synthetic samples is: Among them, n represents the number of synthesized samples required for the minority class j in the current execution cluster K, n safe Represents the number of safe samples of minority class j in the current execution cluster K;

[0051] Step 5.2: Safe Area D safe Each sample x in i is the reference point X in linear interpolation oversampling base , in safe area D safe Overlapping area D ovSafe Get sample x from the joint sample space of i The first n nearest neighbor samples of , and randomly select a nearest neighbor sample as the neighbor X nei0h , the synthesized sample X new Save in In the collection:

[0052] X new =X base +λ(X base +X neigh )

[0053] λ is the interpolation coefficient;

[0054] Step 5.3: After processing all clusters with sampling weights greater than 0, we can obtain the synthetic sample sets of each cluster. Again Sum up all synthetic samples S of minority category j in the original data space j , for the two-category dataset, S j Merge it with the original dataset to get a balanced dataset.

[0055] Step 6 specifically includes the following steps:

[0056] Step 6.1: For the balanced dataset obtained in step 5, train and evaluate the classification performance of the model on multiple classifiers using accuracy, F1 score, AUC value, and precision as evaluation indicators. Based on the evaluation results, use the grid search method to adjust the classifier hyperparameters to obtain the optimal parameter combination.

[0057] Step 6.2: Select the best classifier and its parameter combination to train the balanced data set, generate the final customer churn prediction model, and output the customer churn prediction results, including the predicted category of each sample, i.e., churn or not churn, and the probability value corresponding to the predicted category.

[0058] The beneficial effects of the present invention are as follows: the customer churn prediction method based on oversampling classification of the present invention takes into account the significant impact of noise and class overlap on the quality of sample synthesis, and balances the customer churn data set through an oversampling method of clustering and membership region division to select the safest samples for synthesis, effectively reducing the risk of introducing noise and fuzzy boundaries, and balancing the data category distribution by generating high-quality minority class samples, thereby improving the recognition performance of minority class samples and improving the accuracy of customer churn prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 It is a structural diagram of the customer churn prediction method based on oversampling classification of the present invention;

[0060] Figure 2 It is a schematic diagram of the process of generating a balanced data set in the customer churn prediction method based on oversampling classification of the present invention;

[0061] Figure 3 Schematic diagram of the process of generating a balanced data set in the customer churn prediction method based on oversampling classification of the present invention;

[0062] Figure 4 It is the ROC curve diagram of multiple classifiers in the experiment of the "Credit Card Customers Churn Prediction" data set of the present invention;

[0063] Figure 5 This is a graph showing the results of a comparative experiment between the present invention and the existing prediction method on the "Bank Marketing Dataset" dataset.

[0064] Figure 6 This is a graph showing the results of a comparative experiment between the present invention and the existing prediction method on the "Telco Customer Churn" dataset.

[0065] Figure 7 This is a graph of the columnar evaluation index results of a comparative experiment between the present invention and the existing prediction method on the "Credit Card Customers Churn Prediction" dataset. DETAILED DESCRIPTION

[0066] The present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0067] Example 1

[0068] The present invention provides a customer churn prediction method based on oversampling classification. First, K-means clustering is performed on the churn dataset, and the clusters are filtered. Then, membership-based region partitioning is used on the filtered clusters to divide the samples into a safe region, an overlapping region, and a noise region. Then, oversampling is performed based on the results of the region partitioning, and new samples are synthesized to balance the dataset. Finally, the balanced dataset is used to train and classify the model to achieve customer churn prediction. This method can significantly reduce the negative impact of class overlap and noise in the customer churn dataset on the quality of the synthesized samples, synthesize high-quality minority class samples, and rebalance the data to improve the recognition performance of minority class samples, thereby improving the accuracy of customer churn prediction.

[0069] Example 2

[0070] The present invention provides a customer churn prediction method based on oversampling classification, comprising the following steps:

[0071] Step 1: Obtain and preprocess the customer churn dataset.

[0072] We downloaded various customer churn datasets from public databases such as UCI and Kaggle. Each database provides detailed information about the dataset name, task type, data attributes, value distribution, number of features, number of data instances, and whether there are missing values. We then performed data preprocessing, including missing value handling, data conversion, and label encoding. Data conversion converts non-numeric data to numeric data. For classification problems, we used one-hot encoding. Label encoding converts the label category names to integer encodings. We then performed data normalization and other steps. Finally, we obtained the preprocessed unbalanced customer churn dataset.

[0073] Step 2: Obtain clusters after clustering.

[0074] Use the k-means clustering algorithm to cluster the data. The most noteworthy parameter of k-means is k, the number of clusters. Specifically, the appropriate number of clusters k is determined using the elbow method and the silhouette coefficient method. The elbow method calculates the sum of squared errors (SSE) under different k values and plots the relationship between k and SSE to find the "inflection point" as the optimal k value. The silhouette coefficient method evaluates the compactness and separation of the clustering results by calculating the silhouette coefficient of each sample. The value range of the silhouette coefficient is [-1,1]. The closer the value is to 1, the better the clustering effect. The two methods are combined by first using the elbow method to determine the approximate range of k. Then, the silhouette coefficient method is used to select the optimal k value within this range. Use k-means to cluster the data space with the optimal k value.

[0075] Step 3: Obtain the filtered clusters and cluster sampling weights.

[0076] After clustering, the filtering step selects and marks clusters dominated by the minority class and assigns sampling weights to the filtered clusters to determine the number of minority class samples to be generated in each cluster.

[0077] Step 4: For clusters with sampling weights greater than 0, use the region partitioning method based on membership to divide the space into safe areas, overlapping areas, and noise areas.

[0078] Step 5: Obtain a balanced dataset through oversampling. Perform linear interpolation oversampling based on the results of region division.

[0079] Step 6. Perform customer churn prediction on the balanced dataset. Specifically: Model training and evaluation: For the balanced dataset obtained in step 5, use accuracy, F1 score, AUC value, and precision as evaluation indicators to train and evaluate the model classification effect on multiple classifiers. Based on the evaluation results, use the grid search method to adjust the hyperparameters of the classifier to obtain the optimal parameter combination. Based on the evaluation results, select the classifier with the best result, use the optimal classifier and its parameter combination to train the balanced dataset to generate the final customer churn prediction model. Output the customer churn prediction results, including the predicted category (churn or not) of each sample and its corresponding probability value.

[0080] Example 3

[0081] The present invention provides a customer churn prediction method based on oversampling classification. Based on Example 2, step 3 preferably includes the following steps:

[0082] Step 3.1. Filter based on the ratio of minority and majority instances within each cluster. By default, any cluster consisting of at least 50% minority samples is selected for oversampling. This behavior can be adjusted by adjusting the imbalance ratio threshold, which defaults to 1. The imbalance ratio of a cluster is given by:

[0083]

[0084] Where IR represents the imbalance ratio, majority count(c) represents the number of majority class samples, and minority count(c) represents the number of minority class samples. Adding 1 to the numerator and denominator smooths the result, reducing the impact of extreme values and making the IR more stable and reasonable. Increasing the imbalance ratio threshold makes cluster selection more selective and requires a larger proportion of minority instances to be selected. On the other hand, lowering the threshold relaxes the selection criteria, allowing clusters with a higher majority ratio to be selected.

[0085] Step 3.2: After filtering, the labeled clusters are assigned a sampling weight between 0 and 1. The sampling weight is determined by comparing the density of a single cluster with the average density of all labeled clusters. When measuring cluster density, only the distance between a few instances is considered. The specific process is:

[0086] Step 3.2.1. Assume that K clusters are formed after clustering. For each labeled cluster Ki, only consider the minority class samples and calculate the Euclidean distance matrix. Calculate the average minority class distance Dis within each labeled cluster by adding all the off-diagonal elements of the distance matrix and then dividing by the number of off-diagonal elements. mi .

[0087] Step 3.2.2: To obtain the density of minority samples in K clusters, density(K i ), the number of minority class samples n of each labeled cluster mi Divide by its minority class average distance Dis mi d to the power of d, where d is a defined density factor that can be taken depending on the number of features.

[0088] Step 3.2.3, take density density (K i The reciprocal of ) is the sparsity of the minority class samples (K i ). The sampling weights sampling_weights of each labeled cluster are defined as the sparsity factor of the labeled cluster divided by the sum of the sparsity factors of all labeled clusters.

[0089] Step 3.2.4: After the above process, obtain the sampling weights (sampling_weights) for all labeled clusters, which range from [0 to 1]. Higher sampling weights correspond to lower-density minority samples, resulting in more generated samples. A weight of 0 means that this cluster is not oversampled for the current minority class. The number of samples generated for each labeled cluster is the sampling weight multiplied by the total number of synthetic samples required for that minority class.

[0090] Example 4

[0091] The present invention provides a customer churn prediction method based on oversampling classification. Based on Example 2, step 4 is preferably the following steps:

[0092] Step 4.1: The probability that a sample belongs to a class is expressed by membership. The higher the membership, the closer the sample is to the target class and the farther it is from other classes. Calculate the membership matrix for the samples in the labeled cluster and calculate the average membership of each class. The specific process is:

[0093] Step 4.1.1. Calculate the center of each category. The calculation formula is:

[0094]

[0095] where c j is the center of category j, n j is the number of samples of category j, x i is the eigenvalue of sample i of category j.

[0096] Step 4.1.2: Calculate the distance between each sample and the center of each category. The calculation formula is:

[0097] d ij =|x i -c j |

[0098] where d ij Represents the center c of sample i and category j j distance.

[0099] Step 4.1.3: Calculate the sample membership matrix. The calculation formula is:

[0100]

[0101] μ ij represents the membership of sample i to class j. C is the number of classes. k represents the other classes except class j. d ik is the distance between sample i and category k center. ijrepresents the distance between sample i and the center of cluster j. m is an arbitrary real number greater than 1, corresponding to the degree of ambiguity of the solution. Larger values of m result in blurred clusters, with all samples typically belonging to all clusters. In practical applications, m is typically set to 2. The membership degree ranges from 0 to 1, with the sum of the membership degrees of a sample being 1. A higher membership degree indicates that the sample is closer to the target class and farther from other classes.

[0102] Step 4.1.4: Calculate the average membership of each category. According to the formula:

[0103]

[0104] is the average membership of samples in category k to category j. k is the number of samples of category k.

[0105] For each category j, the average membership of the category is:

[0106] Step 4.2: Based on the calculation results obtained above, use the membership-based partitioning strategy to divide the data space into security areas D safe , overlapping area D ovSafe , noise area D nosiy The specific process is as follows:

[0107] Step 4.2.1. Assume x i For samples of category j, when sample x i The membership degree μ of its true class j ij Greater than the average membership of class j When x i The possibility of belonging to this category is high, so it is considered to be in a safe area.

[0108] Step 4.2.2: Assume x i The sample of category j is k, and the k is other categories except category j. When there is a sample x i Membership μ to other classes k ik Greater than the k-mean membership of other classes When the sample x i It is more likely to belong to other classes, so it is considered to be a noise sample.

[0109] Step 4.2.3: If the sample does not meet the conditions of a safe sample, that is, when the sample x i The membership degree μ of its true class j ij Less than the average membership of class j And x i Membership μ to other classes k ikare all smaller than the k-mean membership of other classes When x i At the junction of minority class samples and majority class samples, it is considered that x i is an overlapping sample. In this case, the overlapping area can also be expressed as the area in the data space where the safe samples and noise samples are removed.

[0110] Example 5

[0111] The present invention provides a customer churn prediction method based on oversampling classification. Based on Example 2, step 5 preferably includes the following steps:

[0112] Step 5.1: Assume that the current minority class label is j and the currently executed cluster is K. Define is a set of synthetic samples. Calculate D safe The target number of synthetic samples for each sample in : Where n represents the number of synthesized elements required for the minority class in the cluster, n safe The number of safe samples representing the safe area of the minority class of this cluster.

[0113] Step 5.2: In order to reduce the noise and ensure the diversity of generated samples, D safe Each sample x in i , do interpolation oversampling. One of the samples in the interpolation oversampling is the safe area D safe Sample X in base , the other sample is in the safe area D safe Overlapping area D ovSafe In the joint sample space of base One of the k nearest neighbor samples of . Save the synthesized sample in In collection.

[0114] Step 5.3: After processing all clusters with sampling weights greater than 0, the synthetic sample sets of each cluster are obtained. The sum is S j , represents all synthetic samples of minority class j in the original data space. For the two-category dataset, S j Merging the dataset with the original one yields a balanced dataset. For multi-class datasets, merging the sum of the synthetic samples from each class with the original dataset yields a balanced dataset. Customer churn prediction datasets are typically two-class datasets.

[0115] Example 6

[0116] The present invention provides a customer churn prediction method based on oversampling classification, such as Figures 1 to 3 As shown, the following steps are included:

[0117] Step 1: Obtain and preprocess the customer churn dataset. The specific process is as follows:

[0118] Step 1.1. Obtain a customer churn dataset. The UCI database, a machine learning dataset developed by the University of California, Irvine, contains public datasets from various fields. A dataset related to customer churn is the "Bank Marketing Dataset." This dataset contains data on bank marketing campaigns. The goal is to predict whether a customer will subscribe to a term deposit. The download link is: https: / / archive.ics.uci.edu / ml / datasets / Bank+Marketing. Kaggle is a data science competition platform that provides a large number of high-quality public datasets. Datasets related to customer churn include the "Telco Customer Churn" and "Credit Card Customer Churn Prediction" datasets. The "Telco Customer Churn" dataset is a telecommunications customer churn dataset with 7,043 instances and 21 features. The goal is to predict customer churn. The download link is: https: / / www.kaggle.com / blastchar / telco-customer-churn. The Credit Card Customers Churn Prediction dataset contains basic customer information and transaction records. The goal is to predict whether a customer will churn. You can download it from: https: / / www.kaggle.com / sakshigoyal7 / credit-card-customers.

[0119] Step 1.2: To standardize the data format and ensure data integrity, preprocess the customer churn dataset. First, perform missing value processing and fill in missing values. For numeric data, use the mean, and for categorical data, use the mode. Next, perform data encoding. LabelEncoder converts the label category names into integer encodings. Furthermore, one-hot encoding is used to convert non-numeric data into numeric data. Data normalization and other steps are then performed to obtain the preprocessed unbalanced dataset.

[0120] Step 2: Use the k-means clustering algorithm to cluster the data. The most noteworthy parameter of k-means is k, the number of clusters. The specific process is:

[0121] Step 2.1: Determine the appropriate number of clusters, k, using the elbow method and the silhouette coefficient method. The elbow method calculates the sum of squared errors (SSE) for different k values and plots the relationship between k and SSE, finding the "inflection point" as the optimal k value. The silhouette coefficient method calculates the silhouette coefficient for each sample to assess the compactness and separation of the clustering results. The silhouette coefficient ranges from [-1 to 1]; values closer to 1 indicate better clustering results. Combining these two methods involves first using the elbow method to determine the approximate range of k. Then, using the silhouette coefficient method, select the optimal k value within this range.

[0122] Step 2.2: Use k-means to cluster the data space using the optimal k value obtained in step 2.1.

[0123] Step 3: Obtain the filtered clusters and their sampling weights. After clustering, the filtering step selects and marks the clusters where the minority class samples dominate, and assigns sampling weights to the filtered clusters to determine the number of minority class samples to be generated in each cluster. The specific process is as follows:

[0124] Step 3.1: Traverse the cluster labels and select the cluster where the minority class samples dominate. The specific process is:

[0125] Step 3.1.1. Calculate the imbalance rate within each cluster using the following formula:

[0126]

[0127] Where IR represents the imbalance rate, majority count (c) represents the number of majority class samples, and minority count (c) represents the number of minority class samples. Adding 1 to the numerator and denominator smooths the result, reducing the impact of extreme values and making the IR more stable and reasonable.

[0128] In step 3.1.2, the imbalance ratio of each cluster is compared with the imbalance ratio threshold. When the imbalance ratio is less than the threshold (there are more minority samples in the cluster) and the number of minority samples is greater than 1, the sampling condition is met. The imbalance ratio threshold defaults to 1, that is, any cluster consisting of at least 50% minority samples is selected for oversampling.

[0129] Step 3.2: After filtering, the labeled clusters are assigned a sampling weight between 0 and 1. The sampling weight is determined by comparing the density of a single cluster with the average density of all labeled clusters. When measuring cluster density, only the distance between a few instances is considered. The specific process is:

[0130] Step 3.2.1. Assuming that K clusters are formed after clustering, for each labeled cluster Ki, consider only the minority class samples and calculate the Euclidean distance matrix. Calculate the average minority class distance within each labeled cluster by adding all off-diagonal elements of the distance matrix and then dividing by the number of off-diagonal elements.

[0131] Step 3.2.2. To obtain a measure of the density of minority class samples in the K clusters, divide the number of minority class samples in each labeled cluster by the power of d of its average minority class distance, where d is a defined density factor that can be taken depending on the number of features.

[0132] Step 3.2.3: Take the inverse of the density as the sparsity of the minority class samples. The sampling weight of each labeled cluster is defined as the sparsity factor of the labeled cluster divided by the sum of the sparsity factors of all labeled clusters.

[0133] Step 3.2.4: After the above process, obtain sampling weights for all labeled clusters, ranging from [0 to 1]. Higher sampling weights correspond to lower-density minority samples, resulting in more generated samples. A weight of 0 means that this cluster is not oversampled for the current minority class. The number of samples generated for each labeled cluster is the cluster's sampling weight multiplied by the total number of synthetic samples required for that minority class.

[0134] Step 4: For clusters with sampling weights greater than 0, use the region partitioning method based on membership to divide the space into safe areas, overlapping areas, and noise areas. The specific process is as follows:

[0135] Step 4.1: The probability that a sample belongs to a class is expressed by membership. The higher the membership, the closer the sample is to the target class and the farther it is from other classes. Calculate the membership matrix for the samples in the labeled cluster and calculate the average membership of each class. The specific process is:

[0136] Step 4.1.1. Calculate the center of each category. The calculation formula is:

[0137]

[0138] where c j is the center of category j, n j is the number of samples of category j, x i is the eigenvalue of sample i of category j.

[0139] Step 4.1.2: Calculate the distance between each sample and the center of each category. The calculation formula is:

[0140] d ij =|x i -c j |

[0141] where dij Represents the center c of sample i and category j j distance.

[0142] Step 4.1.3: Calculate the sample membership matrix. The calculation formula is:

[0143]

[0144] μ ij represents the membership of sample i to class j. C is the number of classes. k represents the other classes except class j. d ik is the distance between sample i and category k center. ij represents the distance between sample i and the center of cluster j. m is an arbitrary real number greater than 1, corresponding to the degree of ambiguity of the solution. Larger values of m result in blurred clusters, with all samples typically belonging to all clusters. In practical applications, m is typically set to 2. The membership degree ranges from 0 to 1, with the sum of the membership degrees of a sample being 1. A higher membership degree indicates that the sample is closer to the target class and farther from other classes.

[0145] Step 4.1.4: Calculate the average membership of each category. According to the formula:

[0146]

[0147] is the average membership of samples in category k to category j. k is the number of samples of category k. For each category j, the average membership of the category is:

[0148] Step 4.2: Based on the calculation results obtained above, use the membership-based partitioning strategy to divide the data space into security areas D safe , overlapping area D ovSafe , noise area D nosiy The specific process is as follows:

[0149] Step 4.2.1. Assume x i For samples of category j, when sample x i The membership degree μ of its true class j ij Greater than the average membership of class j When x i The probability of belonging to this category is high, so it is considered to be in the safe area:

[0150]

[0151] Step 4.2.2: Assume x i The sample of category j is k, and the k is other categories except category j. When there is a sample xi Membership μ to other classes k ik Greater than the k-mean membership of other classes When the sample x i It is more likely to belong to other classes, so it is considered to be a noise sample:

[0152]

[0153] Step 4.2.3: If the sample does not meet the conditions of a safe sample, that is, when the sample x i The membership degree μ of its true class j ij Less than the average membership of class j And x i Membership μ to other classes k ik are all smaller than the k-mean membership of other classes When x i At the junction of minority class samples and majority class samples, it is considered that x i is an overlapping sample. In this case, the overlapping area can also be expressed as the area of the data space where the safe samples and noise samples are removed:

[0154] D ov S a f e =DD safe -D noisy

[0155] Step 5: Obtain a balanced data set through oversampling. Based on the results of the region division, perform linear interpolation oversampling. The specific process is as follows:

[0156] Step 5.1: Assume that the current minority class label is j and the currently executed cluster is K. Define is a set of synthetic samples. Calculate D safe The target number of synthetic samples for each sample in : Where n represents the number of synthesized elements required for the minority class in the cluster, n safe The number of safe samples representing the safe area of the minority class of this cluster.

[0157] Step 5.2: In order to reduce the noise and ensure the diversity of generated samples, D safe Each sample x in i , do: with x i is the reference point X in linear interpolation oversampling base , current security sample x i The first n nearest neighbor samples with the sum of the safe area and the overlapping area as the domain are randomly selected as their neighbor X. neigh . The synthesized sample X newSave in In collection.

[0158] X new =X base +λ(X base +X neigh )

[0159] The interpolation coefficient λ is a random number between 0 and 1.

[0160] Step 5.3: After processing all clusters with sampling weights greater than 0, the synthetic sample sets of each cluster are obtained. The sum is S j , represents all synthetic samples of minority class j in the original data space. For the two-category dataset, S j Merging the dataset with the original one yields a balanced dataset. For multi-class datasets, merging the sum of the synthetic samples from each class with the original dataset yields a balanced dataset. Customer churn prediction datasets are typically two-class datasets.

[0161] Step 6: Perform customer churn prediction on the balanced data set. The specific process is as follows:

[0162] Step 6.1: Model Training and Evaluation. For the balanced dataset obtained in Step 5, train and evaluate the model classification performance on multiple classifiers using accuracy, F1 score, AUC, and precision as evaluation metrics. Based on the evaluation results, use a grid search method to adjust the classifier hyperparameters to obtain the optimal parameter combination.

[0163] Step 6.2: Based on the evaluation results of step 6.1, select the classifier with the best result. Figure 4 The figure shows the area under the receiver operating characteristic (ROC) curve (AUC) for several different classifiers on the "Credit Card Customers Churn Prediction" dataset. XGBoost performs best. XGBoost and its optimal parameter combination are used to train the balanced dataset to generate the final customer churn prediction model. The customer churn prediction results are output, including the predicted category (churn or not) and the corresponding probability for each example. Figure 5 、 Figure 6 、 Figure 7The paper presents a performance comparison of the prediction method using the present invention and other existing oversampling methods on the customer churn datasets "Bank Marketing Dataset," "Telco Customer Churn," and "Credit Card Customer Churn Prediction." The results show that the present invention achieves optimal results far more often than the other comparison methods in terms of Accuracy, F1_score, Precision, and AUC on the three datasets, demonstrating its superior classification performance.

Claims

1. Customer churn prediction method based on oversampling classification, characterized by: The following steps are involved: Step 1: Obtain and preprocess the customer churn dataset; Step 2: Use k-means clustering algorithm to cluster the data; Step 3: Select and mark the cluster where the minority class samples dominate, and calculate the sampling weight according to the sparsity of the minority class samples in the cluster; Step 4: Calculate the membership of the samples in the cluster whose sampling weight is greater than 0, and divide the space into safe area, overlapping area and noise area based on the membership; Step 5: Select samples based on the region division results and perform linear interpolation oversampling to generate a balanced data set; Step 6: Use the balanced data set to train the classifier, select the optimal classifier to build the final customer churn prediction model, and output the customer churn prediction category and probability.

2. The customer churn prediction method based on oversampling classification according to claim 1, characterized in that: When determining the number of clusters k of the k-means clustering algorithm in step 2, the range of k is first determined by the elbow method, and then the silhouette coefficient method is used to select the optimal k value within the determined range.

3. The customer churn prediction method based on oversampling classification according to claim 1, characterized in that: The step 3 specifically includes the following steps: Step 3.1: Traverse the cluster labels and select and mark the clusters where the minority class samples dominate; Step 3.2: Calculate the sampling weights based on the sparsity of minority class samples within the cluster.

4. The customer churn prediction method based on oversampling classification according to claim 3, characterized in that: The step 3.1 specifically includes the following steps: Step 3.1.

1. Calculate the imbalance rate IR within each cluster: Among them, majority count(c) represents the number of majority class samples, and minority count(c) represents the number of minority class samples; Step 3.1.2: Compare the imbalance rate of each cluster with the imbalance ratio threshold, and mark the clusters whose imbalance rate IR is less than the threshold and the number of minority class samples is greater than 1.

5. The customer churn prediction method based on oversampling classification according to claim 4, characterized in that: The step 3.2 specifically includes the following steps: Step 3.2.

1. Calculate the Euclidean distance matrix of the minority class samples for each labeled cluster. Then, sum all the off-diagonal elements of the Euclidean distance matrix and divide by the number of off-diagonal elements to calculate the average distance of the minority class samples within each labeled cluster. Step 3.2.2: Divide the number of minority class samples in each labeled cluster by the average distance raised to the power of d, where d is the density factor, to calculate the density of minority class samples. Step 3.2.3: Take the inverse of the density of the minority class samples to get the sparsity, and then calculate the sampling weight by the ratio of the sparsity of each labeled cluster to the total sparsity of all labeled clusters.

6. The customer churn prediction method based on oversampling classification according to claim 1, characterized in that: The step 4 specifically includes the following steps: Step 4.1: Calculate the membership matrix for the samples in the labeled cluster and calculate the average membership of each category; Step 4.2: Use the membership-based partitioning strategy to divide the data space into security areas D safe , overlapping area D ovSafe , noise area D nosiy .

7. The customer churn prediction method based on oversampling classification according to claim 6, characterized in that: The step 4.1 specifically includes the following steps: Step 4.1.

1. Calculate the center of category j as c j : Among them, n j is the number of samples of category j, x i is the eigenvalue of sample i of category j; Step 4.1.2: Calculate the center c of sample i and category j j The distance d ij : d ij =|x i -c j | Step 4.1.3: Calculate the membership μ of sample i to category j ij : Among them, k is the category other than category j, C is the number of categories, d ik is the distance between sample i and the center of category k, n is any real number greater than 1, corresponding to the degree of fuzziness of the solution; the membership value is between 0 and 1, and the sum of the membership degrees of a sample is 1; Step 4.1.4: Calculate the average membership of samples in category k to category j For each class j, the average membership is:

8. The customer churn prediction method based on oversampling classification according to claim 7, characterized in that: The step 4.2 specifically includes the following steps: Step 4.2.1: If the sample x i The membership degree μ of its true category j ij Greater than the average membership of category j When , we think that the sample x i In a safe area: Step 4.2.2: If there is a sample x i Membership μ to other categories k ik Greater than the average membership of other categories k When , we think that the sample x i In a noisy area: Step 4.2.3, if the sample x i The membership degree μ of its true category j ij Less than the average membership of category j And the sample x i Membership μ to other categories k ik are all smaller than the average membership of other categories k When , we think that the sample x i In the overlapping area, it is represented as the area of the data space where the safe samples and noise samples are removed: D ovSafe =D-D safe -D noisy 。 9. The customer churn prediction method based on oversampling classification according to claim 8, characterized in that: The step 5 specifically includes the following steps: Step 5.

1. Calculate the safe area D safe For each sample x i The target number of synthetic samples is: Among them, n represents the number of synthesized samples required for the minority class j in the current execution cluster K, n safe Represents the number of safe samples of minority class j in the current execution cluster K; Step 5.2: Safe Area D safe Each sample x in i is the reference point X in linear interpolation oversampling base , in safe area D safe Overlapping area D ovSafe Get sample x from the joint sample space of i The first n nearest neighbor samples of , and randomly select a nearest neighbor sample as the neighbor X neigh , the synthesized sample X new Save in In the collection: X new =X base +λ(X base +X neigh ) λ is the interpolation coefficient; Step 5.3: After processing all clusters with sampling weights greater than 0, we can obtain the synthetic sample sets of each cluster. Again Sum up all synthetic samples S of minority category j in the original data space j , for a two-category dataset, S j Merge it with the original dataset to get a balanced dataset.

10. The customer churn prediction method based on oversampling classification according to claim 1, characterized in that: The step 6 specifically includes the following steps: Step 6.1: For the balanced dataset obtained in step 5, train and evaluate the classification performance of the model on multiple classifiers using accuracy, F1 score, AUC value, and precision as evaluation indicators. Based on the evaluation results, use the grid search method to adjust the classifier hyperparameters to obtain the optimal parameter combination. Step 6.2: Select the best classifier and its parameter combination to train the balanced data set, generate the final customer churn prediction model, and output the customer churn prediction results, including the predicted category of each sample, i.e., churn or not churn, and the probability value corresponding to the predicted category.

Images