Loss prediction method and device based on local performance estimation, equipment and medium

Abstract

The invention provides a loss prediction method and device based on local performance estimation, equipment and a medium, and relates to the field of artificial intelligence, and the method comprises the steps: obtaining feature data of a to-be-predicted sample, inputting the feature data into each base model, and obtaining the original prediction output of each base model; obtaining a neighbor sample set of the to-be-predicted sample in the historical verification set, and calculating a local performance index of the base model in the neighbor sample set for each base model; and calculating a dynamic adjustment coefficient of each base model according to the local performance index of each base model and the global performance index of each base model, generating a fusion weight for the to-be-predicted sample in combination with the global weight and the dynamic adjustment coefficient of the base model, and performing weighted fusion on the original prediction output by using the dynamic fusion weight to obtain a loss prediction result. The method can adapt to telecommunication user groups with high data heterogeneity, and the loss prediction precision of individual users is improved.

Description

Loss prediction methods, devices, equipment, and media based on local performance estimation Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a churn prediction method, apparatus, device and medium based on local performance estimation. Background Technology

[0002] In the telecommunications industry, accurately predicting user churn tendency is fundamental to taking timely retention measures. Therefore, churn prediction has become a core component of customer relationship management, aiming to identify the probability of user churn by analyzing historical user behavior data.

[0003] Currently, to improve prediction accuracy, the industry widely adopts machine learning-based model fusion techniques for user churn prediction. Mainstream implementations, such as stacking or weighted averaging, typically involve first training multiple heterogeneous base models (e.g., gradient boosting trees, support vector machines) using historical data. Then, based on the comprehensive performance metrics of these base models across the entire validation dataset (e.g., overall accuracy, AUC), a fixed fusion weight is assigned to each model. During the prediction phase, this pre-defined weight is used to weight and fuse the outputs of each model for all users to be predicted, ultimately yielding the churn probability.

[0004] However, the aforementioned static model fusion method based on global performance has the following problems: Due to the significant heterogeneity within the telecommunications user group, the feature distribution and behavioral patterns of different user subgroups vary greatly. Static weights, essentially the average performance of the model across all user groups, cannot adapt to this data heterogeneity. As a result, for a specific individual user, even if a base model performs well within its own similar user group (locally), it may be assigned a lower weight during fusion due to its generally poor global performance, failing to fully leverage its local advantages. This "one-size-fits-all" fusion strategy is difficult to flexibly adjust according to the specific characteristics of each user, leading to low prediction accuracy for individual users. Summary of the Invention

[0005] This application provides a churn prediction method, apparatus, device, and medium based on local performance estimation, which solves the problem that existing static model fusion methods based on global performance cannot adapt to the heterogeneity of user data, resulting in low prediction accuracy for individual users.

[0006] In a first aspect, this application provides a churn prediction method based on local performance estimation, the method comprising:

[0007] The feature data of the sample to be predicted is obtained, and the feature data of the sample to be predicted is input into each base model to obtain the original prediction output of each base model; the feature data includes user attribute features, consumption behavior features, service usage features and complaint record features;

[0008] Obtain the set of nearest neighbor samples of the sample to be predicted in the historical validation set, and calculate the local performance index of the base model in the set of nearest neighbor samples for each base model;

[0009] Based on the local performance metrics and global performance metrics of each base model, the dynamic adjustment coefficient of each base model is calculated. Combining the global weights of the base models and the dynamic adjustment coefficients, a fusion weight is generated for the sample to be predicted. The original prediction output is then weighted and fused using the fusion weight to obtain the churn prediction result.

[0010] In one possible implementation, calculating the local performance metric of each base model in the nearest neighbor sample set specifically includes:

[0011] Calculate the prediction accuracy of each base model on the nearest neighbor sample set, and use it as the local performance index of the base model on the nearest neighbor sample set.

[0012] In one possible implementation, the method further includes:

[0013] Heterogeneous base models are trained based on user historical data, and each base model adopts a different feature subset and / or model structure;

[0014] The validation samples from the historical validation set are input into each of the base models for prediction to obtain the prediction results of the validation samples; the feature vector of the validation sample, the prediction results of each base model for the validation sample, and the real label corresponding to the validation sample are stored.

[0015] The global weights of each base model are calculated based on the AUC values ​​of each base model on the historical validation set.

[0016] In one possible implementation, the calculation of the global weights of each base model based on the AUC values ​​of each base model in the historical validation set specifically includes:

[0017] For each base model, obtain the AUC value of the base model on the historical validation set;

[0018] The global weight of a base model is obtained by calculating the ratio of the AUC value of each base model to the sum of the AUC values ​​of all base models.

[0019] In one possible implementation, the method further includes:

[0020] For continuous features in the user's historical data, Gaussian distribution is used to handle missing values;

[0021] For the categorical features in the user's historical data, a generative adversarial network is used to handle missing values.

[0022] In one possible implementation, the method further includes:

[0023] Calculate the importance of the SHAP value for each feature in the user's historical data;

[0024] By recursively eliminating features, the feature data with the lowest preset proportion of importance is removed until the preset conditions are met, thus completing the feature selection.

[0025] In one possible implementation, obtaining the set of nearest neighbor samples of the sample to be predicted in the historical validation set specifically includes:

[0026] The nearest neighbor sample set is obtained by selecting the N verification samples with the highest similarity to the feature data of the sample to be predicted from the historical verification set, where N is a preset positive integer.

[0027] Secondly, this application provides a churn prediction device based on local performance estimation, the device comprising:

[0028] The first acquisition module is used to acquire feature data of the sample to be predicted, input the feature data of the sample to be predicted into each base model, and obtain the original prediction output of each base model; the feature data includes user attribute features, consumption behavior features, service usage features, and complaint record features.

[0029] The second acquisition module acquires the set of nearest neighbor samples of the sample to be predicted in the historical validation set, and calculates the local performance index of the base model in the set of nearest neighbor samples for each base model.

[0030] The prediction module is used to calculate the dynamic adjustment coefficient of each base model based on the local performance index and the global performance index of each base model, generate the fusion weight for the sample to be predicted by combining the global weight of the base model and the dynamic adjustment coefficient, and use the fusion weight to perform weighted fusion on the original prediction output to obtain the churn prediction result.

[0031] Thirdly, this application provides an electronic device, including: a processor, and a memory communicatively connected to the processor;

[0032] The memory stores computer-executed instructions;

[0033] The processor executes computer execution instructions stored in the memory to implement the method as described in the first aspect.

[0034] Fourthly, this application provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the method described in the first aspect.

[0035] The churn prediction method, apparatus, device, and medium based on local performance estimation provided in this application have the following technical effects:

[0036] This method acquires the feature data of the current user (the sample to be predicted) and inputs it into each base model to obtain the raw prediction output, thus providing a preliminary judgment of each base model on the sample. In the historical validation set, it finds the set of nearest neighbor samples most similar to the current sample, essentially locating a user group with similar behavioral characteristics within a large user population. Then, for each base model, its local performance index on similar user groups is calculated, allowing evaluation of each base model's performance within the current user's similar user group. Finally, dynamic adjustment coefficients are calculated based on the local and global performance indices of each model. These coefficients, combined with preset global base weights and dynamic adjustment coefficients, generate a set of fusion weights specifically for the current sample. This dynamic weight set is then used to fuse the raw prediction outputs of each base model. Essentially, for the current user, the base model that performs most reliably among its similar user groups contributes the most to the final decision, thus adapting to the highly heterogeneous data of telecommunications users and improving the accuracy of churn prediction for individual users. Attached Figure Description

[0037] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0038] Figure 1 is a schematic flowchart of the churn prediction method based on local performance estimation provided in an embodiment of this application;

[0039] Figure 2 is a schematic flowchart of the global weight calculation method provided in an embodiment of this application;

[0040] Figure 3 is a schematic diagram of the churn prediction method based on local performance estimation provided in the embodiments of this application.

[0041] Figure 4 is a schematic diagram of a loss prediction device based on local performance estimation;

[0042] Figure 5 is a schematic diagram of the electronic device provided in an embodiment of this application.

[0043] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0044] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments matching this application. Rather, they are merely examples of apparatuses and methods matching some aspects of this application as detailed in the appended claims.

[0045] Because the telecommunications user base is highly heterogeneous, with significant differences in behavioral characteristics and data distribution among different user groups, such as inactive users, young users, and elderly users, a model may perform well in one user group but only moderately in others. Existing model fusion methods (such as weighted averaging or stacked ensemble) assign a fixed fusion weight to all models based on their performance across the entire dataset (global). This "one-size-fits-all" static fusion approach cannot flexibly adjust to the data characteristics of the individual user being predicted. For a specific user to be predicted, even if a model consistently performs well within its similar user group (local), it may be assigned a lower weight due to its moderate global performance. Consequently, its "local advantage" within that user's similar user group cannot be reflected in the final prediction result, leading to insufficient accuracy in predicting individual user churn.

[0046] To address the aforementioned problems, the inventors of this application conceived of a method for churn prediction that involves first identifying similar user groups for the current user, then evaluating the performance of each model within those similar user groups, and adjusting the contribution of each model's prediction results to the final prediction result based on the evaluated performance. The model that performs well within the similar user groups contributes more to the final prediction result, thereby enabling dynamic adjustment of the fusion strategy based on the characteristics of the current user.

[0047] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0048] First, let's explain the terms used in this embodiment:

[0049] Robust-Scaler: A data standardization method that uses the median and interquartile range to scale the data. Even if there are extremely large or small outliers in the data, the scaled data will not be distorted because the median and interquartile range are themselves insensitive to these values, resulting in a more robust outcome. For example, if a set of data is [1, 2, 3, 4, 100], the number 100 may be an outlier. After scaling with Standard-Scaler, all data points will be compressed into a very small range due to the extreme value of 100. However, with Robust-Scaler, since the median (3) and interquartile range (the difference between the third quartile 4 and the first quartile 2, i.e., 2) are minimally affected by 100, the scaled data better reflects the distribution of the vast majority of normal data.

[0050] Target-Encoding: A method for encoding categorical features. The basic idea is to replace the category itself with the mean (or other statistic) of the target variable corresponding to each category in a categorical feature. For example, in a task to predict user purchase rates, the encoded value for the city "Beijing" could be the historical average purchase rate of Beijing users.

[0051] The Gaussian Mixture Model (GMM) is based on the idea that any complex data distribution can be viewed as a mixture of multiple simple Gaussian distributions (i.e., normal distributions) with certain weights. The core objective of the GMM is to find the parameters of these Gaussian distributions (the mean, variance, and mixture weights of each distribution). The model works by iteratively optimizing the model parameters through two alternating steps: the E-step (expectation step) calculates the probability of each data point belonging to each Gaussian distribution; the M-step (maximization step) updates the parameters and weights of each Gaussian distribution based on the results of the E-step, until the model converges.

[0052] Mahalanobis distance: This method calculates the distance between two points after considering the covariance structure of the dataset (i.e., the correlation between dimensions). The method first rotates and scales the data to make the dimensions uncorrelated and the variance equal to 1, and then calculates the Euclidean distance. Its key feature is its insensitivity to units, as the method has already normalized the data using the covariance matrix. Therefore, even if the units of the data features are different (e.g., height is in meters, weight is in kilograms), the Mahalanobis distance can be calculated directly without pre-standardization.

[0053] Example 1

[0054] Figure 1 is a schematic flowchart of a churn prediction method based on local performance estimation provided in an embodiment of this application. This method can be applied to servers. As shown in Figure 1, the method includes:

[0055] S101. Obtain the feature data of the sample to be predicted, input the feature data of the sample to be predicted into each base model, and obtain the original prediction output of each base model; the feature data includes user attribute features, consumption behavior features, service usage features, and complaint record features.

[0056] In this step, the server can asynchronously retrieve the required raw feature data fields from business databases (such as user profile databases, billing databases, and signaling databases), which may specifically include feature data of the following modalities:

[0057] User attribute modalities: such as age segments, network duration segments, and package type;

[0058] Consumer behavior modalities: such as average spending over the past three months, spending volatility coefficient, international roaming spending indicators, etc.;

[0059] Service usage modality: such as total monthly data usage, call duration, and activity level of different business apps;

[0060] Interactive complaint modality: such as the number of times customer service was contacted in the past six months, the number of complaint tickets, and the distribution vector of complaint types;

[0061] After obtaining the data for the above modalities, the server can preprocess the data for each modality separately.

[0062] Because telecommunications user data often contains outliers (such as a very small number of users with extremely high consumption or extremely low data usage), these outliers can cause mean-variance-based standardization methods to perform poorly. Therefore, in this embodiment, for continuous features, the server uses Robust-Scaler (scaling using the median and quantiles) for standardization to reduce the impact of outliers.

[0063] For categorical features, the server can use Target-Encoding (mean-based encoding of the target variable) to process them, converting the category value into a statistic of the target variable (whether there is churn) under that category, and directly establishing the association between the category and the prediction target of this method.

[0064] The server concatenates the feature vectors of each modality feature data after processing to obtain the final feature vector of the sample to be predicted. .

[0065] Next, the server loads multiple pre-trained heterogeneous base models. These base models use different algorithms, such as logistic regression, random forest, gradient boosting tree, neural network, etc.

[0066] The server will use the feature vector All base models are input in parallel, and each model outputs a predicted probability. and the corresponding forecast uncertainty indicators For example, for Bayesian neural network models, the prediction uncertainty index It can be approximated as the prediction variance, while for tree models (such as random forests), uncertainty can be expressed as the variance of the prediction results of all trees in the forest.

[0067] Finally, the server records the predicted probability output by each model. and predictive uncertainty indicators .

[0068] S102. Obtain the set of nearest neighbor samples of the sample to be predicted in the historical validation set, and calculate the local performance index of the base model in the set of nearest neighbor samples for each base model.

[0069] In this step, the server can employ a cluster-based hierarchical retrieval strategy. Specifically, during the offline phase, the server uses an unsupervised clustering algorithm (such as the Gaussian Mixture Model, GMM) to divide all samples in the historical validation set into... The system identifies several clusters (user groups) and stores the index of their centroid and representative samples for each cluster. Then, during the online prediction phase, the server can first calculate the feature vector of the sample to be predicted. Probability of belonging to each cluster (The probability of belonging to each cluster is calculated based on the GMM model). Then, the server selects the top clusters with the highest probabilities. Clusters, For example, it could be 2.

[0070] Next, the server is here. Within a set of samples in a cluster, Mahalanobis distance can be used to calculate the feature vector of the sample to be predicted. Distance to samples within the cluster.

[0071] The server eventually got from this Select the nearest cluster from each cluster. These samples together form the nearest neighbor sample set. .

[0072] For each base model, the server can compute its nearest neighbor set. The local Brier score is an indicator that can effectively assess the degree of calibration of probabilistic predictions. For example, the formula for calculating the local performance index is as follows:

[0073]

[0074] in:

[0075] It is a model Historical samples It is a prediction probability. yes The true labels (churn is 1, no churn is 0). In this formula, subtracting the Brier score from 1 makes... The larger the value, the better the model. The better the performance.

[0076] At the same time, the server can utilize the prediction uncertainty index obtained in step S101. Since higher uncertainty means less reliable model predictions for the current sample, the above formula can be modified to obtain the final local performance index. The calculation formula is as follows:

[0077]

[0078] S103. Based on the local performance index and global performance index of each base model, calculate the dynamic adjustment coefficient of each base model. Combine the global weight and dynamic adjustment coefficient of the base model to generate fusion weights for the sample to be predicted. Use the dynamic fusion weights to perform weighted fusion on the original prediction output to obtain the churn prediction result.

[0079] In this step, the server can read each base model from the storage unit. Global performance metrics on the full validation set. The calculation formula can be, for example:

[0080]

[0081] in, This represents the total number of samples in the complete validation set. For the model For the The probability that a sample is predicted to be positive (churn). For the first The true label of each sample (1 for churn, 0 for no churn).

[0082] Dynamic adjustment coefficient for:

[0083]

[0084] For global weights The server can calculate the average correlation coefficient between each model and the predictions of other models. The lower the correlation coefficient, the more "unique perspective" the model provides, and the higher its weight should be assigned. This allows for a greater focus on the diversity of contributions made by the model.

[0085] The server calculates the unnormalized dynamic fusion weights as follows:

[0086]

[0087] Furthermore, to prevent the weights from becoming extremely skewed due to the occasional extremely high or low local performance of a model, the server can set a pruning threshold. For example, it can... The weights are limited to between 0.5 and 2.0, and then all weights are normalized to obtain the final dynamic fusion weights. .

[0088] Finally, the server uses dynamic fusion weights to perform weighted fusion of the original prediction output to obtain the churn prediction result:

[0089]

[0090] The method shown in Figure 1 has the following technical effects:

[0091] This method acquires the feature data of the current user (the sample to be predicted) and inputs it into each base model to obtain the raw prediction output, thus providing a preliminary judgment of each base model on the sample. In the historical validation set, it finds the set of nearest neighbor samples most similar to the current sample, essentially locating a user group with similar behavioral characteristics within a large user population. Then, for each base model, its local performance index on similar user groups is calculated, allowing evaluation of each base model's performance within the current user's similar user group. Finally, dynamic adjustment coefficients are calculated based on the local and global performance indices of each model. These coefficients, combined with preset global base weights and dynamic adjustment coefficients, generate a set of fusion weights specifically for the current sample. This dynamic weight set is then used to fuse the raw prediction outputs of each base model. Essentially, for the current user, the base model that performs most reliably among its similar user groups contributes the most to the final decision, thus adapting to the highly heterogeneous data of telecommunications users and improving the accuracy of churn prediction for individual users.

[0092] Example 2

[0093] The method provided in this embodiment can be executed by one server or a cluster of multiple servers. The server is pre-installed with a Linux operating system and configured with a MySQL database for storing historical data, a Redis cache for storing intermediate results (such as nearest neighbor sample sets and model prediction results), and a Python runtime environment to support the training and prediction of machine learning models.

[0094] Figure 2 is a schematic flowchart of the global weight calculation method provided in an embodiment of this application. As shown in Figure 2, the method includes:

[0095] S201. Train heterogeneous base models based on user historical data, with each base model employing different feature subsets and / or model structures;

[0096] The server can obtain historical data of the sample to be predicted (the current user) from the business database (such as a customer relationship management system). The historical data includes user attribute characteristics (such as age and package type), consumption behavior characteristics (such as average monthly consumption and the interval between the last consumption), service usage characteristics (such as monthly login times and average online time), and complaint record characteristics (such as the number of historical complaints).

[0097] Next, the server enters the data preprocessing stage. Specifically, the preprocessing includes using Gaussian distribution to handle missing values ​​for continuous features in the user's historical data, and using generative adversarial networks to handle missing values ​​for categorical features in the user's historical data.

[0098] The server can handle missing values ​​for continuous features in user historical data (such as "average monthly spending", "network duration", "package value utilization rate") using a Gaussian distribution. Specifically, for each continuous feature with missing values, the server first calculates its sample mean from the non-missing data for that feature. and sample standard deviation These statistics are used to construct a Gaussian (normal) distribution model that approximates the distribution of this characteristic data, namely:

[0099]

[0100] For each missing value in this feature, the server generates a random number that conforms to the distribution pattern to fill the missing value. That is, for each missing position, a random value is generated. ,in .

[0101] Using a Gaussian distribution to handle missing values ​​has the following technical advantages:

[0102] Using fixed-value imputation significantly alters the original distribution of the feature, reducing variance and creating an unrealistic "peak" at the fixed value. In contrast, Gaussian-based random imputation preserves the overall variance of the feature, making the imputed data distribution closer to the original true distribution and avoiding distributional bias introduced by imputation. Furthermore, since machine learning models learn the distribution patterns of feature data during training, imputing a large number of missing values ​​with fixed values ​​might lead the model to mistakenly identify the fixed value as a common and important pattern, thus learning incorrect patterns. The Gaussian distribution method, by introducing randomness consistent with the original distribution, allows the model to learn patterns that more closely resemble real-world data relationships.

[0103] When handling missing values ​​for categorical features (such as "package type", "user level", "main activity area", etc.) in user historical data, the server can employ a method based on Generative Adversarial Networks (GANs). Specifically, this includes the following steps:

[0104] The server first uses all non-missing samples containing categorical features with missing values ​​as training data. To facilitate GAN processing, the server needs to numerically encode these categorical labels. For example, the server can use one-hot encoding to convert each category into a binary vector.

[0105] The server needs to train a dedicated GAN model, which consists of two parts:

[0106] Generator: Its input is a random noise vector sampled from a standard normal distribution, and its goal is to output a synthetic data that looks like a real one-hot encoded vector.

[0107] Discriminator: Its input is a data sample (which may come from real non-missing data or from a generator), and its goal is to determine whether the sample is "real" or "generated".

[0108] Through adversarial training of the generator and discriminator, the generator is eventually able to learn the complex probability distribution of real categorical data and generate synthetic data with sufficient "realism".

[0109] After the GAN model is trained, it enters the stage of generating and filling missing values. For each missing categorical feature value that needs to be filled, the server can use the trained generator to generate a synthetic data sample. Since the generator's output is a vector, the server can convert the vector back into a specific category label by sampling or taking the maximum probability value, and use this generated category label to fill the corresponding missing value.

[0110] Using adversarial generative networks to handle missing values ​​in categorical features has the following technical advantages:

[0111] Because simple imputation methods often completely ignore the complex relationships that may exist between categorical features and other features, while GAN models have a strong distribution learning ability, they can not only learn the distribution of the categorical feature itself (such as which package is most popular), but also implicitly learn the deep, non-linear relationships between the feature and other features (such as consumption behavior and user attributes). For example, it may learn the pattern that "high-spending users are more likely to choose a certain type of package." Therefore, the imputation values ​​generated by the GAN model are not random, but rather the most likely reasonable values ​​given the context of other feature information of the current sample. This ensures that the imputed data maintains high internal consistency and authenticity, reduces noise introduced by the imputation process, and ultimately improves the overall quality of the preprocessed dataset, thus enhancing the prediction accuracy of the method.

[0112] Next, the server can continue with the following processing:

[0113] Step 1: Calculate the importance of the SHAP value for each feature in the user's historical data;

[0114] This step calculates the SHAP importance of each feature data to quantify the contribution of each feature to the model's prediction results. Specifically, the server first trains a baseline machine learning model, such as the Light-GBM model, using the complete feature set (i.e., all preprocessed user attribute features, consumption behavior features, service usage features, and complaint record features), and uses this model as the baseline model.

[0115] Then, the server extracts a subset of samples from the historical validation set and uses the SHAP (Shapley-Additive-exPlanations) framework to interpret the baseline model. For each instance in the sample, the SHAP framework calculates the contribution score of each feature (such as "package value utilization rate" or "number of customer service contacts") to the final predicted value of that instance; this score is the SHAP value. The larger the absolute value of a feature's SHAP value, the greater the influence of that feature on the model output.

[0116] Next, the server calculates the absolute average of the SHAP values ​​of each feature across all samples to obtain the SHAP importance of that feature. The purpose of SHAP importance is to accurately and consistently reflect the overall influence of each feature.

[0117] Step 2: Through recursive feature elimination, remove feature data with the lowest preset proportion of importance until the preset conditions are met to complete feature selection.

[0118] The purpose of this step is to select the optimal feature subset based on the importance of SHAP values ​​through an iterative process.

[0119] Specifically, the server first performs initialization, which initializes the current feature set to a complete feature set.

[0120] Then, the server trains a new model (of the same type as the baseline model, such as Light-GBM) using the current feature set and evaluates its performance on a separate validation set. The performance evaluation metric used in this embodiment is AUC. The server records the AUC value of the model corresponding to the current feature set. Based on the importance of the SHAP values ​​of each feature calculated in step 1, the server sorts all features in the current feature set from highest to lowest importance. Then, the server removes features with the lowest preset percentage of importance (e.g., the lowest preset percentage could be 10%) from the current feature set.

[0121] The server retrains the model using the new feature set after the previous round of elimination and evaluates its AUC performance. Then, it determines whether a preset condition is met. In this embodiment, the preset condition could be that the model performance drops beyond a certain threshold, such as 0.005. Specifically, if the AUC value of the new model decreases by less than or equal to 0.005 compared to the AUC value recorded in the previous round, the server returns to step 1 and continues with the next round of elimination; if the decrease is greater than 0.005, it indicates that the features eliminated in the previous round have caused unacceptable damage to the model performance, and the loop terminates. At this point, the feature set at the end of the previous round is the final optimal feature subset.

[0122] The above-described method for recursive feature elimination based on SHAP value importance has the following technical advantages:

[0123] 1. This method reduces data noise by eliminating redundant and irrelevant features, allowing each base model to focus more on key signals for training, thereby improving the accuracy, training speed, and generalization ability of each base model and avoiding overfitting of the base models.

[0124] 2. Because the selected feature subset is more representative, the similarity calculated is more accurate when selecting the N validation samples with the highest feature data similarity to the sample to be predicted from the historical validation set.

[0125] 3. Since an accurate set of nearest neighbor samples is fundamental for calculating reliable local performance metrics, if the nearest neighbor samples are selected inaccurately due to feature noise, the subsequent dynamic adjustment coefficients and fusion weights will also be biased. This method, by purifying the feature space, can ultimately improve the accuracy of local performance metric calculation.

[0126] 4. By reducing the number of features, the computational complexity and storage overhead during base model training and prediction can be reduced.

[0127] Finally, the server uses the preprocessed user history data to train multiple base models with different structures. For example, the base models may include a gradient boosting tree model (such as XGBoost) that focuses on user attribute features and consumption behavior features, a recurrent neural network (RNN) that focuses on service usage feature sequences, and a wide & deep model that integrates all feature subsets. These models are trained with different hyperparameter settings to ensure the diversity of the base models.

[0128] S202. Input the validation samples from the historical validation set into each base model for prediction to obtain the prediction results of the validation samples; store the feature vector of the validation samples, the prediction results of each base model for the validation samples, and the corresponding true labels of the validation samples.

[0129] In this step, the server uses a portion of historical data that was not used in training as a historical validation set. Each validation sample in the historical validation set is input into all trained base models to obtain the prediction result of each model for each validation sample. The server stores the feature vector of each validation sample, the prediction results of each base model for it, and its true churn label.

[0130] S203. Based on the AUC values ​​of each base model on the historical validation set, calculate the global weights of each base model.

[0131] It should be noted that the purpose of this step is to assign a global weight to each base model, which reflects the overall discriminative ability of the base model on the entire historical validation set. The technical idea behind this step is that the stronger a base model's ability to distinguish between churned and non-churned users globally, the greater the contribution of its prediction results during the initial fusion.

[0132] In this step, the global weights of each base model are calculated based on their AUC values ​​on the historical validation set. Specifically, this includes:

[0133] Step 1: For each base model, obtain the AUC value of the base model on the historical validation set;

[0134] Specifically, obtaining the AUC value of the base model on the historical validation set is achieved through the following steps:

[0135] Server retrieves historical verification sets ,in It is the feature vector of the verification sample. It is its true label (for example, 1 means churn, 0 means no churn).

[0136] Then, the server will take each sample from the historical verification set. Input to each base model In this process, the predicted probability values ​​output by the base model are obtained. (That is, the probability that the base model considers the sample to belong to the churn category).

[0137] For each base model, the server... The predicted probability of it over all validation samples Corresponding real tags The comparison is then performed to calculate the AUC value. It's important to explain that AUC stands for "Area Under the Curve," specifically referring to the area under the ROC curve. AUC measures the overall ability of the base model to rank the predicted probability of "churned users" samples higher than that of "non-churned users" samples. The AUC value ranges from 0.5 to 1.0. An AUC value of 0.5 indicates that the base model's discriminative ability is equal to random guessing; an AUC value greater than 0.5 and less than 1 indicates that the base model has some discriminative ability; and an AUC value equal to 1 indicates that the base model can perfectly distinguish between all positive and negative samples.

[0138] Step 2: Calculate the global weight of each base model by taking the ratio of its AUC value to the sum of the AUC values ​​of all base models.

[0139] In this step, for the first Base Model Its global weight The formula for calculating the global weight is:

[0140]

[0141] in, It is the total number of base models. It is the base model The AUC value, the denominator is all The sum of the AUC values ​​of each base model.

[0142] Next, the global weights of each base model are normalized to ensure that the result of the fusion of the outputs of all base models is a meaningful probability value between 0 and 1.

[0143] Calculating the global weights using the above method yields the following technical advantages:

[0144] Since the global weights are derived from reproducible performance metrics (AUC) statistically obtained from a large amount of historical data of the base models, they provide an accurate benchmark for the subsequent fusion of the outputs of each base model.

[0145] Figure 3 is a schematic diagram of the second step in the churn prediction method based on local performance estimation provided in the embodiments of this application. The method includes:

[0146] As shown in Figure 2, the method includes:

[0147] S301. Obtain the feature data of the sample to be predicted, input the feature data of the sample to be predicted into each base model, and obtain the original prediction output of each base model; the feature data includes user attribute features, consumption behavior features, service usage features, and complaint record features.

[0148] Specifically, the server receives the feature data of the sample to be predicted and inputs this feature data in parallel into all the heterogeneous base models that have been pre-trained using the aforementioned method. Each base model then independently makes a prediction using the feature data and outputs a raw prediction output between 0 and 1, representing the churn probability of the sample to be predicted.

[0149] The server can temporarily store these raw prediction outputs in the server's memory.

[0150] S302. Select the N validation samples with the highest feature data similarity to the sample to be predicted from the historical validation set to obtain the nearest neighbor sample set, where N is a preset positive integer; calculate the prediction accuracy of each base model on the nearest neighbor sample set as the local performance index of the base model on the nearest neighbor sample set.

[0151] In this step, the server finds the most similar neighbors for the current sample to be predicted from the stored historical validation set. Specifically, Euclidean distance or cosine similarity can be used as the similarity metric. For example, the feature space distance between the sample to be predicted and all samples in the historical validation set is calculated, and then the N nearest validation samples (e.g., N=50) are selected to form the nearest neighbor sample set of the sample to be predicted.

[0152] For each sample in the nearest neighbor set, the server retrieves its true churn label and the previous prediction results of each base model. Then, for each base model, the server calculates its prediction accuracy on that nearest neighbor set: the proportion of samples correctly predicted by the statistical model (predicted category matches the true label) out of the total number of nearest neighbor samples. This accuracy serves as the local performance metric of the base model for the current sample to be predicted. .

[0153] S303. Based on the local performance indicators and global performance indicators of each base model, calculate the dynamic adjustment coefficient of each base model. Combine the global weights and dynamic adjustment coefficients of the base models to generate fusion weights for the samples to be predicted. Use the dynamic fusion weights to perform weighted fusion on the original prediction output to obtain the churn prediction result.

[0154] Since the global weights of each base model have already been obtained in the method shown in Figure 2, in this step, the server reads the global weights of each base model from the database or cache.

[0155] For each base model, the server sets its local performance metrics ( Divide it by its global performance metric (i.e., its AUC value on the historical validation set). The dynamic adjustment coefficients of the model are obtained as follows:

[0156]

[0157] This coefficient reflects the relative performance of the base model in the local region of the sample to be tested.

[0158] Next, the server generates the fusion weights, specifically in the following manner:

[0159] The global weights of each base model ( ) and its dynamic adjustment coefficient ( Multiply by , and then normalize to obtain the dynamic fusion weights for the current sample to be predicted. The normalization formula is:

[0160] ,

[0161] in The total number of base models.

[0162] In the weighted fusion stage, the server performs a weighted summation of the original prediction outputs obtained from each base model in step S301 with the calculated dynamic fusion weights. The calculation formula is as follows:

[0163]

[0164] This is the original prediction output of the base model.

[0165] The final weighted average is the final churn prediction probability. The server can make a judgment based on a preset threshold (such as 0.5). If the final churn prediction probability is greater than the threshold, the user is deemed to be at risk of churn, and this result can be returned to the business system.

[0166] Figure 4 is a schematic diagram of a churn prediction device based on local performance estimation. As shown in Figure 4, the device 40 includes:

[0167] The first acquisition module is used to acquire the feature data of the sample to be predicted, input the feature data of the sample to be predicted into each base model, and obtain the original prediction output of each base model; the feature data includes user attribute features, consumption behavior features, service usage features, and complaint record features.

[0168] The second acquisition module acquires the set of nearest neighbor samples of the sample to be predicted in the historical validation set, and calculates the local performance index of the base model in the set of nearest neighbor samples for each base model.

[0169] The prediction module is used to calculate the dynamic adjustment coefficient of each base model based on the local performance index and the global performance index of each base model. It combines the global weights and dynamic adjustment coefficients of the base models to generate fusion weights for the samples to be predicted. The original prediction output is then weighted and fused using the dynamic fusion weights to obtain the churn prediction result.

[0170] Figure 5 is a schematic diagram of an electronic device provided in an embodiment of this application. As shown in Figure 5, the electronic device 50 provided in this embodiment includes at least one processor 501 and a memory 502. Optionally, the device 50 further includes a communication component 503. The processor 501, the memory 502, and the communication component 503 are connected via a bus 504.

[0171] In a specific implementation, at least one processor 501 executes computer execution instructions stored in memory 502, causing at least one processor 501 to perform the above-described method.

[0172] The specific implementation process of processor 501 can be found in the above method embodiments, and its implementation principle and technical effect are similar. It will not be repeated here.

[0173] In the above embodiments, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor.

[0174] The memory may include random access memory (RAM) and may also include non-volatile memory (NVM), such as at least one disk storage device.

[0175] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0176] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0177] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the above-described method.

[0178] The aforementioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0179] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and the readable storage medium can exist as discrete components in the device.

[0180] The division of units is merely a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0181] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0182] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0183] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0184] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0185] Finally, it should be noted that other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This invention is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein, and is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.

Claims

1. A churn prediction method based on local performance estimation, characterized in that, The method includes: acquiring feature data of the sample to be predicted; inputting the feature data of the sample to be predicted into each base model to obtain the original prediction output of each base model; the feature data includes user attribute features, consumption behavior features, service usage features, and complaint record features; acquiring the nearest neighbor sample set of the sample to be predicted in the historical validation set, and calculating the local performance index of each base model in the nearest neighbor sample set for each base model; calculating the dynamic adjustment coefficient of each base model based on the local performance index and the global performance index of each base model; combining the global weight of the base model and the dynamic adjustment coefficient to generate a fusion weight for the sample to be predicted; and using the fusion weight to perform weighted fusion of the original prediction output to obtain the churn prediction result.

2. The method according to claim 1, characterized in that, The step of calculating the local performance index of each base model in the nearest neighbor sample set specifically includes: calculating the prediction accuracy of each base model in the nearest neighbor sample set as the local performance index of the base model in the nearest neighbor sample set.

3. The method according to claim 1, characterized in that, The method further includes: training heterogeneous base models based on user historical data, wherein each base model adopts different feature subsets and / or model structures; inputting validation samples from the historical validation set into each base model for prediction to obtain the prediction results of the validation samples; storing the feature vector of the validation samples, the prediction results of each base model for the validation samples, and the real labels corresponding to the validation samples; and calculating the global weights of each base model based on the AUC values ​​of each base model in the historical validation set.

4. The method according to claim 3, characterized in that, The step of calculating the global weight of each base model based on the AUC value of each base model in the historical validation set specifically includes: for each base model, obtaining the AUC value of the base model in the historical validation set; and obtaining the global weight of the base model by calculating the ratio of the AUC value of each base model to the sum of the AUC values ​​of all base models.

5. The method according to claim 4, characterized in that, The method further includes: processing missing values ​​for continuous features in the user's historical data using a Gaussian distribution; and processing missing values ​​for categorical features in the user's historical data using a generative adversarial network.

6. The method according to claim 4, characterized in that, The method further includes: calculating the importance of the SHAP value of each feature data in the user's historical data; and removing feature data with the lowest preset proportion of importance through recursive feature elimination processing until a preset condition is met to complete feature selection.

7. The method according to any one of claims 1-6, characterized in that, The step of obtaining the set of nearest neighbor samples of the sample to be predicted in the historical verification set specifically includes: selecting N verification samples with the highest similarity to the feature data of the sample to be predicted in the historical verification set to obtain the set of nearest neighbor samples, where N is a preset positive integer.

8. A churn prediction device based on local performance estimation, characterized in that, The device includes: a first acquisition module, configured to acquire feature data of a sample to be predicted, input the feature data of the sample to be predicted into each base model, and obtain the original prediction output of each base model; the feature data includes user attribute features, consumption behavior features, service usage features, and complaint record features; a second acquisition module, configured to acquire the set of nearest neighbor samples of the sample to be predicted in the historical validation set, and calculate the local performance index of each base model in the set of nearest neighbor samples for each base model; and a prediction module, configured to calculate the dynamic adjustment coefficient of each base model based on the local performance index and the global performance index of each base model, combine the global weight of the base model and the dynamic adjustment coefficient to generate a fusion weight for the sample to be predicted, and use the fusion weight to perform weighted fusion of the original prediction output to obtain the churn prediction result.

9. An electronic device, characterized in that, include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory to implement the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1 to 7.

Images