Profit-oriented customer loss prediction method and device based on graph Transform and medium

Abstract

The invention relates to a profit-oriented customer loss prediction method and device based on a graph Transform and a medium, and the method comprises the following steps: constructing a call graph reflecting customer communication behaviors by using CDR data, and supplementing missing links of the graph according to the characteristics of telecommunication services and a ternary closing theory in a social network, so as to improve the accuracy of graph representation; designing a ProfGT model, taking a feature matrix and a graph of nodes as input, and learning representation of the nodes from a complex network relationship; and training a ProfGT model by using a profit-oriented loss function so as to realize the integration of model prediction and customer loss management services. According to the method, a large-scale customer relationship network graph is constructed based on massive CDR data, edge prediction and supplement are performed in combination with telecommunication service characteristics and a common neighbor algorithm, and the problem of poor prediction performance caused by graph sparsity is solved; and a profit-oriented loss function is used for model training, so that the model takes profit as a target and maximizes the target, and therefore, a model prediction result has higher practical value.

Description

Technical Field

[0001] The present invention relates to the technical field of graph neural networks, and particularly to a profit-oriented customer churn prediction method, device and storage medium based on graph transformers. Background Art

[0002] As the telecommunications industry enters the era of the stock market, the dividend space for customer growth has gradually narrowed. Managing the existing customer base and achieving a double jump in revenue and profit have become key issues that telecommunications operators urgently need to solve. Research shows that maintaining existing customers is more cost-effective than acquiring new customers, and the persistence of the business relationship between customers and enterprises is proportional to the profit they contribute. Thanks to the development of big data / AI technology, telecommunications operators have actively carried out the application of big data / AI technology in the field of customer churn prediction, aiming to identify and intervene in churn customers in advance, reduce the customer churn rate, and ultimately maintain a competitive advantage in the fierce market.

[0003] Customer churn prediction (CCP) has become a research hotspot in the telecommunications industry. Many methods based on traditional machine learning (such as logistic regression, random forest, decision tree) and multi-layer perceptron (MLP) have been proposed and applied to customer churn prediction. However, the above methods have significant limitations: 1. In the actual telecommunications industry, it is vulnerable to the influence of customers' social network relationships. Most traditional machine learning methods are based on the assumption of customer independence, and it is difficult to learn the potential impact of the social network relationships between customers on churn, resulting in suboptimal prediction results of the model.

[0004] 2. In commercial applications, operators are more concerned about the profit brought by the prediction results. The customer lifetime value (CLV) of different customers is different. The more high-value customers are predicted, the higher the profit obtained by the operator. However, existing customer churn prediction models often target accuracy or top decile lift, ignoring the differences in the customer lifetime value of different customers.

[0005] 3. With the booming development of the mobile Internet, users tend to use OTT (Over the Top) applications (such as WeChat) for communication, resulting in scarce call graphs constructed based on call detail records (CDR), which cannot reflect the social relationship network of customers, thus weakening the effectiveness of graph algorithms.

[0006] Glossary: CDR (Call Detail Record): The call details record between customers, including fields such as the calling number, called number, start time, and duration.

[0007] Graph Neural Network (GNN): A neural network applied to graph data that can learn features by aggregating the neighbors of nodes and is widely used in fields such as recommendation systems, drug discovery, and traffic flow prediction.

[0008] Graph Transformer: A graph neural network based on Transformer that enables nodes to directly aggregate with all other nodes through a global attention mechanism, addressing drawbacks such as over-smoothing of GNN on large graphs. Summary of the Invention

[0009] A profit-oriented customer churn prediction method, device, and storage medium based on Graph Transformer proposed by the present invention can at least solve one of the technical problems in the background art.

[0010] To achieve the above object, the present invention adopts the following technical solutions: A profit-oriented customer churn prediction method based on Graph Transformer, which is executed by a computer device through the following steps: Construct a call graph reflecting customer communication behavior using CDR data, and supplement the missing links in the graph according to the characteristics of telecom services and the triadic closure theory in social networks to improve the accuracy of graph representation; Design the ProfGT model with the feature matrix of nodes and the graph as inputs to learn the representations of nodes from complex network relationships; Train the ProfGT model with a profit-oriented loss function to integrate model prediction with customer churn management services.

[0011] Further, the specific steps for constructing a call graph reflecting customer communication behavior include: Define the customer call graph as , where the node set is , each node represents a customer, and for each node, there are -dimensional features, which are represented in the form of a matrix for all node feature matrices. is the number of nodes; Let the edge set in the graph be ; The process of constructing the call graph is as follows: (1) If there is a call record between two customers, add an edge to connect them; (2) Link prediction: Add edges based on telecom service characteristics. If customers have the same main card number and the same identifier, add an edge to connect them; Add edges according to the common neighbor algorithm. Calculate the number of common neighbors between nodes and, denoted as , where represents the neighborhood set of nodes ; If , then add an edge.

[0012] Furthermore, the construction process of the ProfGT model is as follows: First, apply positional embeddings to generate the positional encodings of each node. Subsequently, fuse the node features with the positional encodings and input them into three consecutive ProfGT-Layers for processing; finally, use the processed node features for classification prediction; The ProfGT-Layer consists of Global Attention (GlobalAttn), message-passing graph neural networks (MPNNs), and MLP; define the node feature matrix of the model input as: , and the adjacency matrix of the graph is ; Apply two types of positional embeddings: Laplacian positional encoding (LapPE) and random walk positional encoding (RWPE); The Laplacian matrix of the graph is calculated from the adjacency matrix and the degree matrix , and its eigenvalues and eigenvectors are denoted as and respectively; the definition of the Laplacian matrix of the graph:

[0013] The random walk positional encoding (RWPE) uses the random walk operator to define the - step random walk process. When is large enough, the node has a unique topological neighborhood, and RWPE can generate a unique representation; the definition of the RWPE of node is:

[0014] Merge the node feature matrix and the node positional embeddings to generate a new feature matrix, ; this feature matrix is used as the input for the subsequent GlobalAttn module and MPNN module to further extract high - order features; Use Graph Transformers to implement the global attention mechanism; the calculation process of Graph Transformers is as follows:

[0015]

[0016] The input node feature matrix Passes through three fully connected branches respectively, generating , , Three matrices; the weights of the three fully connected branches are defined as , , , d and Are the feature dimension and the number of nodes respectively; And The matrix multiplication gives The weight matrix of, after passing through Softmax, and then multiplying with The matrix multiplies to output the global attention feature matrix of the node ; Uses GraphSAGE as a message passing graph neural network (MPNN); Equation (5) shows the process of aggregating the information of the neighbor nodes of the node, generating the aggregated features, and Equation (6) describes the process of updating the node information with the aggregated features;

[0017] Represents the Layer node The aggregated features of all neighbors, Represents the aggregation function, Represents the node Set of neighbor nodes;

[0018] Represents the node At Layer features, Represents the activation function Relu, Represents the learned weight matrix; combines the aggregated features with Layer features, calculates through the weight matrix, and outputs the new features of the node ;

[0019]

[0020] Updates the features of all nodes in sequence to obtain Feature matrix, ; The feature matrix and The feature matrix are added together to fuse the local embedding information and the global embedding information of the node; finally, through the MLP, the node feature matrix of the Layer is output; the MLP module contains two fully connected layers, and the weight matrices are , 。

[0021] Furthermore, the profit-oriented loss function is as follows: According to the profit calculation formula for customer churn management activities, as shown in Formula (9), a profit-oriented loss function is designed to guide the learning of the model; the profit calculation formula for customer churn management activities:

[0022] where is the profit generated by customer retention activities, is the total number of customers, represents the proportion of target customers, is the proportion of actual churned customers in customer retention activities, When a customer accepts and stays, it is the incentive cost of the company, The proportion of potential target customers who are determined to stay due to incentives is the success rate of incentives, The cost of contacting customers to provide incentives, the customer lifetime value ( ) refers to the average customer lifetime value of retaining customers, refers to the fixed cost of running the customer churn management program; Formula (9) is modified into a formula that can be used for model training and evaluation and is simplified:

[0023] where, represents the set of nodes where the actual situation is churn and the model predicts churn, represents the set of nodes where the actual situation is non-churn and the model predicts churn, and ; represents the cost of retaining customers; The profit loss function includes two parts: revenue and cost; For the revenue part, all churn samples are used as supervision, and let , the predicted probability of each churn sample is multiplied by CLV. On the one hand, it ensures that the loss function is continuous, and on the other hand, the higher the CLV, the higher the weight of the sample, making the model pay more attention to high-value samples; The cost part restricts the model from being too confident and predicting non-churn customers as churn customers; The loss function is designed as in Formula (11):

[0024] In the revenue part, is the number of actual churned customers, is the score of the nodes predicted as churn customers by the model, is the customer lifetime value; when the churn customer score output by the model approaches 1, the profit loss is 0, and thus the revenue is maximized; For the cost part, select the nodes with predicted scores greater than or equal to as the penalty term, denoted as , represents the cost; Combining the cross-entropy loss and the profit loss, the following loss function is finally obtained as formula (12):

[0025] is the cross-entropy loss function, where is the adjustment factor to balance the influence degrees of prediction accuracy and profit.

[0026] On the other hand, the present invention also discloses a computer-readable storage medium storing a computer program, which when executed by a processor causes the processor to execute the steps of the above method.

[0027] On yet another hand, the present invention also discloses a computer device including a memory and a processor, where the memory stores a computer program, and when the computer program is executed by the processor, it causes the processor to execute the steps of the above method.

[0028] As can be seen from the above technical solutions, to solve the deficiencies of the prior art, the present invention proposes a telecom customer churn prediction model based on GraphTransformer, named Profit Graph Transformer (ProfGT). ProfGT can combine and realize profit-oriented customer churn prediction from a complex customer relationship network, providing a novel prediction framework for the telecom industry. The present invention mainly includes: First, this method uses massive CDRs to deeply mine and construct a large-scale customer relationship network, and ProfGT learns node representations by aggregating information of neighboring nodes. In the field of telecom customer churn prediction, it can learn the potential social network relationships of customers.

[0029] Second, to solve the problem of graph sparsity, this method predicts and supplements potential social relationships between customers according to the characteristics of telecom services and the triadic closure principle in social networks, thereby enhancing the integrity of the graph; Third, this method proposes a customer churn prediction model of Graph Transformer, which is based on the global attention mechanism, enabling nodes to interact with all other nodes, and solving the problems of over-smoothing and insufficient expressive ability of traditional GNN models; Fourth, a profit-oriented loss function is designed to train the proposed model. The loss function uses CLV as the weight for churned customers. The higher the CLV, the more important such churned customers are, enabling the model to focus on high-value churned customers, thereby increasing the profit generated by the prediction and achieving the integration of the model with the telecommunications business.

[0030] Finally, the present invention conducts experiments on the million-level call data provided by a provincial telecommunications operator. The experimental results show that the model has significant advantages in predicting customer churn and provides strong support for customer retention strategies in business practices.

[0031] Specifically, the beneficial effects of the present invention are as follows: 1. A large-scale customer relationship network graph is constructed based on massive CDR data, and edge prediction and supplementation are performed in combination with the characteristics of the telecommunications business and the common neighbor algorithm, solving the problem of poor prediction performance caused by graph sparsity; 2. The model is trained using a profit-oriented loss function, enabling the model to aim for maximum profit, so the model prediction results are more practical.

[0032] 3. The ProfGT model proposed by the present invention can learn the potential impact of the relationship between customers on churn through the customer relationship network graph, and has higher prediction performance than machine learning methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 is a flowchart of the present invention; Figure 2 is the composition process of the present invention; Figure 3 is a schematic structural diagram of the ProfGT model of the present invention; Figure 4 is a schematic diagram of the distribution of the degrees of all nodes before and after edge supplementation of the present invention; Figure 5 is a profit curve graph of the model of the present invention with γ = 0.7 under different target customer ratios; Figure 6 is a profit curve of the model of the present invention with γ = 0.3 under different target customer ratios; Figure 7 is a process flow diagram of an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0034] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention.

[0035] The profit-oriented customer churn prediction method based on graph Transformer in this embodiment includes three main parts: a graph construction module, a ProfGT model, and a profit-oriented loss function. Its workflow is as follows Figure 1 shown. First, use CDR data to construct a relational network graph reflecting customer communication behavior, and supplement the missing links in the graph according to the characteristics of telecom services and the triadic closure theory in social networks to improve the accuracy of graph representation. Second, design the ProfGT model to take the feature matrix of nodes and the graph as inputs, and learn the representations of nodes from complex network relationships. Finally, train ProfGT with a profit-oriented loss function to integrate model prediction with customer churn management services.

[0036] The steps for constructing the relational network graph are as follows: Define the customer call graph as , where the node set is , and each node represents a customer. Each node has features of a specific dimension, and the feature matrices of all nodes are represented in matrix form, being the number of nodes. The edge set in the graph is . The process of constructing the call graph: (1) If there is a call record between two customers, add an edge to connect them; (2) Link prediction: Add edges based on telecom service characteristics. If customers have the same main card number and the same identifier, add an edge to connect them; Add edges according to the common neighbor algorithm (CN). Calculate the number of common neighbors between nodes and, denoted as , where represents the neighborhood set of node . If , add an edge.

[0037] Figure 2 Intuitively shows the process of constructing the graph of the present invention. The left side is graph construction based on call records; the middle is edge supplementation based on telecom service characteristics; the right side is edge supplementation by CN, where the dashed line and dotted line respectively represent the edges added based on telecom service characteristics and the common neighbor algorithm.

[0038] The content of the ProfGT model structure is as follows: Graph neural networks (GNNs) learn the features of nodes by aggregating the neighboring information of nodes. In the field of telecom customer churn prediction, it promotes the in-depth exploration of potential social network relationships between nodes and their influence on each other. The model architecture proposed in the present invention is as Figure 3 shown.

[0039] The model first applies positional embeddings to generate the positional encodings of each node. Subsequently, the node features are fused with the positional encodings and input into three consecutive ProfGT-Layers for processing. Finally, the processed node features are used for classification prediction. The ProfGT-Layer consists of Global Attention (GlobalAttn), message-passing graph neural networks (MPNNs), and an MLP. Define the node feature matrix of the model input as: , and the adjacency matrix of the graph as .

[0040] Positional encodings (PE) provide global position information for nodes and play a crucial role in GraphTransformers. This invention applies two types of positional embeddings: Laplacian positional encoding (LapPE) and random walk positional encoding (RWPE).

[0041] LapPE uses Laplacian eigenvectors as node positional embeddings and has properties such as permutation invariance and distance to the graph topology. The Laplacian matrix of a graph is calculated from the adjacency matrix and the degree matrix . Its eigenvalues and eigenvectors are denoted as and respectively. The definition of the Laplacian matrix of a graph:

[0042] In contrast, RWPE is a positional encoding method based on random walk diffusion. It uses the random walk operator to define the step random walk process. When is large enough, a node has a unique topological neighborhood, and RWPE can generate a unique representation. The definition of the RWPE of node :

[0043] Combine the node feature matrix and the node positional embedding to generate a new feature matrix, . This feature matrix is used as the input for the subsequent GlobalAttn module and MPNN module to further extract higher-order features.

[0044] The present invention utilizes Graph Transformers to implement a global attention mechanism. Graph Transformers enable nodes to directly interact with all other nodes, alleviating the limitations of local message passing mechanisms, such as over-smoothing and over-squashing. After being calculated by Graph Transformers, the node representations contain more graph topology information and sequence information. The calculation process of Graph Transformers is as follows:

[0045]

[0046] The input node feature matrix Passes through three fully connected branches respectively, generating 、 、 Three matrices. The weights of the three fully connected branches are defined as 、 、 , d and Are the feature dimension and the number of nodes respectively. And The matrices are multiplied to obtain The weight matrix of, after passing through Softmax, is then multiplied by The matrix to output the global attention feature matrix of the nodes .

[0047] The present invention uses GraphSAGE as a message passing graph neural network (MPNN). Equation (5) shows the process of aggregating the information of the neighbor nodes of the node to generate an aggregated feature, and Equation (6) describes the process of updating the node information with the aggregated feature.

[0048]

[0049] Represents the Layer node The aggregated feature of all neighbors, Represents the aggregation function, Represents the node Set of neighbor nodes.

[0050]

[0051] Represents the node At Layer feature, Represents the activation function Relu, Represents the weight matrix for learning. Combine the aggregated features with the layer features, calculate through the weight matrix, and output the new features of the node.

[0052]

[0053]

[0054] Update the features of all nodes in sequence to obtain the feature matrix, . Add the feature matrix and the feature matrix to fuse the local embedding information and global embedding information of the nodes. Finally, through the MLP, output the node feature matrix of the layer. The MLP module contains two fully connected layers, and the weight matrices are , .

[0055] The profit-oriented loss function is as follows: In business practice, the ultimate goal of the model to predict churn customers is to improve the profitability of the enterprise. However, the existing methods use accuracy or recall as the goal, which is sub-optimal in maximizing the profit of the enterprise. Therefore, according to the profit calculation formula for customer churn management activities, see formula (9), the present invention designs a profit-oriented loss function to guide the learning of the model. Profit calculation formula for customer churn management activities:

[0056] where is the profit generated by the customer retention activity, is the total number of customers, represents the proportion of target customers, is the proportion of truly churned customers in the customer retention activity, is the incentive cost of the company when the customer accepts and stays, is the proportion of potential target customers who decide to stay due to incentives (i.e., the success rate of incentives), is the cost of contacting customers to provide incentives, and the customer lifetime value ( ) refers to the average customer lifetime value of retaining customers, refers to the fixed cost of running the customer churn management program.

[0057] The present invention modifies formula (9) into a formula that can be used for model training and evaluation, and simplifies it:

[0058] where, The set of nodes that are actually churned and predicted to be churned by the model The set of nodes that are actually not churned but predicted to be churned by the model, and 。 Represents the cost paid by retained customers.

[0059] Observing Formula (10), the present invention finds that the profit of the retention activity mainly consists of the customer lifetime value (CLV) and the incentive cost and contact cost paid for retaining customers. The profit loss function proposed by the present invention includes two parts: revenue and cost. The goal of the profit loss function is to maximize the profit through model prediction, which requires the model to maximize the revenue and minimize the cost while accurately identifying all churned customers.

[0060] In the revenue part, all churned samples are used as supervision, and let 。The predicted probability of each churned sample is multiplied by the CLV. On the one hand, it ensures that the loss function is continuous. On the other hand, the higher the CLV, the higher the weight of the sample, making the model pay more attention to high-value samples. The cost part restricts the model from being too confident and predicting non-churned customers as churned customers. The loss function is designed as Formula (11):

[0061] In the revenue part, is the number of truly churned customers, is the score of the nodes predicted to be churned by the model, is the customer lifetime value. When the churned customer score output by the model tends to 1, the profit loss is 0, and thus the revenue is maximized. In the cost part, the nodes with a predicted score greater than or equal to are selected as the penalty term, denoted as , represents the cost.

[0062] Combining the cross-entropy loss and the profit loss, the following loss function is finally obtained as Formula (12):

[0063] is the cross-entropy loss function, where is the adjustment factor to balance the influence degree of prediction accuracy and profit. The present invention finds through experiments that ,the prediction result is optimal.

[0064] The processing flow chart of the present invention is as shown in Figure 7 and is illustrated by the following example: The embodiment of the present invention takes the desensitized customer call record data of a provincial telecommunications operator as an example to describe the whole process from feature extraction, relationship network construction, model training to evaluation.

[0065] The call record data contains the call records of customers for two consecutive months and consists of two parts: Voice of call (VOC) and USER (basic customer information). The VOC data includes fields such as communication object identifier, operator identifier, region, timestamp, call duration, and call type; the USER part covers key information such as customer identity identifier, place of origin, network age, package details (including level, voice and traffic usage), IPTV service status, number of broadband connections, churn status, and billing period. Table 1 details the composition of these data sets.

[0066] Table 1 Call Record Data

[0067] Feature extraction is as follows: The original call record data has problems such as dirty data and serious proportional differences, making these original data difficult to use directly. Therefore, it is necessary to preprocess the data by cleaning and deleting dirty data. After that, statistical features and pattern features are extracted based on this. Statistical features include mean, maximum, minimum, median, quartiles, variance, and standard deviation, which are applied to fields such as call duration to capture the internal distribution law and variability of the data.

[0068] The pattern features in call behavior not only reflect the call habits of users but also reveal in-depth information about network usage behavior. Specifically, the date preference of calls (such as the difference between weekdays and weekends), time preference (such as peak hours in the morning and evening), frequency distribution of incoming and outgoing calls between different networks and the same network, number of roaming calls, and corresponding call durations are extracted. These features together constitute a comprehensive description of the user's communication behavior pattern.

[0069] Finally, a data set containing 105-dimensional feature items is constructed.

[0070] The construction of the relationship network diagram is as follows: Taking customers as nodes to form the node set of the relationship network diagram, and then judging whether there is a call record between customers. If there is, an edge is added. On this basis, potential links between customers are predicted and supplemented according to the characteristics of telecommunications services and the common neighbor algorithm. Figure 4 Shows the distribution of node degrees before and after edge prediction. It can be seen that the degree of nodes increases significantly after edge prediction.

[0071] The model training steps are as follows: The present invention uses two steps of feature extraction and relationship network diagram construction to establish a training set and a test set for model training and evaluation respectively. Table 2 shows the detailed information of the training set and the test set.

[0072] Table 2 Data Set Description

[0073] The present invention utilizes the PyTorch framework and performs all model training and testing on a Linux server equipped with an NVIDIA A10 GPU, 128GB of RAM, and a 32-core Intel(R) Xeon(R) Gold 5218 CPU @ 2.30 GHz. The Adam optimizer is adopted during the model training process, and the initial learning rate is finely set to , and a coefficient of weight decay = 0.0005 is introduced to prevent overfitting.

[0074] The model training is supervised by a profit-oriented loss function. The hyperparameters involved in this loss function are , , .

[0075] Evaluation metrics The present invention evaluates the model using Top Lift and profit.

[0076] Top Lift: The top decile selects the top 10% of customers based on the predicted churn probability, calculates the true churn rate among them, and then divides it by the proportion of churn in the entire customer base. The calculation of the top decile is as shown in formula (13):

[0077] where is the churn rate of the entire customer group. As can be seen from the formula, the higher the top decile value, the better the model. To more comprehensively evaluate the proposed method, this paper uses multiple quantiles to evaluate the churn prediction results, including: the top 1% quantile, the top 5% quantile, and the top 10% quantile.

[0078] The profit evaluation metrics predicted by the model are as shown in formulas (14) and (15):

[0079]

[0080] , is the total number of customers, is the proportion of target customers, represents the cost of retaining customers. is the proportion of churned customers who are incentivized and retained, and unless otherwise specified, it is taken as 0.7. This paper uses multiple quantiles to adjust the proportion of target customers to better evaluate the profit of the model under different proportions of target customers.

[0081] Model Comparison Figure 5 As shown, the present invention selects GraphSAGE based on graph convolution, MLP (Multi-Layer Perceptron), XGBoost, RandomForestClassifier, LogisticRegression, and LightGBM as baseline models for comparison with ProfGT.

[0082] Figure 5 Shows , the profits generated by different model predictions when the proportion of target customers increases from 0.1% to 10%. Figure 6 Shows , the profits generated by different model predictions when the proportion of target customers increases from 0.1% to 5%. The profit generated by the model first increases as the proportion of target customers rises. When the profit reaches its maximum, it then decreases as the proportion of target customers increases. An overly large proportion of target customers leads to an increase in the misjudgment rate of non-churn customers by the model, resulting in a decrease in precision and an increase in marketing costs, which is consistent with the actual marketing activities of the operator. In addition, compared with the baseline models, ProfGT proposed in this paper generates better profits under different customer proportions.

[0083] Due to the overly large number of all samples (exceeding 300,000), in practical applications, we need to select a certain proportion as target customers for marketing activities. Top Lift selects a certain proportion of customers with the top-ranked classification confidence as evaluation objects, so it has more application value in customer churn prediction. It can be seen from Table 3 that the Top1Lift, Top5Lift, and Top10Lift of ProfGT are all better than the baseline models. This indicates that ProfGT is more competitive in customer churn prediction.

[0084] Table 3 Precision Comparison of Different Models

[0085] On the other hand, the present invention also discloses a computer-readable storage medium storing a computer program, which when executed by a processor causes the processor to execute the steps of the above method.

[0086] On yet another aspect, the present invention also discloses a computer device including a memory and a processor, where the memory stores a computer program, which when executed by the processor causes the processor to execute the steps of the above method.

[0087] In another embodiment provided by the present application, there is also provided a computer program product containing instructions, which when running on a computer causes the computer to execute any of the above-mentioned profit-oriented customer churn prediction methods based on graph transformers in the above embodiments.

[0088] It is understandable that the systems, devices, and storage media provided in the embodiments of the present invention correspond to the methods provided in the embodiments of the present invention. For the explanations, examples, and beneficial effects of the relevant content, reference can be made to the corresponding parts in the above methods.

[0089] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in the embodiments of the present application are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by wire (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more integrated available media. The available medium can be a magnetic medium (such as a floppy disk, hard disk, magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid-state disk (SSD)).

[0090] It should be noted that in the present invention, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including", or any other variant thereof is intended to cover non-exclusive inclusion, such that a process, method, article, or device that includes a series of elements includes not only those elements but also other elements that are not explicitly listed, or further includes elements that are inherent to such process, method, article, or device. Without further limitation, an element defined by the statement "including one..." does not exclude the existence of additional identical elements in the process, method, article, or device that includes the element.

[0091] Each embodiment in this specification is described in a related manner. For the same or similar parts among the embodiments, reference can be made to each other. Each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and for the relevant parts, reference can be made to the partial description of the method embodiment.

[0092] The above embodiments are only used to illustrate the technical solutions of the present invention, not to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A profit-oriented customer churn prediction method based on graph Transformer, characterized in that: The following steps are included: Use CDR data to construct a call graph that reflects customer communication behavior, and based on the characteristics of telecommunications services and the triadic closure theory in social networks, supplement the missing links in the graph to improve the accuracy of the graph representation; The ProfGT model is designed to take the feature matrix and graph of nodes as input and learn the representation of nodes from complex network relationships; The ProfGT model is trained with a profit-oriented loss function to integrate model prediction with customer churn management.

2. The profit-oriented customer churn prediction method based on graph Transformer according to claim 1 is characterized by: The specific steps to build a call graph that reflects customer communication behavior include: Define the customer call graph as , where the node set is , each node represents a customer, and each node has a feature of dimension, which is represented by the feature matrix of all nodes in the form of a matrix. is the number of nodes; Suppose the edge set in the graph is ; The call graph construction process is as follows: (1) If there is a call record between two customers, add an edge connecting them; (2) Link prediction: Fill in the edge based on the characteristics of telecommunications services. If the customers have the same primary card number and the same ID, add an edge to connect the two. According to the common neighbor algorithm, the number of common neighbors between nodes and is calculated, which is recorded as , where represents the neighborhood set of the node ;like , then add an edge.

3. The profit-oriented customer churn prediction method based on graph transformer according to claim 1, characterized in that: The ProfGT model construction process is as follows: First, position embedding is applied to generate the position encoding of each node. Then, the node features are fused with the position encoding and input into the three-layer ProfGT-Layer for processing. Finally, the processed node features are used for classification prediction. The ProfGT-Layer layer consists of global attention (GlobalAttn), message-passing graph neural networks (MPNNs) and MLP; the node feature matrix of the model input is defined as: , the adjacency matrix of the graph is ; Two types of position embeddings are applied: Laplacian Position Encoding (LapPE) and Random Walk Position Encoding (RWPE); The Laplacian matrix of a graph is given by the adjacency matrix Sum degree matrix Calculate, its eigenvalue and eigenvector are recorded as and ; The Laplacian matrix of the graph is defined as: Random Walk Position Encoding (RWPE) uses random walk operators To define Step random walk process, when When large enough, the node has a unique For a topological neighborhood, RWPE can produce a unique representation; node Definition of RWPE: Merge the node feature matrix and node position embedding to generate a new feature matrix, ; This feature matrix is ​​used as the input of the subsequent GlobalAttn module and MPNN module to further extract high-order features; Use Graph Transformers to implement the global attention mechanism; the calculation process of Graph Transformers is as follows: Input node feature matrix After three fully connected branches, , , Three matrices; the weights of the three fully connected branches are defined as , , , d and They are feature dimension and number of nodes respectively; and Matrix multiplication gives The weight matrix of Matrix multiplication, output node's global attention feature matrix ; GraphSAGE is used as a message passing graph neural network (MPNN); Formula (5) shows the process of aggregating node neighbor node information to generate aggregate features, and Formula (6) describes the process of updating node information with aggregate features; Indicates Layer Node Aggregate features of all neighbors, Represents an aggregate function, Representation Node Neighbor node set; Representation Node exist The characteristics of the layer, Represents the activation function Relu, Represents the learned weight matrix; merges the aggregated features and Layer features, calculated by weight matrix, output node New features of Update the features of all nodes in turn to get The feature matrix, ; The feature matrix and The feature matrices are added to fuse the local embedding information and global embedding information of the nodes; finally, the MLP is passed to output the The node feature matrix of the layer; The MLP module contains two fully connected modules, and the weight matrices are , .

4. The profit-oriented customer churn prediction method based on graph transformer according to claim 3 is characterized by: The profit-oriented loss function is as follows: According to the profit calculation formula of customer churn management activities, see formula (9), a profit-oriented loss function is designed to guide the learning of the model; the profit calculation formula of customer churn management activities is: in is the profit generated by customer retention activities, is the total number of customers, Represents the proportion of target customers, is the proportion of real lost customers in customer retention activities, When a customer accepts and stays, the company's incentive cost, The proportion of potential target customers who decide to stay due to incentives is the success rate of incentives. The cost of contacting a customer to provide an incentive, the customer lifetime value ( ) refers to the average customer lifetime value of retained customers, Refers to the fixed costs of running a churn management program; Formula (9) is modified into a formula that can be used for model training and evaluation, and is simplified: in, Represents the set of nodes that are actually lost and predicted by the model to be lost. represents the set of nodes that are actually non-churned but predicted by the model to be churned, and ; Represents the cost of retaining a customer; The profit-loss function consists of two parts: revenue and cost; For the benefit part, all the lost samples are used as supervision, and , the predicted probability of each churn sample is multiplied by the CLV, which ensures the continuity of the loss function on the one hand, and on the other hand, the higher the CLV, the higher the weight of the sample, making the model pay more attention to high-value samples; The cost part prevents the model from being overconfident and predicting non-churned customers as churned customers; The loss function is designed as formula (11): In the revenue section, is the actual number of lost customers, The score of the node that predicts the churned customer for the model, is the customer lifetime value; when the model outputs a lost customer score approaching 1, the profit loss is 0, and the profit is maximized; For the cost part, select the predicted score greater than or equal to The node is used as a penalty term, expressed as , Representation cost; Combining the cross entropy loss and profit loss, the following loss function is finally obtained as shown in formula (12): is the cross entropy loss function, where It is an adjustment factor to weigh the impact of forecast accuracy and profit.

5. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the processor is caused to perform the steps of the method according to any one of claims 1 to 4.

6. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the computer program is executed by the processor, the processor is caused to perform the steps of the method according to any one of claims 1 to 4.

Images