Data identification method based on multi-view learning

Through the multi-view learning method, the joint decision-making module of MvESR network and adaptive weight loss fusion is used to solve the problem of insufficient information redundancy and representation capabilities in telecom customer churn prediction, improve the recognition accuracy and representation capabilities, and achieve more detailed feature extraction and decision-making judgment.

CN119026008APending Publication Date: 2024-11-26HEFEI UNIV OF TECH
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202411264311.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-09-10
Publication Date
2024-11-26

AI Technical Summary

Technical Problem

The prior art has problems such as redundancy in information, insufficient characterization capabilities, inaccurate feature extraction and lack of joint decision-making mechanisms when dealing with telecom customer churn prediction, especially in multi-view data processing.

Method used

Using a multi-view learning method, complementary information in multi-view information is extracted through the MvESR network, numerical embedding and multi-head attention mechanisms capture the connection and weight allocation between features in the view, and combined with joint decision classification of adaptive weight loss fusion Methods, construct joint decision modules to improve identification accuracy.

Benefits of technology

It significantly improves the accuracy of telecom customer churn identification, reduces the complexity of model processing, enhances the representation ability of the model, accurately extracts the key features and their interaction relationships of each view, avoids the misleading of single view information, and forms More comprehensive and accurate judgment.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119026008A_ABST
    Figure CN119026008A_ABST
Patent Text Reader

Abstract

The invention provides a data identification method based on multi-view learning, and relates to the technical field of data identification. According to the recognition method, the interaction relation of multi-view information is fully considered, key features are extracted, and a joint decision-making mechanism is constructed, so that the recognition accuracy is improved; by constructing a view encoder and a multi-view information interaction network, redundant information among all views is effectively removed; the relation and weight distribution among features in the views are captured by using numerical value embedding and a multi-head attention mechanism, complementarity and consistency information among the views is further mined by means of a multi-view information interaction network, and the representation capability of the model is remarkably enhanced; by constructing a complex network structure, key features of all views and interaction relationships among the views are accurately extracted; due to the design of the joint decision module, the model can comprehensively consider the information of each view and the interaction relationship thereof, and more comprehensive and accurate judgment is formed; and the accuracy of telecommunication customer loss identification is ensured.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of data recognition, and in particular to a data recognition method based on multi-view learning. Background Art

[0002] With the rapid development of the telecommunications industry and the intensification of market competition, telecommunications customer churn has become a focus of attention for operators. Predicting and preventing customer churn is crucial for telecommunications companies to maintain market competitiveness and improve customer satisfaction. Studies have shown that for every 1% of customer churn, profits will decrease by 5%-16%, and the cost of developing new customers is 5 to 10 times that of retaining existing customers. Therefore, accurately identifying customers and preventing customer churn is an urgent problem for operators to solve.

[0003] In recent years, academia and industry have conducted a lot of research in the field of telecommunications customer churn prediction and proposed a variety of prediction models and methods.

[0004] Zhang Sanniu et al. (2023) studied the application of survival analysis models in telecom customer churn prediction. The Kaplan-Meier (KM) model and the Cox proportional hazard model can effectively predict the time point and influencing factors of customer churn. The KM model estimates the customer survival probability in a parameter-free manner and is suitable for inter-group comparisons; the Cox model further considers the influence of multiple factors and is suitable for complex multivariate analysis. These models not only improve the accuracy of predictions, but also provide telecom companies with a strategic basis for retaining customers.

[0005] Liang Shuang (2023) studied the application of machine learning in telecommunications customer churn prediction. By learning the relationship between features and customer churn from historical data, a high prediction accuracy was achieved. These methods perform well in processing large-scale, high-dimensional data and can capture complex nonlinear relationships.

[0006] Huang Zixuan (2023) used attention mechanism and graph convolutional network to deal with the problem of telecommunications customer churn prediction. These methods can automatically extract features and capture complex temporal and spatial relationships, further improving the accuracy of prediction. In addition, some studies have attempted to combine data mining technology with specific algorithms. For example, Zhang Juanjuan (2023) achieved good prediction results by preprocessing data through data mining and integrating multiple classifiers through Stacking.

[0007] Chen Li (2023) proposed a prediction model based on the CW-Stacking ensemble algorithm and optimized the algorithm based on the characteristics of telecommunications customer data. Li Yi (2023) combined cluster random sampling and cost-sensitive learning to improve the prediction performance on imbalanced data sets. These studies have improved and optimized the algorithms for specific problems in the telecommunications industry, further improving the prediction results.

[0008] Although multi-view data provides a more comprehensive information basis for telecommunication customer churn identification, existing methods still have the following problems when processing multi-view data:

[0009] Information redundancy problem: There may be a large amount of duplicate information between views, which not only increases the complexity of model processing, but also may have a negative impact on the recognition effect of the model.

[0010] Insufficient representation ability: The information complementarity between different views has not been fully utilized, resulting in limited representation ability of the model and difficulty in fully capturing the complex factors that affect financial fraud.

[0011] Inaccurate feature extraction: Existing methods often use simple feature splicing or fusion methods when processing multi-view data, which makes it difficult to accurately extract the key features of each view and the interactions between them.

[0012] Lack of joint decision-making mechanism: The information between views fails to form an effective joint decision-making mechanism during the recognition process, resulting in the recognition results of the model being misled by the information of a single view. Summary of the invention

[0013] In view of the shortcomings of the prior art, the present invention provides a data recognition method based on multi-view learning, which solves the problems of information redundancy, insufficient representation ability, inaccurate feature extraction and lack of joint decision-making mechanism in multi-view data processing.

[0014] To achieve the above objectives, the present invention is implemented through the following technical solutions:

[0015] A data recognition method based on multi-view learning, the recognition method comprises the following steps:

[0016] S1. Collect multi-view feature data of telecom customers, and perform data cleaning, denoising, standardization, and missing value filling operations;

[0017] S2. Divide user features into user features U, service features S, contract features C, and cost features E. The four views of USCE are all numerical features. The numerical embedding method is used to make low-dimensional numerical features play their value through high-dimensional transformation under the deep learning framework;

[0018] S3. In order to extract complementary information from multi-view information, the representation codes of the four views of USCE are input into the MvESR network, which includes a consistency network, a complementarity network, and a joint decision module. In the joint decision process, the MvESR network includes the independent view information of each enhancement, the interactive information of each view, and the summary information of each enhancement.

[0019] S4. Based on the extraction of the above interactive information, taking into account the different contributions of different information to classification, a joint decision-making classification method with adaptive weight loss fusion is adopted.

[0020] Preferably, in S1, for the classification column, one-hot encoding is used, and the lost customers are encoded as 1, and the non-lost customers are encoded as 0;

[0021] For numerical features, the k nearest neighbor method is used to fill missing values. The implementation steps are as follows:

[0022] Select neighbors: For each missing value in the dataset, determine a fixed number of nearest neighbor samples that are closest to the sample where the missing value is located in the feature space; this number is defined by the parameter n_neighbors;

[0023] Distance metric: Use appropriate distance functions to quantify the similarity between data points. Commonly used distance metrics include Euclidean distance and Manhattan distance. For data points with missing values, the distance calculation needs to be appropriately adjusted to ensure that it is based only on non-missing features.

[0024] Weight distribution: weight distribution is performed according to the distance between samples. Samples with closer distance have higher weight in missing value estimation. The weight is uniform or proportional to the inverse of the distance.

[0025] Missing value estimation: For each missing feature, the missing value estimate is calculated based on the selected neighbor samples and their corresponding weights; the estimation is achieved by simple arithmetic mean or weighted average; the missing value estimate can be expressed by the following formula:

[0026]

[0027] Among them, w i Represents the weight of the i-th neighbor sample;

[0028] v i Represents the value of the i-th neighbor sample on the missing feature;

[0029] k is the number of neighbors specified by the parameter n_neighbors.

[0030] Preferably, in S2, the numerical features of the four views of USCE are converted into vector embeddings by a numerical embedding module, and then the vector embeddings are processed by a Transformer module to finally obtain a high-dimensional vector representation; the specific implementation process of a single view is shown in formulas (1)-(6):

[0031] T j =b j +f j (x j0∈R d f j :X j →R d #(1)

[0032]

[0033] Among them, b j is the bias term of the jth feature;

[0034] f j (·) refers to the original eigenvector and W j ∈R d Perform element-wise multiplication;

[0035] is the one-hot encoding of the corresponding classification feature. This paper is a binary classification task, where fraudulent enterprises are 1 and non-fraudulent enterprises are 0;

[0036] The numerical features of a single view are and the corresponding classification features Perform high-dimensional mapping. During this process, the internal features of the view will be given different weights, and the information will interact with each other. During the training process, the weight matrix Will continue to iterate; after high-dimensional mapping Concatenate into T, which contains both numerical and categorical feature information;

[0037] T0=stack[[CLS],T]#(5)

[0038] T i =F i (T i-1 )#(6)

[0039] Use Transformer to merge [CLS] with T, and pass through L layers F i (·) Transform to get T L Finally, we get As a downstream task; It is expressed as an aggregated feature information representation, which contains not only the original feature information but also the relationship between features.

[0040] Preferably, in S2, in order to remove redundant information between features in a view, a multi-head attention mechanism is used to adaptively encode features in each view; the attention function maps a query and a set of key-value pairs to an output, wherein the query, key, value and output are all vectors, the output is a weighted sum of values, and each weight is calculated from the query and the key; for each feature, the multi-head attention assigns a trainable query vector Q to it, aiming to extract the correlation between the features; the query vector Q is initially random and then optimized during the training process;

[0041] This paper embeds the output of the numerical embedding layer As the query vector Q∈R k×d , key vector K∈R k×d , value vector V∈R k×d The initial input focuses on the contribution of financial fraud features within each view;

[0042]

[0043] The multi-head attention mechanism consists of multiple attention modules, each of which has its own independent linear transformation of query, key and value, and then concatenates the outputs of multiple heads and performs a final linear transformation; multi-head attention allows the model to jointly focus on different subspace information;

[0044] MultiHead(Q,K,V)=Concat(head1,...,head h )W O #(8)

[0045] head i =Attention(QW i Q ,KW i K ,VW i v )#(9)

[0046]

[0047] is a parameter matrix, which is continuously optimized during the training process. This paper uses 8 attention heads, i.e., h = 8. In each attention head, there are d K =d V =d model / h=8; M=MultiHead(Q,K,V)∈R k×d , where the initial The numerical embedding vectors of the four views of USCE are input into four different multi-head attention modules respectively, and the representation codes MG, MO, MN, and ME of the four views are output.

[0048] Preferably, in S3, in order to extract consistency information between different views, an alignment strategy based on correlation, i.e., interactive calculation, is used; based on the multi-view vector representation of the coding layer, the input vector M∈R k×d The dimension reduction is represented as y∈R D , where D = kd; the interaction calculation of each view is as follows:

[0049]

[0050] Among them, v≠u;

[0051] Represents batch matrix multiplication;

[0052] ψ v,u (·) is the input dimension D 2 , a two-layer neural network with an output dimension of D;

[0053] Used to Mapping to space R D The consistency matrix of all interaction views is expressed as follows, where V represents the number of views;

[0054]

[0055] For a certain view, it interacts with other views to utilize enhanced view-specific information; it is spliced ​​with other views, and the spliced ​​view is fused with the original view to obtain complementary information;

[0056]

[0057] Among them, f concate (·) represents the view stitching function;

[0058] is enhanced view information;

[0059] ⊙ represents element addition;

[0060] χ v (·) represents a neural network with D-dimensional output.

[0061] Preferably, in S4, the decision information set includes Figure 1 Information Y b And the complementary information of V views It is expressed as follows:

[0062]

[0063] Construct a three-layer fully connected classification network with input nodes D m Represents Y mFeature dimension, the output is the node is the classification type C, and the classification of each information is calculated;

[0064]

[0065] In order to make joint decisions using information from each view, a fusion loss strategy for multi-view classification is adopted. The formula is as follows:

[0066]

[0067] Among them, α m is the weight of the mth view;

[0068] l i is the one-hot encoding of the sample label;

[0069] L m (·) is the cross entropy loss. Different weights are adaptively assigned to each view during the training process, and the information learned by each view complements other views.

[0070] The present invention provides a data recognition method based on multi-view learning. Compared with the prior art, it has the following beneficial effects:

[0071] In the present invention, the recognition method improves the recognition accuracy by fully considering the interactive relationship of multi-view information, extracting key features, and constructing a joint decision-making mechanism; by constructing a view encoder and a multi-view information interaction network, the redundant information between views is effectively removed, and the complexity of model processing is reduced; by utilizing numerical embedding and a multi-head attention mechanism to capture the connection and weight distribution between features in the view, and with the help of a multi-view information interaction network, the complementarity and consistency information between views are further mined, and the representation ability of the model is significantly enhanced; by constructing a complex network structure, the key features of each view and the interactive relationship between them are accurately extracted, providing a more refined input for fraud identification; the design of the joint decision-making module enables the model to comprehensively consider the information of each view and its interactive relationship, form a more comprehensive and accurate judgment, and avoid the misleading that may be caused by a single view information; and ensure the accuracy of telecommunications customer churn identification. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0073] Figure 1 is a flow chart of an identification method in an embodiment of the present invention;

[0074] Figure 2 Schematic diagram of view weights for each training round of the TCCIMVL model in an embodiment of the present invention. DETAILED DESCRIPTION

[0075] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention are clearly and completely described. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0076] The embodiments of the present application provide a data recognition method based on multi-view learning, which solves the problems of information redundancy, insufficient representation capability, inaccurate feature extraction, and lack of a joint decision-making mechanism in multi-view data processing.

[0077] The technical solution in the embodiment of the present application is to solve the above technical problems, and the overall idea is as follows:

[0078] 1. In the embodiment of the present invention, the interactive relationship of the numerical information in the view is considered at the encoding layer, the numerical features are represented by high-dimensional vectorization, and the connection between the features in the view is captured by numerical embedding. Taking into account the different contribution levels of the features, the multi-head attention mechanism is used to adaptively give different weights to the degree of interaction between the features in the view.

[0079] 2. In the embodiment of the present invention, the encoded high-dimensional features are respectively input into the multi-view interaction network, including the complementary network and the consistency network. In the complementary network, the vectors of the four views are spliced ​​together, and then enter different multi-layer perceptron (MLP) layers to learn the deep interaction information of each view, and finally merged with the input vector of each view to obtain the complementary representation of each view.

[0080] 3. In the embodiment of the present invention, in the multi-view network, the vectors of the four views are interacted with each other, the consistency information between the views is learned, and the consistency information is concatenated and represented as a feature vector, which is used together with the complementary information of each view as incremental information and the sub-features of the original view to make the final joint decision. The weight of the important view in the decision can be adaptively assigned to a higher level, thereby improving the recognition accuracy of the model.

[0081] In order to better understand the above technical solution, the above technical solution will be described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0082] Example:

[0083] like Figure 1As shown, the present invention provides a data recognition method based on multi-view learning, and the recognition method comprises the following steps:

[0084] S1. Collect multi-view feature data of telecom customers, including user view, service view, contract view, and fee view; perform pre-processing operations such as cleaning, denoising, standardization, and missing value filling on the data to ensure data quality and consistency.

[0085] For categorical columns, one-hot encoding is used to encode lost customers as 1 and non-lost customers as 0.

[0086] For numerical features, the k nearest neighbor method is used to fill missing values. The implementation steps are as follows:

[0087] Select neighbors: For each missing value in the dataset, determine a fixed number of nearest neighbor samples that are closest to the sample with the missing value in feature space. This number is defined by the parameter n_neighbors.

[0088] Distance metric: Use an appropriate distance function to quantify the similarity between data points. Commonly used distance metrics include Euclidean distance, Manhattan distance, etc. For data points with missing values, the distance calculation needs to be appropriately adjusted to ensure that it is based only on non-missing features.

[0089] Weight assignment: Optionally, weights are assigned based on the distance between samples, with closer samples having higher weights in missing value estimation. Weights can be uniform (weights = 'uniform') or based on the inverse of the distance (weights = 'distance').

[0090] Missing value estimation: For each missing feature, an estimate of the missing value is calculated based on the selected neighbor samples and their corresponding weights. The estimation can be achieved by a simple arithmetic mean or a weighted average. The estimation of missing values ​​can be expressed by the following formula:

[0091]

[0092] Among them, w i Represents the weight of the i-th neighbor sample;

[0093] v i Represents the value of the i-th neighbor sample on the missing feature;

[0094] k is the number of neighbors specified by the parameter n_neighbors.

[0095] S2. Multi-view encoder construction

[0096] According to industry standards, user features are divided into user features (user), service features (serve), contract features (contract), and expense features (expense). The four views of USCE are all numerical features. In order to extract the potential connections between information and facilitate the capture of information by the downstream multi-view learning framework, the numerical embedding (Feature Tokenizer+Transformer, FTT) method is used to allow low-dimensional numerical features to be transformed into high dimensions and to play their value under the deep learning framework.

[0097] The numerical features of the four views of USCE are converted into vector embeddings through the numerical embedding module, and then the vector embeddings are processed by the Transformer module to finally obtain a high-dimensional vector representation. The specific implementation process of a single view is shown in formulas (1)-(6).

[0098] T j =b j +f j (x j )∈R d f j :X j →R d #(1)

[0099]

[0100] Among them, b j is the bias term of the jth feature;

[0101] f j (·) refers to the original eigenvector and W j ∈R d Perform element-wise multiplication;

[0102] It is the one-hot encoding of the corresponding classification feature. This article is a binary classification task, where fraudulent enterprises are 1 and non-fraudulent enterprises are 0.

[0103] The numerical features of a single view are and the corresponding classification features Perform high-dimensional mapping. During this process, the internal features of the view will be given different weights, and the information will interact with each other. During the training process, the weight matrix It will continue to iterate. and Concatenated into T, T now contains both numerical and categorical feature information.

[0104] T o =stack[[CLS],T]#(5)

[0105] T i =F i (T i-1 )#(6)

[0106] Use Transformer to merge [CLS] with T, and pass through L layers F i (·) Transform to get T L Finally, we get As a downstream task. It is expressed as an aggregated feature information representation, which contains not only the original feature information but also the relationship between features.

[0107] In order to remove redundant information between features in a view, a multi-head attention mechanism is used to adaptively encode features in each view. The attention function maps a query and a set of key-value pairs to an output, where the query, key, value, and output are all vectors, and the output is a weighted sum of values, with each weight calculated from the query and key. For each feature, multi-head attention assigns it a trainable query vector Q, which aims to extract the correlation between features. The query vector Q is initially random and then optimized during training.

[0108] This paper embeds the output of the numerical embedding layer As the query vector Q∈R k×d , key vector K∈R k×d , value vector V∈R k×d The initial input focuses on the contribution of financial fraud features within each view.

[0109]

[0110] The multi-head attention mechanism consists of multiple attention modules, each of which has its own independent linear transformation of query, key and value, and then concatenates the outputs of multiple heads and performs a final linear transformation. Multi-head attention allows the model to jointly focus on different subspace information.

[0111] MultiHead(Q,K,V)=Concat(head1,...,head h )W O #(8)

[0112] head i =Attention(QW i Q ,KW i K ,VW i V )#(9)

[0113]

[0114] is a parameter matrix that is continuously optimized during the training process. This paper uses 8 attention heads, i.e. h = 8. In each attention head, there are d K =d V =d model / h=8. M=MultiHead(Q,K,V)∈R k×d , where the initial The numerical embedding vectors of the four views of USCE are input into four different multi-head attention modules respectively, and the representation codes MG, MO, MN, and ME of the four views are output.

[0115] S3. Multi-view interaction module considering consistency and complementarity

[0116] In order to extract complementary information from multi-view information, the representation encoding of the four views of USCE is input into the MvESR network, which includes a consistency network, a complementarity network, and a joint decision module. In the joint decision process, the MvESR network includes the independent view information of each enhancement, the interactive information of each view, and the summary information of each enhancement.

[0117] In order to extract the consistency information between different views, a correlation-based alignment strategy, namely interactive calculation, is used. Based on the multi-view vector representation of the encoding layer, the input vector M∈R k×d The dimension reduction is represented as y∈R D , where D = kd. The interaction calculation of each view is as follows:

[0118]

[0119] Among them, v≠u;

[0120] Represents batch matrix multiplication;

[0121] ψ v,u (·) is the input dimension D 2 , a two-layer neural network with an output dimension of D;

[0122] Used to Mapping to space R D The consistency matrix of all interaction views is expressed as follows, where V represents the number of views.

[0123]

[0124] For a certain view, it interacts with other views to utilize enhanced view-specific information. By splicing with other views, the spliced ​​view is fused with the original view to obtain complementary information.

[0125]

[0126] Among them, f concate (·) represents the view stitching function;

[0127] is enhanced view information;

[0128] ⊙ represents element addition;

[0129] χ v (·) represents a neural network with D-dimensional output.

[0130] S4. Joint decision module considering view weights

[0131] Based on the extraction of the above interactive information, considering the different contributions of different information to classification, a joint decision classification method with adaptive weight loss fusion is adopted. Figure 1 Information Y b And the complementary information of V views It is expressed as follows:

[0132]

[0133] Construct a three-layer fully connected classification network with input nodes D m Represents Y m Feature dimension, the output is the node is the classification category C, and the classification of each information is calculated.

[0134]

[0135] In order to make joint decisions using information from each view, a fusion loss strategy for multi-view classification is adopted. The formula is as follows:

[0136]

[0137] Among them, α m is the weight of the mth view;

[0138] l i is the one-hot encoding of the sample label;

[0139] L m (·) is the cross entropy loss. Different weights are adaptively assigned to each view during the training process, and the information learned by each view complements other views.

[0140] In order to verify the effectiveness of the method proposed in this paper, this paper selects a public telecommunications customer data set to prove the effectiveness of the model. The data set has 21 fields and a total of 7043 records. Each record contains the characteristics of a unique customer. There are 1869 lost customers, accounting for 26.5% of the total sample, and 5174 retained customers, accounting for 73.4% of the total sample. The sample data is unbalanced.

[0141] 1. Comparison Model

[0142] This paper uses three machine learning methods and four multi-view learning methods as comparison methods. The machine learning methods are:

[0143] 1) LR (Logistic Regression): Logistic regression is a statistical learning algorithm for binary classification problems. It determines the model parameters through maximum likelihood estimation and maps the linear regression output to a probability value. It uses binary cross entropy loss as the loss function and optimizes the model through gradient descent.

[0144] 2) SVM (Support Vector Machines): The core idea of ​​SVM is to find an optimal hyperplane in the feature space to maximize the interval between data points, and to maximize the soft interval by introducing slack variables, thereby improving the generalization ability of the model. SVM is particularly suitable for processing high-dimensional data sets and can effectively handle nonlinear problems through kernel techniques.

[0145] 3) RF (Random Forest): RF is an integrated learning framework that builds a classifier or regression model composed of multiple decision trees, and uses the ensemble effect of decision trees to enhance the predictive performance and robustness of the overall model. Through the bootstrap aggregating technique, multiple resampled subsets are generated for the original training set, and decision trees are trained independently on these subsets. Random feature selection is introduced in the construction process of each tree to increase the diversity of the model and reduce overfitting.

[0146] Multi-view learning is:

[0147] 1) DNN-Con (Concatenation + DNN), this method concatenates all views together and then uses DNN for classification.

[0148] 2) MvDML (sharable and individual multi-view deep metric learning), which not only extracts unique information from each view, but also builds a common network to extract consistent information from different views

[0149] 3) MvDN (multi-view deep network), this method constructs view-specific sub-networks and shared sub-networks to eliminate complex differences between views.

[0150] 4) MvESR (Learning Enhanced Specific Representations for Multi-view Feature Learning), this method learns enhanced view information through interaction between views.

[0151] 2. Evaluation Indicators

[0152] This article uses the area under the ROC curve (AUC) and Kolmogorov-Smirnov (KS) as model evaluation indicators, where TP (True Positive) indicates that the sample is actually a positive sample, and the predicted result is also a positive sample. FN (False Negative) indicates that the actual value of the sample is a positive sample, and the predicted result is a negative sample. FP (False Positive) indicates that the actual value of the sample is a negative sample, and the predicted result is a positive sample. TN (True Negative) indicates that the actual value is a negative sample, and the predicted result is also a negative sample.

[0153] The AUC indicator reflects the comprehensive discrimination ability of the model for lost customers and normal customers. The area under the curve (AUC) is the area under the ROC curve. The ROC curve is drawn with the true positive rate (TPR) and the false positive rate (FPR) as the horizontal and vertical axes. The calculation formulas of TPR and FPR are as follows:

[0154]

[0155] The KS indicator is the maximum difference between the cumulative distribution of lost customers and normal customers predicted by the model, reflecting the model's ability to distinguish between the two types of samples. For a binary classification problem, its calculation formula can be expressed as:

[0156] KS=MAX x |F1(X)-F2(X)|

[0157] in:

[0158] F1(X) is the cumulative distribution function (CDF) of the positive cases (churned customers).

[0159] F2(X) is the cumulative distribution function (CDF) of negative examples (retained customers).

[0160] x is the probability or score predicted by the model.

[0161] These two indicators can accurately evaluate the performance of the recognition method. The higher the value, the more consistent the recognition result is with the actual situation, and the better the performance of the classification method.

[0162] 3. Model parameters

[0163] The parameter settings of this model are shown in Table 1:

[0164] Table 1 Experimental parameter settings

[0165]

[0166] IV. Experimental Results

[0167] The effect and stability of the model were verified by ten 10-fold cross-validation. Ten 10-fold cross-validation is a model evaluation method widely used in machine learning and deep learning to evaluate the generalization ability of the model.

[0168] Dataset segmentation: First, the dataset is randomly divided into 10 parts of equal size. The amount of data in each part is 10% of the total data.

[0169] Cycle validation: Then perform 10 independent cycle validations. In each cycle: select one of the parts as the test set (validation set). Combine the remaining 9 parts as the training set (training set).

[0170] Model training and evaluation: In each cycle: Use the training set to train the model. Use the test set to evaluate the performance of the model.

[0171] Performance recording: Record the performance indicators, AUC and KS values ​​of the model in each cycle

[0172] Performance evaluation: After 10 cycles, the mean and standard deviation of the model performance indicators are calculated. The mean provides an estimate of the generalization ability of the model, and the standard deviation provides an estimate of the stability ability of the model.

[0173] Here are the steps for ten fold cross validation:

[0174] The mean and standard deviation of the experimental results are shown in Table 2:

[0175] Table 2 Experimental results

[0176]

[0177] The results show that the proposed method is significantly better than other methods. In real data scenarios, financial fraud companies can be identified more accurately. LR and SVM are relatively poor, and the potential reason is that the features are larger than the sample size. General multi-view learning methods are better than LR and SVM, but ensemble learning methods XGB and RF are still better than general multi-view learning methods in this experimental scenario.

[0178] V. Ablation Experiment

[0179] In order to further prove the effectiveness of each module, ablation experiments are conducted. Table 3 shows the details of the ablation experiments. The ablation experiments are as follows: (1) MH-Attention+MvESR: remove the numerical embedding module based on the complete model; (2) FTT+MvESR removes the multi-head attention module based on the complete model; (3) FTT+MH-Attention removes the multi-view classification network based on the complete model.

[0180] Table 3 Ablation experiment results

[0181]

[0182] The experimental results show that the three core modules of the TCCIMVL model play an indispensable role in the model. When the numerical embedding module is removed from the model, the overall AUC value decreases by 5.51% and the KS value decreases by 5.95%, indicating that the numerical embedding captures the connection between features within the view, which is conducive to the subsequent multi-view classification network to extract important interactive information. When the multi-head attention module is removed from the model, the overall AUC value decreases by 2.66% and the KS value decreases by 5.73%, indicating that the multi-head attention mechanism can adaptively assign different weights to different features, which is conducive to finding complementary information between features in each view and enhancing the complementarity within the view. When the multi-view learning network is removed from the model, the overall AUC value decreases by 4.71% and the KS value decreases by 8.92%, indicating that the multi-view network effectively captures the interactive information between views, and it is effective to use the consistency information and complementary information extracted by the multi-view network as incremental information and make joint decisions.

[0183] VI. Explanatory Analysis

[0184] By visualizing the weight of each view information in the joint decision, the contribution of each view information can be intuitively measured. The weight of each view in each epoch is assigned a different color based on the value. The larger the value, the darker the color. Figure 1 Represents user characteristics (user), Figure 2 The service characteristics (serve), view 3 represents the contract characteristics (contract), and view 4 represents the expense characteristics (expense). Figure 2 shown.

[0185] From the results, we can see that the complementary feature view S and the consistent feature view C have similar colors to the sub-views of USCE, indicating that their contribution to the joint decision is comparable to that of each sub-view. The color of the C view is obviously darker, indicating that the importance of the C view that integrates the mutual information is significantly better than other single views.

[0186] The telecommunication customer churn identification method based on multi-view learning proposed in this paper has the following advantages and benefits:

[0187] 1) Improve recognition accuracy: By fully considering the interactive relationship of multi-view information, extracting key features, and building a joint decision-making mechanism, the method proposed in this paper shows a higher accuracy in identifying telecom customer churn. Experimental results show that this method is significantly better than the traditional single-view recognition method.

[0188] 2) Reduce information redundancy: By constructing a view encoder and a multi-view information interaction network, the method proposed in this paper can effectively remove redundant information between views and reduce the complexity of model processing.

[0189] 3) Enhanced representation capability: By using numerical embedding and multi-head attention mechanism to capture the connection and weight distribution between features within the view, the method proposed in this paper can more comprehensively represent the information of each view. At the same time, with the help of multi-view information interaction network, the complementarity and consistency information between views are further mined, which significantly enhances the representation capability of the model.

[0190] 4) Accurate feature extraction: Different from simple feature splicing or fusion methods, this method constructs a complex network structure to accurately extract the key features of each view and their interactions, providing more refined input for fraud identification.

[0191] 5) Flexible joint decision-making: The design of the joint decision-making module enables the model to comprehensively consider the information of each view and their interactive relationship to form a more comprehensive and accurate judgment, avoiding the misleading information from a single view.

[0192] 6) Enhanced explainability: Since the proposed method is based on the USCE theoretical framework, each view and its interactions have clear economic and managerial significance, making the decision-making process of the model more transparent and explainable, which helps regulators and investors understand the logic and basis of fraud identification.

[0193] 7) Broad application prospects: The method proposed in this paper is not only suitable for telecommunications customer churn identification, but can also be extended to multi-view data analysis problems in other fields, and has broad application prospects.

[0194] In summary, the multi-view learning-based method for identifying listed telecom customer churn has shown significant advantages in terms of recognition accuracy, information redundancy processing, representation ability, feature extraction, joint decision-making, interpretability, and application prospects. The successful application of this method will bring revolutionary changes to the field of telecom customer churn identification.

[0195] In summary, compared with the prior art, the present invention has the following beneficial effects:

[0196] In the embodiment of the present invention, the recognition method improves the recognition accuracy by fully considering the interactive relationship of multi-view information, extracting key features, and constructing a joint decision-making mechanism; by constructing a view encoder and a multi-view information interaction network, the redundant information between views is effectively removed, and the complexity of model processing is reduced; by using numerical embedding and a multi-head attention mechanism to capture the connection and weight distribution between features in the view, the complementarity and consistency information between views are further mined with the help of the multi-view information interaction network, and the representation ability of the model is significantly enhanced; by constructing a complex network structure, the key features of each view and the interactive relationship between them are accurately extracted, providing a more refined input for fraud identification; the design of the joint decision-making module enables the model to comprehensively consider the information of each view and its interactive relationship, form a more comprehensive and accurate judgment, and avoid the misleading that may be caused by a single view information; and ensure the accuracy of telecommunications customer churn identification.

[0197] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0198] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A data recognition method based on multi-view learning, characterized in that: The identification method comprises the following steps: S1. Collect multi-view feature data of telecom customers, and perform data cleaning, denoising, standardization, and missing value filling operations; S2. Divide user features into user features U, service features S, contract features C, and cost features E. The four views of USCE are all numerical features. The numerical embedding method is used to make low-dimensional numerical features play their value through high-dimensional transformation under the deep learning framework; S3. In order to extract complementary information from multi-view information, the representation encoding of the four views of USCE is input into the MvESR network, which includes a consistency network, a complementary network and a joint decision module; In the joint decision-making process, the MvESR network includes the independent view information of each enhancement, the interactive information of each view, and the summary information of each enhancement; S4. Based on the extraction of the above interactive information, taking into account the different contributions of different information to classification, a joint decision-making classification method with adaptive weight loss fusion is adopted.

2. The data recognition method based on multi-view learning according to claim 1, characterized in that: In S1, for the classification column, one-hot encoding is used to encode the lost customers as 1 and the non-lost customers as 0; For numerical features, the k nearest neighbor method is used to fill missing values. The implementation steps are as follows: Select neighbors: For each missing value in the data set, determine a fixed number of nearest neighbor samples that are closest to the sample where the missing value is located in the feature space; This number is defined by the parameter n_neighbors; Distance metric: Use appropriate distance functions to quantify the similarity between data points. Commonly used distance metrics include Euclidean distance and Manhattan distance. For data points with missing values, the distance calculation needs to be appropriately adjusted to ensure that it is based only on non-missing features. Weight allocation: weight allocation is performed based on the distance between samples. Samples with closer distances have higher weights in missing value estimation. The weights are either uniform or proportional to the inverse of the distance; Missing value estimation: For each missing feature, the missing value estimate is calculated based on the selected neighbor samples and their corresponding weights; the estimation is achieved by simple arithmetic mean or weighted average; the missing value estimate can be expressed by the following formula: Among them, w i Represents the weight of the i-th neighbor sample; v i Represents the value of the i-th neighbor sample on the missing feature; k is the number of neighbors specified by the parameter n_neighbors.

3. The data recognition method based on multi-view learning according to claim 1, characterized in that: In S2, the numerical features of the four views of USCE are converted into vector embeddings through the numerical embedding module, and then the vector embeddings are processed by the Transformer module to finally obtain a high-dimensional vector representation; the specific implementation process of a single view is shown in formulas (1)-(6): T j =b j +f j (x j )∈R d f j ∶X j →R d #(1) Among them, b j is the bias term of the jth feature; f j (·) refers to the original eigenvector and W j ∈R d Perform element-wise multiplication; is the one-hot encoding of the corresponding classification feature. This paper is a binary classification task, where fraudulent enterprises are 1 and non-fraudulent enterprises are 0; The numerical features of a single view are and the corresponding classification features Perform high-dimensional mapping. During this process, the internal features of the view will be given different weights, and the information will interact with each other. During the training process, the weight matrix W j (num) , W j (cat) It will continue to iterate; the high-dimensional mapped T j (num) With T j (cat) Concatenate into T, which contains both numerical and categorical feature information; T0=stack[[CLS],T]#(5) T i =F i (T i-1 )#(6) Use Transformer to merge [CLS] with T, and pass through L layers F i (·) Transform to get T L Finally, we get As a downstream task; It is expressed as an aggregated feature information representation, which contains not only the original feature information but also the relationship between features.

4. The data recognition method based on multi-view learning according to claim 1, characterized in that: In S2, in order to remove redundant information between features in the view, a multi-head attention mechanism is used to adaptively encode features in each view; The attention function maps a query and a set of key-value pairs to an output, where the query, key, value, and output are all vectors, and the output is a weighted sum of the values, with each weight calculated from the query and the key. For each feature, multi-head attention assigns it a trainable query vector Q, which aims to extract the correlation between features. The query vector Q is initially random and then optimized during training; This paper embeds the output of the numerical embedding layer As the query vector Q∈R k×d , key vector K∈R k×d , value vector V∈R k×d The initial input focuses on the contribution of financial fraud features within each view; The multi-head attention mechanism consists of multiple attention modules, each of which has its own independent linear transformation of query, key and value, and then concatenates the outputs of multiple heads and performs a final linear transformation; multi-head attention allows the model to jointly focus on different subspace information; MultiHead(Q,K,V)=Concat(head1,…,head h )W O #(8) is a parameter matrix, which is continuously optimized during the training process. This paper uses 8 attention heads, i.e., h = 8. In each attention head, there are d K =d V =d model / h=8; M=MultiHead(Q,K,V)∈R k×d , where the initial The numerical embedding vectors of the four views of USCE are input into four different multi-head attention modules respectively, and the representation codes MG, MO, MN, and ME of the four views are output.

5. The data recognition method based on multi-view learning according to claim 1, characterized in that: In S3, in order to extract the consistency information between different views, an alignment strategy based on correlation, i.e., interactive calculation, is used; based on the multi-view vector representation of the encoding layer, the input vector M∈R k×d The dimension reduction is represented as y∈R D , where D = kd; the interaction calculation of each view is as follows: Among them, v≠u; Represents batch matrix multiplication; ψ v,u (·) is the input dimension D 2 , a two-layer neural network with an output dimension of D; Used to Mapping to space R D The consistency matrix of all interaction views is expressed as follows, where V represents the number of views; For a certain view, it interacts with other views to utilize enhanced view-specific information; it is spliced ​​with other views, and the spliced ​​view is fused with the original view to obtain complementary information; Among them, f concate (·) represents the view stitching function; is enhanced view information; ⊙ represents element addition; χ v (·) represents a neural network with D-dimensional output.

6. The data recognition method based on multi-view learning according to claim 1, characterized in that: In S4, the decision information set includes view consistency information Y b And the complementary information of V views It is expressed as follows: Construct a three-layer fully connected classification network with input nodes D m Represents Y m Feature dimension, the output is the node is the classification type C, and the classification of each information is calculated; In order to make joint decisions using information from each view, a fusion loss strategy for multi-view classification is adopted. The formula is as follows: Among them, α m is the weight of the mth view; l i is the one-hot encoding of the sample label; L m (·) is the cross entropy loss. Different weights are adaptively assigned to each view during the training process, and the information learned by each view complements other views.

Citation Information

Cited By

  • Data processing method and system based on multi-time scale dynamic behavior

    CN121412814A