A bank customer classification method and system based on federated semi-supervised graph learning

Abstract

The present invention provides a bank customer classification method and system based on federated semi-supervised graph learning. The method comprises: performing multiple rounds of global learning until a global customer classification model converges, wherein each round of global learning comprises: sending a global pseudo-label matrix and global model parameters of this round after adding noise to a training bank end; the training bank end performs reinforcement processing on a local customer data graph based on the global pseudo-label matrix of this round, performs local iterative training on a local customer classification model, and uploads the local model parameters and local customer node embedding after adding noise; a central server obtains updated global model parameters and global customer node embedding; the bank end generates a bank-end customer classification model using the updated global model parameters, and inputs the local customer data graph into the bank-end customer classification model to obtain a bank-end prediction label matrix. The present invention improves privacy security, as well as the classification accuracy of the local customer classification model and the global customer classification model.

Description

Technical Field

[0001] The present invention relates to the technical field of big data analysis, and in particular to a bank customer classification method and system based on federated semi-supervised graph learning. Background Art

[0002] In the financial transaction network, each bank has its own data graphs such as customer information, transaction network, and default history. These data graphs take each customer as a customer node and include multiple edges connecting customer nodes. Each edge indicates that there is a transaction relationship between the two customers it connects through the bank. The edge has a direction, which is used to indicate the direction of fund flow between the two customer nodes it connects. For each edge connected to a customer node, it indicates that the customer node spends or receives funds. There are some common customers between banks, but there are also differences in customers, which leads to the heterogeneity and complementarity of the local customer data graphs on the bank side. Banks hope to accurately classify customers (customer nodes) through cooperation, that is, to obtain accurate credit rating labels for customers and determine a common industry blacklist so as to conduct business with low risk. However, due to privacy security and industry competition, it is almost impossible to collect local customer data graphs of various banks and merge them into a large graph data, and banks may be reluctant to share data. Therefore, the local customer data graphs on the bank side have problems of isolation, statistical heterogeneity, and high non-independent and identically distributed (Non-I ID). In this scenario, the existing technology uses personalized federated graph neural networks to classify customer nodes, that is, to assign accurate credit rating labels to customers to solve the data island problem while protecting data privacy.

[0003] In the prior art, the customer classification model training process based on personalized federated graph neural network for multiple banks and a central server of federated learning is as follows: Figure 2 As shown. The banks communicate with each other, or use a third-party server to enhance the local customer data graph of the bank, and then use the local customer data graph to train the local customer classification model, and upload the local model parameters to the central server of federated learning for weighted aggregation. The central server then broadcasts the aggregated parameters to all banks for updating. This iteration is repeated until the global customer classification model converges, solving the data isolation problem between banks. Based on Figure 2 In some optimized implementation schemes, in view of the statistical heterogeneity and high non-independent and identically distributed (Non-I ID) of local data graphs of each bank, the client transmits the local data graph to a third-party server, generates data using the GAN network, or communicates between servers to achieve federated transfer learning and data enhancement. Figure 2In some other optimized implementation schemes, in order to utilize the complementary information distributed in different bank-side data graphs, the global pseudo graph is constructed using the customer node embedding of the bank side, and the global pseudo label is constructed using the bank-side prediction information, which is then passed to other bank-side supplementary labels and structures for global self-supervised learning. Figure 2 and based on Figure 2 The optimization implementation plan partially solves the problems of data isolation, statistical heterogeneity of bank-side data, missing labels for some customers, and high non-independent and identically distributed (Non-IID) data. However, the following problems still exist:

[0004] (1) Data enhancement processing, including private data processing and external data utilization, relies on communication between banks or between banks and service providers, which may lead to privacy leakage.

[0005] (2) In the process of constructing global pseudo-labels, the existing scheme uses the prediction information on the bank side to construct global pseudo-labels. It does not take into account the damage of node similarity by the customer classification model based on graph neural network and the damage of heterogeneous bank sides to the prediction results, which reduces the accuracy of the customer classification model. Summary of the invention

[0006] The present invention aims to at least solve the technical problems of privacy security and low customer classification accuracy in the prior art, and provide a bank customer classification method and system based on federated semi-supervised graph learning.

[0007] In order to achieve the above-mentioned purpose of the present invention, according to the first aspect of the present invention, the present invention provides a bank customer classification method based on federated semi-supervised graph learning, performing multiple rounds of global learning until the global customer classification model converges, and each round of global learning includes: the central server sends the current round of global pseudo-label matrix and the current round of global model parameters after noise to the training bank end; the training bank end performs reinforcement processing on the local customer data graph based on the current round of global pseudo-label matrix to obtain reinforcement data, uses the current round of global model parameters, the local customer data graph and the reinforcement data to perform local iterative training on the local customer classification model, obtains local model parameters and local customer node embedding, and uploads the local model parameters and the local customer node embedding after noise to the central server; wherein the local customer data graph includes multiple customer nodes of the training bank end and multiple edges connecting the customer nodes; the central server aggregates the local customer data graph and the local customer data graph to obtain the local customer node embedding. The updated global model parameters are obtained by combining the noisy local model parameters, and the global customer node embeddings are obtained by aggregating the noisy local customer node embeddings; the central server updates the global customer classification model using the updated global model parameters. If the updated global customer classification model converges, the updated global model parameters are distributed to all bank terminals. If the updated global customer classification model does not converge, the central server calculates the global pseudo-label matrix for the next round of global learning using the global customer node embeddings, and uses the updated global model parameters as the global model parameters for the next round of global learning; after completing multiple rounds of global learning, the bank terminal generates a bank-side customer classification model using the updated global model parameters, and inputs the local customer data graph into the bank-side customer classification model to obtain the bank-side prediction label matrix, wherein the prediction label matrix includes labels of different customer nodes at different credit levels on the bank terminal.

[0008] In order to achieve the above-mentioned purpose of the present invention, according to the second aspect of the present invention, the present invention provides a customer classification system based on federated semi-supervised graph learning, which is used to implement the customer classification method based on federated semi-supervised graph learning described in the first aspect of the present invention, including multiple bank terminals and a central server, and the central server is connected to and communicates with the multiple bank terminals respectively.

[0009] The training bank of the present invention performs reinforcement processing on the local customer data graph based on the global pseudo-label matrix of this round, and only performs reinforcement processing through the local customer data graph of the bank. The training bank ends no longer communicate with each other, which not only reduces the risk of privacy leakage but also improves the accuracy of the local customer classification model of each training bank end, which is beneficial to improving the performance of the global classification model. At the same time, the differential privacy mechanism is used to perform noise processing on the global model parameters, local model parameters, customer node embedding, etc., which further enhances security; the global pseudo-label matrix is ​​obtained by the central server using the global customer node embedding calculation, which avoids the destruction of the customer node similarity by the convolutional neural network and the destruction of the prediction results by the heterogeneous bank end. The central server generates a global pseudo-label matrix based on the customer node similarity, which makes the global pseudo-label matrix more accurate, thereby improving the classification accuracy of the local customer classification model and the global customer classification;

[0010] The present invention proposes a data graph enhancement technology based on adding pseudo labels, constructing KNN graphs, and adding self-loops to high PageRank value nodes. The technology can significantly improve the accuracy of local customer classification models, especially in scenarios with heterogeneous graphs and high node overlap, and the accuracy is 2%-4% higher than that of the comparative federated graph learning method. Through the joint optimization of data graph enhancement and pseudo labeling algorithms, the present invention improves the accuracy of each local customer classification model as a whole, thereby enhancing their positive contribution to the performance of the global classification model of the central server.

[0011] In the process of fusing local model parameters and client node embedding, the central server of the present invention proposes a federal aggregation technology based on L2 norm weight distribution, which can significantly improve the model's ability to generalize from overfitted bank-side distribution to global distribution and enhance the generalization of the customer classification model. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 It is a flow chart of a bank customer classification method based on federated semi-supervised graph learning in a preferred embodiment of the present invention;

[0013] Figure 2 It is a schematic diagram of the customer classification model training process in the prior art;

[0014] Figure 3 It is a schematic diagram of a customer classification model training process in a preferred embodiment of the present invention;

[0015] Figure 4 It is a schematic diagram of the process of training the local customer data graph enhancement at the bank end in a preferred embodiment of the present invention. DETAILED DESCRIPTION

[0016] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be understood as limiting the present invention.

[0017] In the description of the present invention, it is necessary to understand that the terms "longitudinal", "lateral", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc., indicating the orientation or position relationship, are based on the orientation or position relationship shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention.

[0018] In the description of the present invention, unless otherwise specified and limited, it should be noted that the terms "installed", "connected" and "connected" should be understood in a broad sense. For example, it can be a mechanical connection or an electrical connection, or it can be the internal connection of two components. It can be a direct connection or an indirect connection through an intermediate medium. For ordinary technicians in this field, the specific meanings of the above terms can be understood according to the specific circumstances.

[0019] Suppose the customer data graph owned by the central server is G = (V, E), the customer node set is G, and the edge set is E. The total number of customer nodes in the customer data graph G is N = |V|. The adjacency matrix, customer node feature matrix, and true label matrix of the customer data graph are expressed as and Where d is the feature dimension in the feature matrix of the customer node; C is the number of credit ratings. Each customer node in Y has C credit rating labels, and the cth credit rating label can be represented as 0 or 1, 1 or 0 respectively indicating that the customer node belongs to or does not belong to the cth credit rating. Each bank terminal stores the local customer image data of the bank. The local customer data graph G of bank terminal k k =(V k ,E k ), the node set is V k , the edge set is E k , the total number of nodes is N k =|V k |, the local adjacency matrix, local customer node feature matrix and local true label matrix of bank side k are expressed as and The above data graph takes each customer as a customer node, including multiple edges connecting customer nodes. An edge connecting two customer nodes indicates that there is a transaction relationship between the two corresponding customers through the bank. The edge has a direction, which is used to indicate the direction of fund flow between the two customer nodes it connects. For each customer node, the edge connected to it indicates the expenditure or income of the customer node. The feature matrix of the customer node includes multi-dimensional features such as customer transaction records and credit history.

[0020] The real label matrix is ​​used to train the supervision information of the customer classification model, indicating the credit rating of some customer nodes. In actual situations, only a small number of customer nodes will be marked with real credit rating labels. There is a large amount of overlap and information complementarity between customers of each bank, and the entire network will have a certain degree of heterogeneity. Therefore, the present invention aims to use the local customer data graphs of all banks, use self-supervised learning to personalize and train a global customer classification model, and predict the credit rating of unlabeled customer nodes.

[0021] The present invention discloses a bank customer classification method based on federated semi-supervised graph learning. In a preferred embodiment, referring to Figure 1 and Figure 3 As shown, including:

[0022] Step A: Perform multiple rounds of global learning until the global customer classification model converges. Each round of global learning includes:

[0023] Step A1: The central server sends the noisy global pseudo-label matrix and the noisy global model parameters to the training bank.

[0024] In this embodiment, the bank terminal refers to a bank terminal device, preferably but not limited to a server, computer or computer cluster of the bank terminal. The central server will select a part of the bank terminals as training bank terminals in each round of global learning. Specifically, K' bank terminals can be selected as training bank terminals according to the proportion of participants C'∈(0,1):

[0025] K′=max(C′×NUM,1)

[0026] Among them, NUM represents the number of bank terminals, and the training bank terminal set is SUM train .

[0027] In this embodiment, the central server preferably but not limited to adopting a differential privacy mechanism to add noise, such as Gaussian noise, to the current round global pseudo-label matrix and the current round global model parameters.

[0028] Step A2: The training bank performs reinforcement processing on the local customer data graph based on the global pseudo-label matrix of this round to obtain reinforcement data, and uses the global model parameters of this round, the local customer data graph and the reinforcement data to perform local iterative training on the local customer classification model to obtain local model parameters and local customer node embedding, and uploads the noisy local model parameters and local customer node embedding to the central server; wherein the local customer data graph includes multiple customer nodes of the training bank and multiple edges connecting the customer nodes. Each edge indicates that there is a transaction relationship between the two customer nodes it connects.

[0029] In step A3, the central server aggregates the noisy local model parameters to obtain updated global model parameters, and aggregates the noisy local client node embeddings to obtain global client node embeddings.

[0030] In step A4, the central server uses the updated global model parameters to update the global customer classification model. If the updated global customer classification model converges, the updated global model parameters are distributed to all banks. If the updated global customer classification model does not converge, the central server uses the global customer node embedding to calculate the global pseudo-label matrix for the next round of global learning, and uses the updated global model parameters as the global model parameters for the next round of global learning.

[0031] In this embodiment, the convergence judgment criterion of the updated global customer classification model can be specifically that the change in the value of the global loss function is less than a preset change threshold. The global loss function is preferably, but not limited to, an existing cross entropy loss function, which is used to evaluate the difference between the predicted label matrix and the true label matrix obtained by the updated global customer classification model.

[0032] Step B, after completing multiple rounds of global learning, the bank side (which can be a training bank side or a testing bank side) uses the updated global model parameters to generate a bank side customer classification model, and inputs the local customer data graph into the bank side customer classification model to obtain the bank side prediction label matrix. The prediction label matrix includes labels of different customer nodes on the bank side at different credit levels. For example, the label is 1 or 0, 1 represents true, and 0 represents false.

[0033] In this embodiment, preferably, in step A2, the training bank performs noise processing on the local model parameters (i.e., the gradient parameters of the local customer classification model) and the local customer node embedding obtained by the local iterative training in the following manner:

[0034] Assume that after the local iteration training of training bank k is completed, the local model parameters are obtained and local client node embedding Then the local model parameters after adding noise are:

[0035]

[0036] The local client node embedding after noise is:

[0037]

[0038] ∈1,∈2 represent the first noise term and the second noise term respectively, and their distribution is:

[0039]

[0040] The above two distributions have mean 0 and covariance matrices are Gaussian distribution. By designing a secure and trusted model aggregation framework based on δ-differential privacy protection, ε-differential privacy is satisfied. Under the premise of ensuring the privacy security of user data, the privacy budget ε and noise ratio δ are precisely controlled to achieve the best balance between security and model performance, and to achieve quantitative privacy protection of sensitive information at the training sample level.

[0041] In a preferred embodiment, in step A2, the training bank performs enhancement processing on the local customer data graph based on the global pseudo-label matrix of this round to obtain enhanced data, referring to Figure 4 ,include:

[0042] Enhanced data includes pseudo-label matrix on the bank side Characteristic matrix and the self-loop degree matrix

[0043] Step A21, based on the local customer data graph G of the training bank end k k Get the local adjacency matrix A k , local customer node feature matrix X k and the local true label matrix Y k , k represents the index of the training bank.

[0044] Step A22: transform the global pseudo-label matrix Project to the local client node of the training bank k to obtain the pseudo label matrix of the bank

[0045] Preferably, the projection formula is:

[0046]

[0047] Represents the pseudo label matrix of the i-th customer node of the training bank k on the bank side Pseudo label information in index(V k (i)) indicates finding the node index of the i-th client node of training bank k on the central server.

[0048] Step A23, based on the local customer node feature matrix X of the training bank end k k Get the characteristic matrix

[0049] Step A24, based on the local customer data graph G of the training bank end k k Get the self-loop degree matrix

[0050] In this embodiment, further preferably, in step A23, based on the local customer node feature matrix X of the training bank end k k Get the characteristic matrix include:

[0051] Step A231, based on the local customer node feature matrix X of the training bank end k k Calculate the similarity of each pair of customer nodes.

[0052] Suppose a set of client node pairs is (v i ,v j ), v i ,v j ∈V k ,x i ,x j ∈X k , client node (v i ,v j ) is:

[0053]

[0054] Among them, x i ,x j Represent the feature matrices of customer nodes i and j respectively.

[0055] Step A232, sort the customer node pairs in descending order according to the similarity, select the first K1 groups of customer node pairs in the sorted sequence to form a high similarity customer node pair set HS, K1 is a positive integer, that is, select the customer node pair set with the first K1 high similarity values.

[0056] Step A233, constructing a KNN graph, specifically, generating a feature adjacency matrix of the training bank end k based on the high similarity customer node pair set HS

[0057]

[0058] Among them, (v i ,v j) represents the i-th client node v local to the training bank k i and the jth customer node v j The client node pairs composed of; Represents the feature adjacency matrix The element in the i-th row and j-th column of .

[0059] Step A234, based on the feature adjacency matrix of training bank end k Generate the feature matrix for training bank k

[0060]

[0061] Among them, N k Represents the local customer data graph G of training bank k k The number of client nodes in Represents the feature adjacency matrix The element in the i-th row and s-th column of ; Represents the characteristic matrix The element in the i-th row and j-th column of .

[0062] Through the above steps A231 to A234, a KNN graph can be generated. Figure 4 .

[0063] In this embodiment, further preferably, in step A24, based on the local customer data graph G of the training bank end k k Get the self-loop degree matrix include:

[0064] Step A241, calculate the PageRank value of the customer node in the local customer data graph of the training bank end k, the PageRank value of the i-th customer node is PR(v i ):

[0065]

[0066] Wherein, d′ represents the first coefficient, which is usually 0.7 to 0.9, preferably 0.85; In(v i ) represents the set of customer nodes pointing to customer node i in the local customer data graph of training bank k; Out(v j ) represents the set of customer nodes pointing to customer node j in the local customer data graph of training bank k.

[0067] Step A242: Sort the client nodes in descending order of PageRank value, and select the first K2 client nodes in the sorted sequence to form a high centrality node set. K2 is a positive integer, that is, the client node set HC with the highest PageRank value before K2 is selected.

[0068]

[0069] Step A243, generating a self-circulation degree matrix based on the high centrality node set HC

[0070]

[0071] in, Represents the self-loop degree matrix The element in the i-th row and j-th column of i Represents the i-th client node local to the training bank k.

[0072] In this embodiment, steps A22, A23 and A24 are used to implement the local customer data graph enhancement function of the training bank k to obtain enhanced data.

[0073] In this embodiment, the training bank receives pseudo-label data from the central server and generates a KNN graph based on the similarity between client nodes to obtain the feature matrix Then calculate the Pagerank value of each customer node, select the highest few customer nodes to add self-loops, and obtain the self-loop degree matrix The enhanced data is formed, and the local customer data graph incorporating the enhanced data is put into the local graph neural network (local customer classification model) for training. The incoming data graph can directly perform feature aggregation, and the new customer node representation (customer node embedding) is classified through the fully connected layer to complete the local training. This embodiment proposes a data graph enhancement technology based on adding pseudo-labels, constructing KNN graphs, and adding self-loops to high PageRank value nodes. This technology can significantly improve the classification accuracy of the local customer classification model, especially in scenarios with high overlap between heterogeneous graphs and customer nodes. The accuracy is 2%-4% higher than other federated graph learning methods. Through the joint optimization of graph data enhancement and pseudo-labeling algorithms, this solution improves the accuracy of each customer classification model as a whole, thereby enhancing their positive contribution to the performance of the global classification model on the central server side. The local data enhancement technology on the training bank side realizes the expansion of the label quantity, protects the similarity of customer nodes, and significantly improves the performance of the federated model.

[0074] In a preferred embodiment, in step A2, the training bank k performs local iterative training on the local customer classification model using the global model parameters of this round, the local customer data graph and the enhanced data to obtain the local model parameters and the local customer node embedding, including:

[0075] Step A201, update the local customer classification model with the current round global model parameters, the local customer classification model includes a cascaded first graph convolutional neural network layer, a second graph convolutional neural network layer and a fully connected layer. Graph convolutional neural network is referred to as GCN.

[0076] Step A202, construct a propagation matrix based on the local customer data graph and reinforcement data of the training bank end k.

[0077] Further preferably, the propagation matrix P is constructed k :

[0078]

[0079] in, Represents the local customer data graph G of training bank k k The local adjacency matrix A k The sum matrix obtained by adding to the identity matrix; Representation and Matrix The degree matrix of Representation Matrix The inverse matrix of the arithmetic square root of ; ω represents the balancing hyperparameter, which is used to balance the original matrix and the feature matrix; Represents the characteristic matrix of training bank k The inverse matrix of the square root of ; represents the feature adjacency matrix of training bank end k; μ represents the number of self-loops, which is a hyperparameter, i.e., μ self-loops are added; Represents the self-circulation degree matrix of training bank end k.

[0080] Step A203: The training bank k performs multiple local iterative training on the updated local customer classification model until the iterative training stop condition is reached. In each local iterative training:

[0081] Step A2031: The first graph convolutional neural network layer processes the input feature matrix of the local client node in combination with the propagation matrix to obtain the first layer feature representation of the local client node. Further preferably, the first layer feature representation of the local client node

[0082]

[0083] in, represents the learnable weight parameter of the first graph convolutional neural network layer; σ1(·) represents the first nonlinear activation function.

[0084] Step A2032: The second graph convolutional neural network layer processes the first layer feature representation of the local client node in combination with the propagation matrix to obtain the second layer feature representation of the local client node. Further preferably, the second layer feature representation of the local client node

[0085]

[0086] in, represents the learnable weight parameter of the second graph convolutional neural network layer; σ2(·) represents the second nonlinear activation function.

[0087] Step A2033: The fully connected layer processes the second layer feature representation of the local client node to obtain the local prediction label matrix of the local client node. Further preferably, the local prediction label matrix of the local client node

[0088]

[0089] in, Represents the weight parameters of the fully connected layer.

[0090] Step A204, when the local iterative training is completed, the weight parameters of the local customer classification model obtained in the last iterative training and the local customer node embedding are output, and the local customer node embedding is the second-layer feature representation of the local customer node obtained in the last iterative training. The weight parameters of the local customer classification model include wait.

[0091] In this embodiment, the iterative training stopping condition is preferably, but not limited to, that the change of the local loss function is less than a preset local change threshold, or that the number of local iterative training reaches a preset maximum number of iterations.

[0092] In this implementation, the reinforced data is placed in the local graph neural network (local customer classification model) for training. The incoming data graph can be directly feature aggregated, and the new customer node representation (local customer node embedding) is classified through the fully connected layer to complete the local training. By introducing reinforced data in the local iterative training, the accuracy of the local customer classification model can be significantly improved, especially in scenarios with heterogeneous graphs and high node overlap.

[0093] In a preferred embodiment, the local loss function of the training bank end k in the local iterative training is:

[0094]

[0095] Among them, (Y k ) o,cRepresents the true label matrix Y of the local customer nodes of training bank k k The true label of the oth customer node at the cth credit level, c∈[1,C], C represents the credit level number, o∈[1,N k ],N k represents the number of local client nodes of the training bank side k; Represents the local prediction label matrix of training bank side k The predicted label of the o-th customer node at the c-th credit level in ; α represents the self-supervised learning hyperparameter; represents the bank-side pseudo-label matrix for training bank-side k The pseudo label of the o-th customer node at the c-th credit level.

[0096] In this embodiment, the parameter θ of the local customer classification model is updated by the cross entropy loss function: Where η represents the learning rate. The cross entropy loss function introduces the bank-side pseudo-label matrix of training bank k Can significantly improve the accuracy of local customer classification models.

[0097] In a preferred embodiment, in each round of global learning, it also includes:

[0098] The central server selects one or more training bank terminals and one or more test bank terminals. The central server sets the training bank terminals SUM train Then, at the remaining bank end SUM-SUM train Select J test bank terminals from the test bank terminal set SUM test J is a positive integer. SUM represents the bank end set.

[0099] After completing the local iterative training to obtain the local model parameters, the training bank sends the noisy local model parameters to the test bank. Preferably, each training bank distributes the noisy local model parameters to all test banks to ensure test accuracy.

[0100] The test bank tests the local customer classification model of the training bank based on the local model parameters sent by the training bank to obtain the test set accuracy, and uploads the test set accuracy to the central server. Specifically, for the test bank j′, it uses the test set to test the local customer classification model of the training bank k based on the local model parameters sent by the training bank k, and obtains the test set accuracy

[0101] The central server obtains the weight of each training bank based on the test set accuracy of the local customer classification model of all training banks. Specifically, it includes:

[0102] Step a: The central server aggregates the test set accuracy of all test banks to obtain the global test accuracy Acc of training bank k. k

[0103]

[0104] Assume there are J test banks; N j′ is the number of client nodes at the test bank j′,

[0105] Step b, calculate the weight of each training bank end by scoring the accuracy of each training bank end, and the weight p of training bank end k k :

[0106]

[0107] Among them, K′ represents the number of training banks; the scoring function is the L2 norm function: By dividing each element of the input x by the L2 norm of the entire vector, the weight of the training bank with low accuracy (the local overfitting training bank that is not adapted to the global distribution) can be reduced, thereby highlighting the important training bank (the training bank that adapts to the global distribution).

[0108] The central server aggregates the local model parameters of all training banks after adding noise based on the weight of each training bank to obtain the global model parameter update, and aggregates the local customer node embeddings of all training banks after adding noise to obtain the global customer node embedding. Specifically:

[0109] The global model parameters are updated as:

[0110]

[0111] The global client node embedding is:

[0112]

[0113] In this embodiment, each training bank end uploads the local model parameters and node embedding obtained by local iterative training to the central server, and uploads the noisy local model parameters to each test bank end. After receiving the local model parameters of the training bank end, the test bank end uses the local data graph of the test bank end for testing, and uploads the test accuracy to the central server. The central server aggregates the accuracy, calculates the weights of the local model parameters and local node embedding of each training bank end, and then aggregates them according to the weight. This embodiment solves the balance of each training bank end, handles the impact of extremely heterogeneous training bank ends on the global customer classification model, and trains the local model parameters uploaded by each training bank end through a group of test bank ends to obtain the accuracy of each training bank end for the test bank end data graph. Then these accuracy rates are scored by the L2 norm function and normalized to obtain the weight. This embodiment proposes a federated aggregation technology based on L2 norm weight allocation, which can significantly improve the ability of the customer classification model to generalize from the overfitting training bank end distribution to the global distribution. It achieves rapid adaptation from generalization of the training bank to global distribution, and has stronger ability to adapt to heterogeneous distribution than direct federated average aggregation.

[0114] In a preferred embodiment, in step A4, the central server uses the global client node embedding to calculate the global pseudo-label matrix for the next round of global learning, including:

[0115] Step A41, using the global client node embedding H to calculate the global node similarity matrix S:

[0116] S=Φ(HH T )

[0117] Where Φ(·) represents the T The largest element of the matrix and 0 is selected as the element of the global node similarity matrix;

[0118] Step A42, normalize the global node similarity matrix S to obtain a normalized similarity matrix

[0119] Step A43, by normalizing the similarity matrix The global prediction matrix P is obtained by summing the global true label matrix Y. The element P in the oth row and cth column of the global prediction matrix P is oc for:

[0120]

[0121] Among them, P oc represents the label of the oth customer node at the cth credit level in the global prediction matrix P, Represents the normalized similarity matrix The element in row o and column c, Y oc The o-th row and c-th column element of the global true label matrix Y in denoted;

[0122] Step A44, calculate the global pseudo-label matrix through the global prediction matrix P The pseudo label of the oth customer node at the cth credit level in for:

[0123]

[0124] Among them, λ represents the preset pseudo-label threshold, P o Represents the o-th row element of the global prediction matrix P.

[0125] In this implementation, a pseudo-label generation method based on a label propagation algorithm based on client node similarity is adopted. The central server propagates pseudo-labels according to global node similarity, and broadcasts the global pseudo-label matrix to each training bank end, solving the impact of heterogeneous training bank ends on self-supervised learning caused by the destruction of prediction results. A new pseudo-label generation algorithm is designed to achieve rapid propagation generation of labels. Compared with the calculation of local prediction results by the training bank end, this technology has stronger reliability and avoids interference from heterogeneous clients.

[0126] The present invention also discloses a customer classification system based on federated semi-supervised graph learning, which is used to implement the customer classification method based on federated semi-supervised graph learning, including multiple bank terminals and a central server, wherein the central server is connected and communicated with the multiple bank terminals respectively. The specific customer classification process refers to the description of the customer classification method based on federated semi-supervised graph learning, which will not be repeated here.

[0127] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", "an implementation", "a preferred implementation" or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0128] Although the embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the claims and their equivalents.

Claims

1. A bank customer classification method based on federated semi-supervised graph learning, characterized in that: Perform multiple rounds of global learning until the global customer classification model converges. Each round of global learning includes: The central server sends the noisy global pseudo-label matrix and global model parameters of this round to the training bank. The training bank side performs reinforcement processing on the local customer data graph based on the current round global pseudo-label matrix to obtain reinforcement data, performs local iterative training on the local customer classification model using the current round global model parameters, the local customer data graph and the reinforcement data, obtains local model parameters and local customer node embedding, and uploads the noisy local model parameters and local customer node embedding to the central server; wherein the local customer data graph includes multiple customer nodes of the training bank side and multiple edges connecting the customer nodes; wherein the training bank side performs reinforcement processing on the local customer data graph based on the current round global pseudo-label matrix to obtain reinforcement data, including: Based on the local customer data graph G of the training bank k k Get the local adjacency matrix A k , local customer node feature matrix X k and the local true label matrix Y k , k represents the index of the training bank; The global pseudo-label matrix Project to the local client node of the training bank side k to obtain the pseudo label matrix of the bank side Based on the local customer node feature matrix X of training bank k k Get the characteristic matrix Based on the local customer data graph G of the training bank k k Get the self-loop degree matrix include: The local customer data graph G based on the training bank end k k Get the self-loop degree matrix include: Calculate the PageRank value of the customer node in the local customer data graph of the training bank k; Sort the customer nodes in descending order of PageRank values, and select the first K2 customer nodes in the sorted sequence to form a high centrality node set HC, where K2 is a positive integer; Generating self-circulation degree matrix based on high centrality node set HC in, Represents the self-loop degree matrix The element in the i-th row and j-th column of i represents the i-th client node local to the training bank side k; The enhanced data includes a bank-side pseudo-label matrix Characteristic matrix and the self-loop degree matrix The central server aggregates the noisy local model parameters to obtain updated global model parameters, and aggregates the noisy local client node embeddings to obtain global client node embeddings; The central server updates the global customer classification model using the updated global model parameters. If the updated global customer classification model converges, the updated global model parameters are distributed to all banks. If the updated global customer classification model does not converge, the central server uses the global customer node embedding to calculate the global pseudo-label matrix for the next round of global learning, and uses the updated global model parameters as the global model parameters for the next round of global learning. After completing multiple rounds of global learning, the bank uses the updated global model parameters to generate a bank-side customer classification model, and inputs the local customer data graph into the bank-side customer classification model to obtain the bank-side prediction label matrix, which includes labels of different customer nodes on the bank side at different credit levels.

2. A bank customer classification method based on federated semi-supervised graph learning as claimed in claim 1, characterized in that: The local customer node feature matrix X based on the training bank end k k Get the characteristic matrix include: Based on the local customer node feature matrix X of training bank k k Calculate the similarity of each group of customer node pairs; Sort the customer node pairs in descending order of similarity, and select the first K1 groups of customer node pairs in the sorted sequence to form a high-similarity customer node pair set HS, where K1 is a positive integer; Generate the feature adjacency matrix of training bank end k based on the high similarity customer node pair set HS Among them, (v i ,v j ) represents the i-th client node v at the training bank k i and the jth customer node v j The client node pairs composed of; Represents the feature adjacency matrix The element in the i-th row and j-th column of ; Based on the feature adjacency matrix of training bank end k Generate the feature matrix for training bank k Among them, N k Represents the local customer data graph G of training bank k k The number of client nodes in Represents the feature adjacency matrix The element in the i-th row and s-th column of ; Represents the characteristic matrix The element in the i-th row and j-th column of .

3. A bank customer classification method based on federated semi-supervised graph learning as claimed in claim 1 or 2, characterized in that: The training bank uses the global model parameters, local customer data graph and enhanced data to perform local iterative training on the local customer classification model to obtain local model parameters and local customer node embedding, including: Updating a local customer classification model with the current round global model parameters, wherein the local customer classification model includes a cascaded first graph convolutional neural network layer, a second graph convolutional neural network layer, and a fully connected layer; Construct a propagation matrix based on the local customer data graph and reinforcement data of the training bank side k; The training bank performs multiple local iterative training on the updated local customer classification model until the iterative training stop condition is reached. In each local iterative training: The first graph convolutional neural network layer processes the input feature matrix of the local client node in combination with the propagation matrix to obtain the first-layer feature representation of the local client node; The second graph convolutional neural network layer processes the first-layer feature representation of the local client node in combination with the propagation matrix to obtain the second-layer feature representation of the local client node; The fully connected layer processes the second-layer feature representation of the local client node to obtain the local prediction label matrix of the local client node; When the local iterative training is completed, the weight parameters of the local customer classification model obtained in the last iterative training and the local customer node embedding are output, where the local customer node embedding is the second-layer feature representation of the local customer node obtained in the last iterative training.

4. A bank customer classification method based on federated semi-supervised graph learning as claimed in claim 3, characterized in that: Construct the propagation matrix P based on the local customer data graph and reinforcement data of the training bank k k : in, Represents the local customer data graph G of training bank k k The local adjacency matrix A k The sum matrix obtained by adding to the identity matrix; Representation and Matrix The degree matrix of Representation Matrix The inverse matrix of the arithmetic square root of ; ω represents the balanced hyperparameter; Represents the characteristic matrix of training bank k The inverse matrix of the square root of ; represents the feature adjacency matrix of training bank end k; μ represents the number of self-loops; represents the self-loop degree matrix of training bank end k; And / or, the first-layer feature representation of the local client node in, represents the learnable weight parameter of the first graph convolutional neural network layer; σ1(·) represents the first nonlinear activation function; And / or, the second layer feature representation of the local client node in, represents the learnable weight parameter of the second graph convolutional neural network layer; σ2(·) represents the second nonlinear activation function; and / or, the local predicted label matrix of the local client node in, Represents the weight parameters of the fully connected layer.

5. A bank customer classification method based on federated semi-supervised graph learning as claimed in claim 4, characterized in that: The local loss function of the training bank k in the local iterative training is: Among them, (Y k ) o,c Represents the true label matrix Y of the local customer nodes of training bank k k The true label of the oth customer node at the cth credit level, c∈[1,C], C represents the credit level number, o∈[1,N k ],N k represents the number of local client nodes of the training bank side k; Represents the local prediction label matrix of training bank side k The predicted label of the o-th customer node at the c-th credit level in ; α represents the self-supervised learning hyperparameter; represents the bank-side pseudo-label matrix for training bank-side k The pseudo label of the o-th customer node at the c-th credit level.

6. A bank customer classification method based on federated semi-supervised graph learning as described in claim 1, 2, 4 or 5, characterized in that: In each round of global learning, it also includes: The central server selects one or more training bank terminals and one or more testing bank terminals; After completing the local iterative training and obtaining the local model parameters, the training bank sends the noisy local model parameters to the test bank. The testing bank tests the local customer classification model of the training bank based on the local model parameters sent by the training bank to obtain the test set accuracy, and uploads the test set accuracy to the central server; The central server obtains the weight of each training bank terminal based on the test set accuracy of the local customer classification model of all training bank terminals; The central server aggregates the noisy local model parameters of all training banks based on the weights of each training bank to obtain global model parameter updates, and aggregates the noisy local customer node embeddings of all training banks to obtain global customer node embeddings.

7. A bank customer classification method based on federated semi-supervised graph learning as described in claim 1, 2, 4 or 5, characterized in that: The central server uses the global client node embedding to calculate the global pseudo-label matrix for the next round of global learning, including: The global node similarity matrix S is calculated using the global client node embedding H: S=Φ(HH T ) Where Φ(·) represents the T The largest element of the matrix and 0 is selected as the element of the global node similarity matrix; Normalize the global node similarity matrix S to obtain the normalized similarity matrix By normalizing the similarity matrix The global prediction matrix P is obtained by summing the global true label matrix Y. The element P in the oth row and cth column of the global prediction matrix P is oc for: Among them, P oc represents the label of the oth customer node at the cth credit level in the global prediction matrix P, Represents the normalized similarity matrix The element in row o and column c, Y oc The o-th row and c-th column element of the global true label matrix Y in . Calculate the global pseudo-label matrix through the global prediction matrix P The pseudo label of the oth customer node at the cth credit level in for: Among them, λ represents the preset pseudo-label threshold, P o Represents the o-th row element of the global prediction matrix P.

8. A customer classification system based on federated semi-supervised graph learning, used to implement a bank customer classification method based on federated semi-supervised graph learning as claimed in any one of claims 1 to 7, characterized in that: It includes multiple bank terminals and a central server, and the central server is connected and communicated with the multiple bank terminals respectively.

Images