Model training method and related equipment

Abstract

The invention discloses a model training method and related equipment, and belongs to the technical field of model training. The method comprises the steps of obtaining target graph data of each node in a sub-network where a client is located, and training a first graph neural network model corresponding to the client according to the target graph data of each node to obtain a second graph neural network model; obtaining model parameters of the second graph neural network model, and compressing the target graph data to obtain target compressed graph data; sending the model parameters and the target compressed graph data to a server; and receiving a target gradient sent by the server, and updating model parameters of the second graph neural network model according to the target gradient. According to the invention, the performance of the model trained by the client is improved.

Description

Technical Field

[0001] The present application belongs to the field of model training technology, and specifically relates to a model training method and related equipment. Background Art

[0002] Subgraph federated learning is an emerging federated learning method that trains models using distributed graph data.

[0003] Currently, the mainstream method for model training in subgraph federated learning is model aggregation. This involves the client training the model locally and uploading the model parameters to the server, which then aggregates them to update the global model.

[0004] However, the graph data of different clients may have significant structural and feature differences, and the graph data of each client is heterogeneous. The heterogeneous graph data will make it difficult for the server to train the model, resulting in low accuracy of the model parameters of the model trained by the server. The model in the client is updated through the model parameters of the model trained by the server, resulting in poor performance of the model trained by the client. Summary of the Invention

[0005] The purpose of the embodiments of the present application is to provide a model training method and related equipment to solve the problem of poor performance of the model trained by the client.

[0006] In a first aspect, an embodiment of the present application provides a model training method, which is applied to a client. The model training method includes:

[0007] Obtain target graph data of each node in the subnetwork where the client is located, and train the first graph neural network model corresponding to the client based on the target graph data of each node to obtain a second graph neural network model;

[0008] Obtaining model parameters of the second graph neural network model, and compressing the target graph data to obtain target compressed graph data;

[0009] Sending the model parameters and the target compressed graph data to the server, wherein the server constructs a global compressed graph of the target network based on the target compressed graph data sent by each client, and performs model training based on the global compressed graph and the model parameters sent by each client to obtain the target gradient of the trained model, wherein the target network is composed of the sub-networks where each client is located, and the model in the server has the same model structure as the graph neural network model corresponding to the client;

[0010] Receive the target gradient sent by the server, and update the model parameters of the second graph neural network model according to the target gradient.

[0011] In some possible implementations, compressing the target graph data to obtain target compressed graph data includes:

[0012] Compressing the target graph data to obtain initial compressed graph data;

[0013] Inputting the initial compressed graph data into the second graph neural network model;

[0014] Predicting the initial compressed graph data according to the second graph neural network model to obtain a target parameter, where the target parameter is used to indicate a loss parameter between a true label and a predicted label of the initial compressed graph data;

[0015] The initial compressed graph data is optimized according to the target parameters to obtain target compressed graph data.

[0016] In some possible implementations, compressing the attribute features in the target graph data to obtain initial compressed graph data includes:

[0017] Obtaining initial compressed attribute features corresponding to attribute features in the target graph data, and constructing an initial compressed adjacency matrix based on the initial compressed attribute features;

[0018] Constructing initial compressed graph data according to the labels in the target graph data, the initial compressed attribute features, and the initial compressed adjacency matrix;

[0019] Optimizing the initial compressed graph data according to the target parameters to obtain target compressed graph data includes:

[0020] Optimizing the initial compression attribute characteristics according to the target parameters to obtain target compression attribute characteristics, and constructing a target compression adjacency matrix according to the target compression attribute characteristics;

[0021] Target compressed graph data is constructed according to the labels in the target graph data, the target compressed attribute features, and the target compressed adjacency matrix.

[0022] In some possible implementations, constructing an initial compressed adjacency matrix according to the initial compressed attribute features includes:

[0023] Determining similarities between initial compressed attribute features of adjacent nodes in the subnetwork;

[0024] The weights of the edges connecting the adjacent nodes are determined according to the image similarities, and an initial compressed adjacency matrix is ​​constructed according to the initial compressed attribute features of the adjacent nodes, the weights, and a sparsification function.

[0025] In some possible implementations, obtaining the initial compressed attribute feature corresponding to the attribute feature in the target graph data includes:

[0026] Obtaining the number of times the graph neural network model of the client has been trained;

[0027] When the number of training times is less than a preset number, performing random initialization processing of Gaussian distribution on the attribute features in the target graph data to obtain initial compressed attribute features;

[0028] When the number of training times is higher than or equal to the preset number of times, the target compression attribute features in the target compression graph data corresponding to the last training of the client's graph neural network model are used as the initial compression attribute features.

[0029] In a second aspect, the present application provides a model training method, which is applied to a server. The model training method includes:

[0030] Receive target compressed graph data of the sub-network sent by each client and model parameters of the trained graph neural network model in each of the clients;

[0031] Aggregating each of the target compressed image data to obtain a global compression graph of a target network, and aggregating each of the model parameters to obtain a target model parameter, wherein the target network is composed of the sub-networks where each of the clients is located;

[0032] According to the target model parameters and the global compression graph, the model in the server is trained to obtain the target gradient, and the target gradient is sent to each of the clients so that each client can update the model parameters of the graph neural network model based on the target gradient.

[0033] In a third aspect, an embodiment of the present application provides a client, including:

[0034] A first acquisition module is used to obtain target graph data of each node in the sub-network where the client is located, and train the first graph neural network model corresponding to the client according to the target graph data of each node to obtain a second graph neural network model;

[0035] a second acquisition module, configured to acquire model parameters of the second graph neural network model, and compress the target graph data to obtain target compressed graph data;

[0036] A sending module, configured to send the model parameters and the target compressed graph data to a server, wherein the server performs model training based on the global compressed graph of the target network of the target compressed graph data sent by each client and the model parameters sent by each client to obtain a target gradient, wherein the target network is composed of the subnetworks where each client is located, and the model in the server has the same model structure as the graph neural network model corresponding to the client;

[0037] A receiving module is used to receive the target gradient sent by the server and update the model parameters of the second graph neural network model according to the target gradient.

[0038] In a fourth aspect, an embodiment of the present application provides a server, including:

[0039] A receiving module, configured to receive the target compressed graph data of the sub-network sent by each client and the model parameters of the trained graph neural network model in each of the clients;

[0040] an aggregation module, configured to aggregate the target compressed image data to obtain a global compression graph of a target network, and to aggregate the model parameters to obtain target model parameters, wherein the target network is composed of the sub-networks where the clients are located;

[0041] A training module is used to train the model in the server according to the target model parameters and the global compression graph, obtain the target gradient, and send the target gradient to each of the clients so that each client can update the model parameters of the graph neural network model based on the target gradient.

[0042] In a fifth aspect, an embodiment of the present application provides an electronic device comprising a processor and a memory, wherein the memory stores programs or instructions that can be run on the processor, and when the programs or instructions are executed by the processor, the steps of the method described in the first aspect or the second aspect are implemented.

[0043] In a sixth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the steps of the method described in the first aspect or the second aspect are implemented.

[0044] In a seventh aspect, an embodiment of the present application provides a computer program product, which is stored in a storage medium and executed by at least one processor to implement the method described in the first aspect or the second aspect.

[0045] In an embodiment of the present application, a graph neural network model is trained on the graph data of each node in the sub-network where the client is located to obtain the model parameters of the trained graph neural network model, and the graph data is compressed to obtain compressed graph data, and then the compressed graph data and the model parameters are sent to the server, so that the server performs model training based on the global compressed graph of the target network constructed by the target compressed graph data sent by each client and the model parameters sent by each client to obtain the target gradient of the trained model. The server sends the target gradient to each client, so that the client updates the model parameters of the graph neural network model based on the target gradient. In the present application, the server constructs a global compressed graph of the target network composed of each sub-network based on the compressed graph data sent by each client. Training the model through the global compressed graph can bring a more consistent optimization direction to the model, thereby reducing the interference of heterogeneity caused by the structural and distribution differences of the sub-networks where each client is located, improving the accuracy of the model parameters of the model trained by the server, and improving the performance of the model trained by the client based on the model parameters sent by the server. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 This is a schematic diagram of the application scenario of the model training method involved in this application

[0047] Figure 2 This is one of the flowcharts of a model training method according to an exemplary embodiment;

[0048] Figure 3 This is a second flow chart of a model training method according to an exemplary embodiment;

[0049] Figure 4 This is a third flow chart of a model training method according to an exemplary embodiment;

[0050] Figure 5 This is a fourth flowchart of a model training method according to an exemplary embodiment;

[0051] Figure 6 is a structural block diagram of a client according to an exemplary embodiment;

[0052] Figure 7 is a structural block diagram of a server according to an exemplary embodiment;

[0053] Figure 8 The figure is a structural block diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0054] The following will be combined with the accompanying drawings in the embodiments of this application to clearly describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field are within the scope of protection of this application.

[0055] The terms "first", "second", etc. in the specification and claims of this application are used to distinguish similar objects, and are not used to describe a specific order or precedence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of this application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of the same type, and do not limit the number of objects. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally represents the front and back.

[0056] Subgraph federated learning is an emerging federated learning method designed to address the privacy protection issues of distributed graph data. Unlike traditional federated learning methods, the main challenge faced by subgraph federated learning lies in the complexity and heterogeneity of graph data. Because graph data subsets owned by data holders (such as hospitals, enterprises, or social platforms) have different nodes and edges, these data subsets may have significant differences in graph structure and characteristics. For example, a hospital's graph data may involve disease transmission relationships between patients, while a social platform's graph data involves interactions between users. This data heterogeneity makes it more difficult to collaboratively train models using subgraph federated learning while preserving data privacy. Subgraph federated learning needs to address these differences in structure and characteristics to ensure that the model can effectively learn and adapt to different data subsets, thereby improving model effectiveness and accuracy. Subgraph federated learning has broad application potential in fields such as medical diagnosis, social network analysis, and financial risk assessment. It can enable knowledge sharing among different data holders while protecting user privacy.

[0057] Currently, mainstream approaches to subgraph federated learning include graph embedding-based and model aggregation-based technologies. Graph embedding methods reduce information dimensionality by converting graph data into low-dimensional vector representations, allowing model training without directly sharing graph data. Model aggregation methods involve the client training the model locally and uploading the model parameters to the server, which then aggregates the parameters to update the global model. Graph embedding and model aggregation-based technologies have been applied in fields such as healthcare, social networks, and finance. For example, disease prediction models can be trained using graph data from different hospitals, or user interaction networks can be analyzed on social platforms without exposing specific user data. While these technologies enable collaborative model training while protecting privacy, they still face the challenge of effectively handling the heterogeneity of graph data and reducing information loss.

[0058] While existing methods have made some progress in privacy protection and model aggregation, they still face several unresolved difficulties. First, while protecting privacy, the efficiency of data transmission and processing remains insufficient. For example, while the model parameters uploaded by the client reflect the learning state of the model, they may not fully preserve the detailed features of the original data. In addition, graph features often need to be encoded during transmission, which may lead to the loss of fine structure and local relationship information between nodes, thereby affecting the model's ability to learn the graph structure. This information loss reduces the information available to the server when aggregating the model, which in turn affects the effectiveness of the collaborative strategy. Second, the problem of graph heterogeneity remains prominent. For example, graph data from different clients may have significant structural and feature differences. Existing technologies find it difficult to accurately measure the similarity between different graphs and balance these differences in the global model. This problem limits the effectiveness and stability of the model in cross-client graph data fusion.

[0059] To solve the above problems, this application proposes a model training method, which compresses the original graph data on the client and uploads the compressed graph data to the server, thereby significantly reducing the amount of data transmission while retaining the original graph information as much as possible. The server aggregates these local model parameters, generates the gradient of the global model, and returns the updated gradient to the client. The model training method not only improves communication efficiency, but also makes the graph structure and features between different clients more consistent by compressing the graph data, thereby alleviating the graph heterogeneity problem. The model training method has been widely used in fields such as medicine, finance, and social networks, providing more efficient privacy protection and model training solutions.

[0060] Reference Figure 1 , Figure 1This is a scenario diagram of the model training method of this application. The server is connected to multiple clients, and the client is a user device in a sub-network. Each sub-network constitutes a target network. The client trains the graph neural network model in the client based on the graph data of the sub-network in which it is located. The client then compresses the graph data to obtain compressed graph data. The client sends the model parameters W of the trained graph neural network and the compressed graph data S to the server. For example, client 1 sends the model parameters W1 and the compressed graph data S1 to the server. The server aggregates the model parameters W sent by each client to obtain The server then aggregates the compressed graph data sent by each client to obtain the global compressed graph of the target network. Thus, based on as well as Train the model on the server to obtain the gradient of the trained model The server will gradient Sent to each client, the client passes Update the model parameters of its own graph neural network model to conduct a new round of model training.

[0061] The following combination Figure 1 The model training method and related equipment provided in the embodiments of the present application are described in detail with specific embodiments and their application scenarios.

[0062] The model training method provided in the embodiments of the present application is described in detail below.

[0063] Reference Figure 2 , Figure 2 One of the flow charts of the model training method provided in this application is applied to the client, such as Figure 2 As shown, the model training method includes the following steps:

[0064] Step S201: Obtain target graph data of each node in the sub-network where the client is located, and train the first graph neural network model corresponding to the client based on the target graph data of each node to obtain a second graph neural network model.

[0065] In this embodiment, the execution subject is the client, which is a user terminal in the sub-network, that is, the client can be a smart phone, smart watch, smart tablet, etc. The client has basic model training, response and data transmission functions, and can obtain sensitive information of itself and the user devices (nodes) on the sub-network. Sensitive information includes connection structure, attribute characteristics and classification labels. Different sub-networks constitute the target network, and the target network represents a social system. The social system is, for example, a social network system, a medical system, a financial system, etc. Social networks, medical care, finance and other fields contain private data. When training application models in these fields, the model can be trained by sub-graph federated learning, which realizes collaborative training of the model while protecting private data.

[0066] The social system consists of n sub-networks, represented as Each subnetwork It consists of multiple user terminal devices and is represented as Among them A i represents the adjacency matrix of the ith subnetwork, X i Represents the attribute characteristics of the user terminal in the sub-network, Y i Indicates the classification of user terminals. Each sub-network has a graph neural network model GNN locally i , the model parameter is W i , used to perform node classification tasks. Any user terminal in the sub-network can be used as a client, and the client is used to train the graph neural network model. Specifically, the client obtains the target graph data of each node in the sub-network. The target graph data of each node is {A i ,X i ,Y i}, each node is a user terminal in the sub-network.

[0067] The client trains the first graph neural network model corresponding to the client based on the target graph data of each node. The first graph neural network model corresponding to the client refers to the local graph neural network model of the sub-network where the client is located. The model after the client trains the first graph neural network model is defined as the second graph neural network model.

[0068] For example, the target graph data is input into the graph neural network model GNN i In the example, the target graph data includes the adjacency matrix A i , the node's attribute feature X i and the true labels.

[0069] Through GNN i Perform forward propagation to generate the predicted category probability distribution for each node This process can be expressed as:

[0070]

[0071] Where W i are the parameters of the model.

[0072] The graph neural network model uses the cross entropy loss function to calculate the error between the predicted result and the true label. The cross entropy loss function is expressed as:

[0073]

[0074] Where N is the total number of nodes, C is the number of categories, and Y i,j,c is the true label (usually 0 or 1) that the jth node belongs to category c, and is the probability that the jth node is predicted to belong to category c.

[0075] After obtaining the loss function, calculate the loss function against the model parameters W i The gradient of the model is then used to update the model parameters. If the client is training the graph neural network model for the first time, the initialized local gradient is used to update the model parameters. The specific process is as follows:

[0076]

[0077] Where η is the learning rate. If the client is training the graph neural network model for the second or later round, it obtains the global optimization gradient sent by the server in the previous round and combines the global optimization gradient with the local gradient to update the model parameters. The process is as follows:

[0078]

[0079] Among them, α is a hyperparameter used to balance the local and global optimization directions, is the local gradient, Optimize the gradient globally.

[0080] It should be noted that each client needs to perform multiple rounds of training on its corresponding graph neural network model. For example, in the fifth round, each client needs to train its corresponding graph neural network model.

[0081] Step S202: Obtain model parameters of the second graph neural network model, and compress the target graph data to obtain target compressed graph data.

[0082] After training the second graph neural network model, the client obtains the model parameters W of the second graph neural network model and compresses the target graph data to obtain target compressed graph data. Exemplarily, each data item in the target graph data is represented by a corresponding code. The fields formed by each code are the compressed graph data, which serves as the target compressed graph data. The target compressed graph data represents the simplified graph structure and its attributes.

[0083] In step S203, the model parameters and target compressed graph data are sent to the server, wherein the server constructs a global compressed graph of the target network based on the target compressed graph data sent by each client, and performs model training based on the global compressed graph and the model parameters sent by each client to obtain the target gradient of the trained model. The target network is composed of the sub-networks where each client is located, and the model in the server has the same model structure as the graph neural network model corresponding to the client.

[0084] After obtaining the model parameters and the target compressed graph data, the client sends the model parameters and the target compressed graph data to the server.

[0085] The server receives all model parameters {W1,…,W n} and target compressed graph data Finally, the model parameters of all clients are averaged and aggregated to obtain the global model Parameters The polymerization process can be expressed by the following formula:

[0086]

[0087] The server merges the target compressed graph data uploaded by all clients to generate a global graph data Server in the global model Calculate the loss function of the classification task

[0088]

[0089] Among them, n×N' is The total number of nodes, C is the number of categories, Y' j,c is the true label (usually 0 or 1) that the jth node belongs to category c, and is the probability that the jth node is predicted to belong to category c.

[0090] The server calculates the loss function For global model parameters The target gradient The target gradient represents the optimization direction of the global model. The server returns the target gradient to each client for the next round of local training and parameter updates.

[0091] It should be noted that the model structure of the server-side model is the same as that of the corresponding graph neural network model on the client.

[0092] Step S204: Receive the target gradient sent by the server, and update the model parameters of the second graph neural network model according to the target gradient.

[0093] The client receives the target gradient sent by the server and updates the model parameters of the second graph neural network model based on the target gradient, allowing the client to perform the next round of model training. That is, the client updates the second graph neural network model after the model parameter update to the first graph neural network model, and then returns to steps S201 to S204. When the number of times the client has trained the model reaches the set maximum number of iterations, the client does not need to perform the next round of model training.

[0094] The maximum number of iterations T varies based on the size of the dataset. For example, for a small dataset, T is set to 100, and for a large dataset, T is set to 200.

[0095] This embodiment is applicable to various complex network data consisting of entities and their relationships, especially subgraph federated learning for distributed training in scenarios involving privacy protection. Specific applications include but are not limited to social networks, financial transaction networks, transportation networks, bioinformatics, and knowledge graphs.

[0096] In these applications, data is usually stored in the form of graphs, where nodes represent entities, such as users in social networks, accounts in financial transactions, and stations in transportation systems, while edges represent the relationships between them, such as friend relationships, capital flows, road connections, etc.

[0097] For example, in social network analysis, nodes can represent users, while edges represent friendships or interactive behaviors (such as likes, comments, and shares). Because user data in social networks is highly private, traditional centralized learning methods are prone to privacy leaks. The solution provided in this embodiment allows social subgraphs in different regions or platforms to be trained locally, and only the compressed graph data and model parameters are uploaded to the cloud. This enables collaborative training under privacy protection, improving application scenarios such as personalized recommendations and social relationship prediction.

[0098] In this embodiment, the graph neural network model is trained on the graph data of each node in the sub-network where the client is located to obtain the model parameters of the trained graph neural network model, and the graph data is compressed to obtain compressed graph data, and then the compressed graph data and the model parameters are sent to the server, so that the server performs model training based on the global compressed graph of the target network constructed by the target compressed graph data sent by each client and the model parameters sent by each client to obtain the target gradient of the trained model. The server sends the target gradient to each client, so that the client updates the model parameters of the graph neural network model based on the target gradient. In this application, the server constructs a global compressed graph of the target network composed of each sub-network based on the compressed graph data sent by each client. The global compressed graph training model can bring a more consistent optimization direction to the model, thereby reducing the interference of heterogeneity caused by the structural and distribution differences of the sub-networks where each client is located, improving the accuracy of the model parameters of the model trained by the server, and improving the performance of the model trained by the client based on the model parameters sent by the server.

[0099] Reference Figure 3 , Figure 3 This is the second flow chart of the model training method provided by this application, based on Figure 1 In the illustrated embodiment, step S202 includes:

[0100] Step S301: compress the target graph data to obtain initial compressed graph data.

[0101] In this embodiment, the client compresses the target graph data to obtain compressed graph data, which is defined as the initial compressed graph data. For example, each data item in the target graph data is represented by a corresponding code, and the field composed of each code is the compressed graph data, which is used as the initial compressed graph data.

[0102] Step S302: input the initial compressed graph data into the second graph neural network model.

[0103] Step S303: predict the initial compressed graph data according to the second graph neural network model to obtain a target parameter, where the target parameter is used to indicate a loss parameter between the true label and the predicted label of the initial compressed graph data.

[0104] Step S304: Optimize the initial compressed graph data according to the target parameters to obtain target compressed graph data.

[0105] In order to ensure that the compressed graph data of the sub-network can still maintain high performance for classification tasks in the simplified structure, the attribute feature X′ of the compressed network is adjusted. i To make it in GNN iThe classification loss on the model is minimized. Specifically, the initial compressed graph data is input into the second graph neural network model, and the initial compressed graph data is predicted based on the second graph neural network model to obtain target parameters. The target parameters are used to indicate the loss parameters between the true labels of the initial compressed graph data and the predicted labels, thereby optimizing the initial compressed graph data through the target parameters to obtain target compressed graph data.

[0106] For example, the initial compressed graph data As GNN i The input of , the loss parameter calculated is:

[0107]

[0108] Where N' is the total number of nodes, C is the number of categories, and Y' i,j,c is the true label (usually 0 or 1) that the jth node belongs to category c, and is the probability that the jth node is predicted to belong to category c.

[0109] The client calculates the loss function for X′ i The gradient of , and update X′ according to the following formula i The target compressed graph data can be obtained. The target compressed graph data is:

[0110]

[0111] Where μ is the learning rate.

[0112] The above compression process is iterated d times to obtain the final compressed sub-network. For example, d can be set to 10.

[0113] In this embodiment, the initial compressed graph data is compressed by a graph neural network model to obtain target compressed graph data, and the target compressed graph data is uploaded to the server, thereby significantly reducing the amount of data transmission while retaining the information in the original graph as much as possible, so that the model in the server can learn the structure of the sub-network as much as possible, thereby improving the accuracy of the model parameters of the model trained by the server.

[0114] Reference Figure 4 , Figure 4 This is the third flow chart of the model training method provided in this application, based on Figure 3 In the embodiment shown, step S301 includes step S401 and step S402, and step S304 includes step S403 and step S404, wherein:

[0115] Step S401: Obtain initial compressed attribute features corresponding to attribute features in target graph data, and construct an initial compressed adjacency matrix based on the initial compressed attribute features.

[0116] In this embodiment, the target graph data includes node attribute features, and the node attribute features are compressed to obtain initial compressed attribute features. For example, the initial compressed attribute features are obtained by replacing the node attribute features with codes. In addition, other methods can be used to obtain the initial compressed attribute features.

[0117] Exemplarily, the client obtains the number of times its own graph neural network model has been trained, and the number of times the graph neural network model has been trained. If the number of times it has been trained is lower than the preset number of times, the attribute features in the target graph data are randomly initialized with a Gaussian distribution to obtain initial compressed attribute features. The preset number of times is, for example, 1 time, that is, when the graph neural network model needs to be trained for the first time, the initial compressed attribute features are obtained by sampling the attribute features in the target graph data with a standard normal distribution. It should be noted that after the attribute features are randomly initialized with a Gaussian distribution to obtain the initial compressed attribute features, the inner product of each initial compressed attribute feature is performed, and the tanh function is used to compress the initial compressed attribute features after the inner product to [-1,1], and then the self-connection is removed, so that the initial compressed adjacency matrix can be generated. If the number of times it has been trained is higher than or equal to the preset number of times, the target compressed attribute features obtained by optimizing the compressed graph data in the last training of the client's graph neural network model are used as the initial compressed attribute features. It should be noted that when each client performs model training for the first time, the model parameters of the graph neural network model are initialized, and based on the set fixed random seed, the initialized model parameters of the graph neural network model corresponding to each client are ensured to be the same.

[0118] For example, if the current client needs to perform the fifth model training, the target compression attribute feature optimized in the fourth model training is obtained. The optimized target compression attribute feature is the formula shown in the above embodiment:

[0119] After obtaining the initial compressed attribute features, an initial compressed adjacency matrix is ​​constructed based on the initial compressed attribute features.

[0120] In one example, the similarity between the initial compressed attribute features of adjacent nodes in the subnetwork is obtained, the weights of the edges connecting the adjacent nodes are determined based on the similarity, and an initial compressed adjacency matrix is ​​constructed based on the initial compressed attribute features and weights of the adjacent nodes. The initial compressed adjacency matrix is ​​generated, for example, as follows:

[0121]

[0122] Among them, X′ i,j , X′ i,k Represent the initial compressed attribute features of node j and node k respectively. For node j and node k, A′ i (X′i,j ,X′ i,k ) represents the initial compressed adjacency matrix.

[0123] In another example, an initial compressed adjacency matrix is ​​constructed based on the initial compressed attribute features, sparse functions, and weights of adjacent nodes. The initial compressed adjacency matrix is ​​generated, for example, as follows:

[0124]

[0125] The sparsification function is max(·).

[0126] By using the sparsification function to perform sparseness, the computational complexity and storage overhead can be reduced.

[0127] Step S402: constructing initial compressed graph data according to the labels, initial compressed attribute features and initial compressed adjacency matrix in the target graph data.

[0128] After constructing the initial compressed adjacency matrix, the initial compressed graph data is constructed using the labels in the target graph data, the initial compressed attribute features, and the initial compressed adjacency matrix. The labels in the initial compressed graph data are the same as the labels in the target graph data, and in subsequent steps, the labels remain unchanged, ensuring that the label distribution of the subnetwork is not disrupted.

[0129] Step S403: Optimizing the initial compression attribute features according to the target parameters to obtain target compression attribute features, and constructing a target compression adjacency matrix according to the target compression attribute features.

[0130] After the client inputs the initial compressed graph data into the second graph neural network model, it can obtain the target compression attribute features and construct the target compression adjacency matrix through the target compression attribute features. The way the target compression attribute features construct the target compression adjacency matrix is ​​the same as the way the initial compression attribute features construct the initial adjacency matrix. Please refer to the above instructions for details and will not repeat them here.

[0131] Step S404: construct target compressed graph data according to the labels in the target graph data, the target compressed attribute features, and the target compressed adjacency matrix.

[0132] After determining the target compression adjacency matrix, the target compression graph data is constructed based on the labels in the target graph data, the target compression attribute features, and the target compression adjacency matrix.

[0133] In this embodiment, a compressed adjacency matrix is ​​constructed by compressing attribute nodes, so that the compressed graph data can preserve the original information of the sub-network as much as possible, so that the server can obtain accurate gradients.

[0134] Reference Figure 5 , Figure 5This is the fourth flowchart of the model training method provided by this application, which is applied to the server. The model training method includes:

[0135] Step S501: Receive the target compressed graph data of the sub-network sent by each client and the model parameters of the trained graph neural network model in each client.

[0136] In this embodiment, the execution subject is the server, which communicates with multiple clients. The client is a user terminal in the sub-network, that is, the client can be a smart phone, smart watch, smart tablet, etc. The client has basic model training, response and data transmission functions, and can obtain sensitive information of itself and user devices (nodes) on the sub-network. Sensitive information includes connection structure, attribute characteristics and classification labels. Different sub-networks constitute the target network, and the target network represents a social system. The social system is, for example, a social network system, a medical system, a financial system, etc. There are private data in the fields of social networks, medical care, and finance. When training application models in these fields, the model can be trained by sub-graph federated learning, which realizes collaborative training of the model while protecting private data.

[0137] The social system consists of n sub-networks, represented as Each subnetwork It consists of multiple user terminal devices and is represented as Among them A i represents the adjacency matrix of the ith subnetwork, X i Represents the attribute characteristics of the user terminal in the sub-network, Y i Indicates the classification of user terminals. Each sub-network has a graph neural network model GNN locally i , the model parameter is W i , used to perform node classification tasks. Any user terminal in the sub-network can be used as a client, and the client is used to train the graph neural network model. Specifically, the client obtains the target graph data of each node in the sub-network. The target graph data of each node is {A i ,X i ,Y i}, each node is a user terminal in the sub-network.

[0138] The client trains the first graph neural network model corresponding to the client based on the target graph data of each node. The first graph neural network model corresponding to the client refers to the local graph neural network model of the sub-network where the client is located. The model after the client trains the first graph neural network model is defined as the second graph neural network model.

[0139] For example, the target graph data is input into the graph neural network model GNN iIn the example, the target graph data includes the adjacency matrix A i , the node's attribute feature X i and the true labels.

[0140] Through GNN i Perform forward propagation to generate the predicted category probability distribution for each node This process can be expressed as:

[0141]

[0142] Where W i are the parameters of the model.

[0143] The graph neural network model uses the cross entropy loss function to calculate the error between the predicted result and the true label. The cross entropy loss function is expressed as:

[0144]

[0145] Where N is the total number of nodes, C is the number of categories, and Y i,j,c is the true label (usually 0 or 1) that the jth node belongs to category c, and is the probability that the jth node is predicted to belong to category c.

[0146] After obtaining the loss function, calculate the loss function against the model parameters W i The gradient of the model is then used to update the model parameters. If the client is training the graph neural network model for the first time, the initialized local gradient is used to update the model parameters. The specific process is as follows:

[0147]

[0148] Where η is the learning rate. If the client is training the graph neural network model for the second or later round, it obtains the global optimization gradient sent by the server in the previous round and combines the global optimization gradient with the local gradient to update the model parameters. The process is as follows:

[0149]

[0150] Among them, α is a hyperparameter used to balance the local and global optimization directions, is the local gradient, Optimize the gradient globally.

[0151] It should be noted that each client needs to perform multiple rounds of training on its corresponding graph neural network model. For example, in the fifth round, each client needs to train its corresponding graph neural network model.

[0152] After training the second graph neural network model, the client obtains the model parameters W of the second graph neural network model and compresses the target graph data to obtain target compressed graph data. Exemplarily, each data item in the target graph data is represented by a corresponding code. The fields formed by each code are the compressed graph data, which serves as the target compressed graph data. The target compressed graph data represents the simplified graph structure and its attributes.

[0153] After obtaining the model parameters and the target compressed graph data, the client sends the model parameters and the target compressed graph data to the server.

[0154] In step S502 , each target compressed image data is aggregated to obtain a global compression graph of the target network, and each model parameter is aggregated to obtain a target model parameter. The target network is composed of the sub-networks where each client is located.

[0155] In step S503, the model in the server is trained according to the target model parameters and the global compression graph to obtain the target gradient, and the target gradient is sent to each client so that each client can update the model parameters of the graph neural network model based on the target gradient.

[0156] The server receives all model parameters {W1,…,W n} and target compressed graph data Finally, the model parameters of all clients are averaged and aggregated to obtain the global model Parameters The polymerization process can be expressed by the following formula:

[0157]

[0158] The server merges the target compressed graph data uploaded by all clients to generate a global graph data Server in the global model Calculate the loss function of the classification task

[0159]

[0160] Among them, n×N' is The total number of nodes, C is the number of categories, Y' j,c is the true label (usually 0 or 1) that the jth node belongs to category c, and is the probability that the jth node is predicted to belong to category c.

[0161] The server calculates the loss function For global model parameters The target gradient The target gradient represents the optimization direction of the global model. The server returns the target gradient to each client for the next round of local training and parameter updates.

[0162] It should be noted that the model structure of the server-side model is the same as that of the corresponding graph neural network model on the client.

[0163] The client receives the target gradient sent by the server and updates the model parameters of the second graph neural network model based on the target gradient, allowing the client to perform the next round of model training. That is, the client updates the second graph neural network model after the model parameter update to the first graph neural network model, and then returns to steps S201 to S204. When the number of times the client has trained the model reaches the set maximum number of iterations, the client does not need to perform the next round of model training.

[0164] The maximum number of iterations T varies based on the size of the dataset. For example, for a small dataset, T is set to 100, and for a large dataset, T is set to 200.

[0165] This embodiment is applicable to various complex network data consisting of entities and their relationships, especially subgraph federated learning for distributed training in scenarios involving privacy protection. Specific applications include but are not limited to social networks, financial transaction networks, transportation networks, bioinformatics, and knowledge graphs.

[0166] In these applications, data is usually stored in the form of graphs, where nodes represent entities, such as users in social networks, accounts in financial transactions, and stations in transportation systems, while edges represent the relationships between them, such as friend relationships, capital flows, road connections, etc.

[0167] For example, in social network analysis, nodes can represent users, while edges represent friendships or interactive behaviors (such as likes, comments, and shares). Because user data in social networks is highly private, traditional centralized learning methods are prone to privacy leaks. The solution provided in this embodiment allows social subgraphs in different regions or platforms to be trained locally, and only the compressed graph data and model parameters are uploaded to the cloud. This enables collaborative training under privacy protection, improving application scenarios such as personalized recommendations and social relationship prediction.

[0168] In this embodiment, the graph neural network model is trained on the graph data of each node in the sub-network where the client is located to obtain the model parameters of the trained graph neural network model, and the graph data is compressed to obtain compressed graph data, and then the compressed graph data and the model parameters are sent to the server, so that the server performs model training based on the global compressed graph of the target network constructed by the target compressed graph data sent by each client and the model parameters sent by each client to obtain the target gradient of the trained model. The server sends the target gradient to each client, so that the client updates the model parameters of the graph neural network model based on the target gradient. In this application, the server constructs a global compressed graph of the target network composed of each sub-network based on the compressed graph data sent by each client. Training the model through the global compressed graph can bring a more consistent optimization direction to the model, thereby reducing the interference of heterogeneity caused by the structural and distribution differences of the sub-networks where each client is located, improving the accuracy of the model parameters of the model trained by the server, and improving the performance of the model trained by the client based on the model parameters sent by the server.

[0169] Based on the above embodiments, the effects achieved by the present application are described through simulation experiments.

[0170] In simulation experiments, collaborative training was conducted on six real-world datasets, along with node classification tasks, to evaluate the average classification accuracy of each client in the social system. Each dataset was divided into 5, 10, and 20 subnetworks, which served as client subnetworks for federated learning. It should be noted that the nodes within each subnetwork were non-intersecting, ensuring the independence of the subnetworks. Three independent experiments were conducted on the subnetworks for each dataset, and the average classification accuracy of the clients was used as the final result.

[0171] Based on the above final results, on 5 of the 6 data sets, the model training method of the present application significantly outperforms most existing methods and demonstrates excellent performance. This not only shows that the model training method of this embodiment is very effective in compressing and transmitting graph data, but also that its advantages become more obvious as the number of client divisions increases. When the data set is divided into 5, 10, and 20 sub-networks as clients, this exacerbates the heterogeneity problem of the graph data. Specifically, as the number of sub-networks increases, the differences in graph structure and features between clients increase significantly, resulting in the poor performance of the traditional federated averaging algorithm in this scenario. The federated averaging algorithm fails to fully address the graph heterogeneity between clients, resulting in a significant decrease in the effectiveness of its global model when processing highly heterogeneous data.

[0172] However, for those methods designed specifically for heterogeneity problems, such methods partially alleviate the adverse effects caused by data heterogeneity by introducing additional regularization terms in the aggregation process. However, compared with such methods, the method provided by the present application performs better in processing heterogeneous graph data. By compressing the graph locally and uploading the compressed graph, not only the communication overhead is reduced, but also the differences in graph structure and features between different clients are effectively balanced, thereby better integrating information from different clients in the global model. Ultimately, the model training method of the present application achieves better classification performance under all partitioning conditions, especially when the number of clients is large and the heterogeneity is large, the advantage is more obvious.

[0173] Based on the same inventive concept, the present application also provides a client. Figure 6 The client provided in the embodiment of the present application is described in detail.

[0174] Figure 6 The figure is a structural block diagram of a client according to an exemplary embodiment.

[0175] like Figure 6 As shown, the client 600 may include:

[0176] A first acquisition module 610 is configured to acquire target graph data of each node in the sub-network where the client is located, and train the first graph neural network model corresponding to the client based on the target graph data of each node to obtain a second graph neural network model;

[0177] A second acquisition module 620 is used to obtain model parameters of the second graph neural network model and compress the target graph data to obtain target compressed graph data;

[0178] A sending module 630 is configured to send the model parameters and target compressed graph data to the server. The server performs model training based on the global compressed graph of the target network of the target compressed graph data sent by each client and the model parameters sent by each client to obtain a target gradient. The target network is composed of the subnetworks where each client is located. The model in the server has the same model structure as the graph neural network model corresponding to the client.

[0179] The receiving module 640 is used to receive the target gradient sent by the server and update the model parameters of the second graph neural network model according to the target gradient.

[0180] In one embodiment, the client 600 further includes:

[0181] Compressing the target graph data to obtain initial compressed graph data;

[0182] Inputting the initial compressed graph data into the second graph neural network model;

[0183] The initial compressed graph data is predicted according to the second graph neural network model to obtain a target parameter, which is used to indicate the loss parameter between the true label and the predicted label of the initial compressed graph data;

[0184] The initial compressed graph data is optimized according to the target parameters to obtain the target compressed graph data.

[0185] In one embodiment, the client 600 further includes:

[0186] Obtaining initial compressed attribute features corresponding to attribute features in the target graph data, and constructing an initial compressed adjacency matrix based on the initial compressed attribute features;

[0187] Construct initial compressed graph data based on the labels, initial compressed attribute features, and initial compressed adjacency matrix in the target graph data;

[0188] Optimize the initial compressed graph data according to the target parameters to obtain the target compressed graph data, including:

[0189] Optimizing the initial compression attribute features according to the target parameters to obtain the target compression attribute features, and constructing the target compression adjacency matrix according to the target compression attribute features;

[0190] Target compressed graph data is constructed according to the labels in the target graph data, target compressed attribute features, and target compressed adjacency matrix.

[0191] In one embodiment, the client 600 further includes:

[0192] Determine the similarity between the initial compressed attribute features of adjacent nodes in the sub-network;

[0193] The weights of the edges connecting adjacent nodes are determined according to image similarity, and the initial compressed adjacency matrix is ​​constructed according to the initial compressed attribute features, weights and sparsification functions of the adjacent nodes.

[0194] In one embodiment, the client 600 further includes:

[0195] Get the number of times the client's graph neural network model has been trained;

[0196] When the number of training times is less than the preset number, the attribute features in the target graph data are randomly initialized with Gaussian distribution to obtain the initial compressed attribute features;

[0197] When the number of training times is higher than or equal to the preset number of times, the target compression attribute features in the target compression graph data corresponding to the last training of the client's graph neural network model are used as the initial compression attribute features.

[0198] The client provided in the embodiment of the present application can achieve Figure 1-4 The various processes implemented in the illustrated embodiments achieve the same technical effects and will not be described again here to avoid repetition.

[0199] Based on the same inventive concept, this application also provides a server. Figure 7 The server provided in the embodiment of the present application is described in detail.

[0200] Figure 7 The figure is a structural block diagram of a server according to an exemplary embodiment.

[0201] like Figure 7 As shown, the server 700 may include:

[0202] Receiving module 710, used to receive the target compressed graph data of the sub-network sent by each client and the model parameters of the trained graph neural network model in each client;

[0203] Aggregation module 720, for aggregating each target compressed image data to obtain a global compression graph of the target network, and aggregating each model parameter to obtain target model parameters, where the target network is composed of the sub-networks where each client is located;

[0204] The training module 730 is used to train the model in the server according to the target model parameters and the global compression graph, obtain the target gradient, and send the target gradient to each client so that each client can update the model parameters of the graph neural network model based on the target gradient.

[0205] The server provided in the embodiment of the present application can achieve Figure 5 The various processes implemented in the illustrated embodiments achieve the same technical effects and will not be described again here to avoid repetition.

[0206] In some embodiments, as Figure 8 As shown, an embodiment of the present application also provides an electronic device 800, including a processor 801 and a memory 802, wherein the memory 802 stores a program or instruction that can be run on the processor 801, and when the program or instruction is executed by the processor 801, the various steps of the above-mentioned model training method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0207] It should be noted that the electronic devices in the embodiments of the present application include the above-mentioned mobile electronic devices and non-mobile electronic devices.

[0208] An embodiment of the present application also provides a computer-readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the various processes of the above-mentioned model training method embodiment are implemented and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0209] The processor is the processor in the electronic device in the above embodiment. The readable storage medium includes a computer-readable storage medium, such as a computer read-only memory, a random access memory, a magnetic disk, or an optical disk.

[0210] An embodiment of the present application provides a computer program product, which is stored in a storage medium. The program product is executed by at least one processor to implement the various processes of the above-mentioned model training method embodiment and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0211] It should be noted that, in this article, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the statement "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or device comprising the element. In addition, it should be noted that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in the opposite order according to the functions involved. For example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0212] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods of each embodiment of the present application.

[0213] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of this application, ordinary technicians in this field can also make many forms without departing from the purpose of this application and the scope of protection of the claims, all of which are within the protection of this application.

Claims

1. A model training method, characterized in that: Applied to the client, the model training method includes: Obtain target graph data of each node in the subnetwork where the client is located, and train the first graph neural network model corresponding to the client based on the target graph data of each node to obtain a second graph neural network model; Obtaining model parameters of the second graph neural network model, and compressing the target graph data to obtain target compressed graph data; Sending the model parameters and the target compressed graph data to the server, wherein the server constructs a global compressed graph of the target network based on the target compressed graph data sent by each client, and performs model training based on the global compressed graph and the model parameters sent by each client to obtain the target gradient of the trained model, wherein the target network is composed of the sub-networks where each client is located, and the model in the server has the same model structure as the graph neural network model corresponding to the client; Receive the target gradient sent by the server, and update the model parameters of the second graph neural network model according to the target gradient.

2. The model training method according to claim 1, characterized in that The compressing the target graph data to obtain target compressed graph data includes: Compressing the target graph data to obtain initial compressed graph data; Inputting the initial compressed graph data into the second graph neural network model; Predicting the initial compressed graph data according to the second graph neural network model to obtain a target parameter, where the target parameter is used to indicate a loss parameter between a true label and a predicted label of the initial compressed graph data; The initial compressed graph data is optimized according to the target parameters to obtain target compressed graph data.

3. The model training method according to claim 2, characterized in that The compressing the attribute features in the target graph data to obtain initial compressed graph data includes: Obtaining initial compressed attribute features corresponding to attribute features in the target graph data, and constructing an initial compressed adjacency matrix based on the initial compressed attribute features; Constructing initial compressed graph data according to the labels in the target graph data, the initial compressed attribute features, and the initial compressed adjacency matrix; Optimizing the initial compressed graph data according to the target parameters to obtain target compressed graph data includes: Optimizing the initial compression attribute characteristics according to the target parameters to obtain target compression attribute characteristics, and constructing a target compression adjacency matrix according to the target compression attribute characteristics; Target compressed graph data is constructed according to the labels in the target graph data, the target compressed attribute features, and the target compressed adjacency matrix.

4. The model training method according to claim 3, characterized in that The constructing an initial compressed adjacency matrix according to the initial compressed attribute features includes: Determining similarities between initial compressed attribute features of adjacent nodes in the subnetwork; The weights of the edges connecting the adjacent nodes are determined according to the image similarities, and an initial compressed adjacency matrix is ​​constructed according to the initial compressed attribute features of the adjacent nodes, the weights, and a sparsification function.

5. The model training method according to claim 3, characterized in that: The obtaining of the initial compressed attribute feature corresponding to the attribute feature in the target graph data includes: Obtaining the number of times the graph neural network model of the client has been trained; When the number of training times is less than a preset number, performing random initialization processing of Gaussian distribution on the attribute features in the target graph data to obtain initial compressed attribute features; When the number of training times is higher than or equal to the preset number of times, the target compression attribute features in the target compression graph data corresponding to the last training of the client's graph neural network model are used as the initial compression attribute features.

6. A model training method, characterized in that: Applied to the server, the model training method includes: Receive target compressed graph data of the sub-network sent by each client and model parameters of the trained graph neural network model in each of the clients; Aggregating each of the target compressed image data to obtain a global compression graph of a target network, and aggregating each of the model parameters to obtain a target model parameter, wherein the target network is composed of the sub-networks where each of the clients is located; According to the target model parameters and the global compression graph, the model in the server is trained to obtain the target gradient, and the target gradient is sent to each of the clients so that each client can update the model parameters of the graph neural network model based on the target gradient.

7. A client, characterized in that: include: A first acquisition module is used to obtain target graph data of each node in the sub-network where the client is located, and train the first graph neural network model corresponding to the client according to the target graph data of each node to obtain a second graph neural network model; a second acquisition module, configured to acquire model parameters of the second graph neural network model, and compress the target graph data to obtain target compressed graph data; A sending module, configured to send the model parameters and the target compressed graph data to a server, wherein the server performs model training based on the global compressed graph of the target network of the target compressed graph data sent by each client and the model parameters sent by each client to obtain a target gradient, wherein the target network is composed of the subnetworks where each client is located, and the model in the server has the same model structure as the graph neural network model corresponding to the client; A receiving module is used to receive the target gradient sent by the server and update the model parameters of the second graph neural network model according to the target gradient.

8. A server, characterized in that: include: A receiving module, configured to receive the target compressed graph data of the sub-network sent by each client and the model parameters of the trained graph neural network model in each of the clients; an aggregation module, configured to aggregate the target compressed image data to obtain a global compression graph of a target network, and to aggregate the model parameters to obtain target model parameters, wherein the target network is composed of the sub-networks where the clients are located; A training module is used to train the model in the server according to the target model parameters and the global compression graph, obtain the target gradient, and send the target gradient to each of the clients so that each client can update the model parameters of the graph neural network model based on the target gradient.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a program or instruction, and when the program or instruction is executed by the processor, the steps of the model training method according to any one of claims 1 to 6 are implemented.

Images