Federal map learning method and system for distributed graph data processing

Abstract

The invention belongs to the technical field of machine learning, and provides a federated graph learning method and system for distributed graph data processing, and the method comprises the steps: receiving an initialization parameter of a local graph neural network model; through graph neural network parameter decoupling, local graph neural network model parameters are decomposed into potential factors and linear combination coefficients thereof; performing feature interaction enhancement on the original features by using implicit feature intersection, and inputting generated high-order feature representation into a graph neural network to obtain updated node features; in combination with local graph data, local model updating is carried out, and the updated potential factors and linear combination coefficients thereof are uploaded to a server side for aggregation updating; and next round of local graph neural network model updating is carried out, loop iteration training is carried out until a preset stop condition is met, each client outputs a target personalized federated graph learning model, and related tasks are executed on the distributed graph data. According to the method, the influence of graph isomerism is reduced, and data privacy protection is enhanced.

Description

Technical Field

[0001] This invention belongs to the field of machine learning technology, specifically relating to a federated graph learning method and system for distributed graph data processing. Background Technology

[0002] Graphs, as a fundamental data form, are widely used to represent various complex relationships and interactions, such as user relationships in social networks, user-product interactions in e-commerce systems, and user transactions in financial networks. However, in these application scenarios, due to concerns about data privacy and conflicts of interest, graph data is often stored in a distributed manner and owned by different institutions, leading to the problem of data silos. For example, in multiple social apps, each user's personal information and interaction records are only held by the respective app's administrator, and other institutions cannot access this private data.

[0003] Federated learning, as a data privacy-preserving distributed deep learning framework, provides an effective solution to problems such as data silos, data discretization, and uneven data distribution. Some studies alleviate feature heterogeneity by predicting potential neighbor features across clients, while others address topological biases between clients by constructing a global pseudograph. Despite significant progress in federated graph learning, two key challenges remain: (1) Lack of joint consideration of features and topological biases: Node features in graph data are passed through the graph structure, potentially exacerbating graph heterogeneity between different clients and affecting model performance. This is especially true for the heterogeneity of graph structures on local clients, limiting the effectiveness of federated graph learning. (2) Non-personalized aggregation of client characteristics: Existing model aggregation methods typically employ average aggregation or weighted aggregation on the server side. While simple and effective, these methods fail to fully capture the personalized characteristics of clients. Weighted aggregation, in particular, while attempting to measure the correlation between clients through weighting, lacks bridging to global common information and carries the risk of data leakage. Summary of the Invention

[0004] The technical problem to be solved by this invention is to provide a federated graph learning method and system for distributed graph data processing, avoiding the lack of synergistic consideration of feature and topological bias, neglect of client-personalized feature aggregation, and the risk of data leakage.

[0005] The present invention includes: Firstly, the present invention provides a federated graph learning method for distributed graph data processing, comprising:

[0006] The client inputs local graph data and initialization parameters of the local graph neural network model. The local graph data includes nodes, edges and their feature information, and the initialization parameters include the weight matrix and bias vector of each layer.

[0007] Based on the initialization parameters, the parameters of the local graph neural network model are decomposed into latent factors and their linear combination coefficients by decoupling the local graph neural network model parameters.

[0008] By utilizing implicit feature cross-referencing, the original features of the local graph data are enhanced through feature interaction to generate higher-order feature representations. These higher-order feature representations are then input into the local graph neural network model to obtain updated node features. The higher-order feature representations include common features and unique features. The original features of the local graph data include user age and occupation.

[0009] Based on the updated node features and local graph data, the local model is updated, and the updated latent factors and their linear combination coefficients are uploaded to the server for aggregation and update.

[0010] The server receives the aggregated and updated model parameters and performs the next round of local graph neural network model updates. The training is iterated until a preset stopping condition is met. Each client outputs its target personalized federated graph learning model. The target personalized federated graph learning model is applied to the field of distributed graph data processing and is used to perform node classification, link prediction or recommendation tasks on distributed graph data. The aggregated and updated model parameters are a weighted sum of latent factors and their linear combination coefficients. The latent factors are predefined and shared by the server or globally.

[0011] Furthermore, the step of decomposing the local graph neural network model parameters into latent factors and their linear combination coefficients based on the initialization parameters by decoupling the local graph neural network model parameters includes:

[0012] Based on the initialization parameters, the parameters of the local graph neural network model are decomposed using basis decomposition techniques to obtain multiple latent factors and their corresponding linear combination coefficients.

[0013] The calculation formula for decoupling the parameters of the local graph neural network is as follows:

[0014]

[0015] in,

[0016] Where θ represents the set of latent factors, e m The coefficients of its linear combination, e m,b Let θ represent the coefficient of the b-th latent factor under client m. b Let B represent the b-th latent factor, and let R be a positive integer. B Represents a vector of length B from the real number field;

[0017] By imposing sparsity constraints on the linear combination coefficients, some latent factors are allowed to participate in the parameter reconstruction of the local graph neural network model, thus obtaining the latent factors and their linear combination coefficients.

[0018] The calculation formula that enables some latent factors to participate in the parameter reconstruction of the local graph neural network model is as follows:

[0019] L sp (e m )=||e m ||1

[0020] Where ||·||1 represents the L1 norm.

[0021] Furthermore, the implicit feature cross-validation is used to enhance the original features of the local graph data, generating higher-order feature representations. These higher-order feature representations are then input into the local graph neural network model to obtain updated node features, including:

[0022] By utilizing implicit feature cross-multiplication, local common features and unique features are multiplied separately to obtain higher-order interaction information;

[0023] The formulas for calculating the feature product of local common features and local unique features are as follows:

[0024]

[0025]

[0026] Where m represents the order of the clients, Represents common feature interactions, X m This indicates the existence of its original characteristics, Ψ u This represents a learnable parameter. This represents another learnable parameter. Indicates interaction of characteristics and features;

[0027] The Hilbert-Schmidt independence criterion is used to impose independence constraints on interactions of common features and interactions of unique features.

[0028] The calculation formula for applying the Hilbert-Schmidt independence criterion to constrain the independence of common feature interactions and unique feature interactions is as follows;

[0029]

[0030] Where HSIC(·,·) represents the Hilbert-Schmidt independence loss function;

[0031] Based on the independence constraint, the higher-order interaction information is concatenated with the original features and used as input to the graph neural network to obtain the updated node features.

[0032] The formula for calculating its splicing features is as follows:

[0033]

[0034] Here, || represents the concatenation operation. This represents the input features of a graph neural network.

[0035] Furthermore, the step of updating the local model based on the updated node features and the local graph data, and uploading the updated latent factors and their linear combination coefficients to the server for aggregation and updating, includes:

[0036] Based on the updated node features and the local graph data, the local model is updated, and the updated latent factors and their linear combination coefficients are uploaded to the server and used as a personalized representation of the local graph neural network model.

[0037] Aggregate updates are performed based on personalized representations derived from local graph neural network models.

[0038] Furthermore, the process involves receiving the aggregated and updated model parameters from the server, performing the next round of local graph neural network model updates, iterating training until a preset stopping condition is met, and each client outputting its target personalized federated graph learning model. This target personalized federated graph learning model is then applied to the field of distributed graph data processing to perform node classification, link prediction, or recommendation tasks on the distributed graph data, including:

[0039] The receiving server uses the latent factors and their linear combination coefficients after weighted aggregation based on the relevance measurement results as aggregation parameters. The relevance measurement results include weighting factors, cosine similarity, and cosine similarity matrix.

[0040] The local graph neural network model is updated based on the aggregation parameters to prepare for the next round of local training. The training is iterated until the preset stopping condition is met. Each client outputs its target personalized federated graph learning model and applies the target personalized federated graph learning model to the field of distributed graph data processing to perform node classification, link prediction or recommendation tasks on distributed graph data.

[0041] Furthermore, the relevance measurement results are obtained by generating virtual feature vectors that conform to Gaussian noise as input for each client and calculating the cosine similarity between different clients.

[0042] Based on the calculated cosine similarity values, construct a similarity matrix between clients;

[0043] Based on the cosine similarity matrix, the correlation between clients is quantified, a weighting factor for aggregation is generated, and this correlation information is transmitted to the server for server-side aggregation.

[0044] Furthermore, the preset stopping condition includes calculating a loss function based on the updated node features and local graph data, determining when the loss function value reaches a set range, and the corresponding constraints satisfied when the set range is reached.

[0045] Furthermore, after completing multiple rounds of updates to the local graph neural network model and reaching the preset stopping condition, the latest decoupled potential factors and their linear combination coefficients are periodically uploaded to the server for collaborative optimization of the global model.

[0046] Furthermore, during the collaborative optimization process of the global model, a global symbiotic space is constructed to align and coordinate the personalized representations of each client, map the personalized representations to a shared representation space, and impose parameter consistency constraints within this space to achieve parameter consistency among clients.

[0047] Secondly, the present invention provides a federated graph learning system for distributed graph data processing, comprising:

[0048] The receiving module is used to input local graph data and initialization parameters of the local graph neural network model into the client. The local graph data includes nodes, edges and their feature information, and the initialization parameters include the weight matrix and bias vector of each layer.

[0049] The decoupling module is used to decompose the parameters of the local graph neural network model into latent factors and their linear combination coefficients based on the initialization parameters by decoupling the parameters of the local graph neural network model.

[0050] The cross module is used to enhance the original features of the local graph data by implicit feature cross, generate higher-order feature representations, and input the higher-order feature representations into the local graph neural network model to obtain updated node features. The higher-order feature representations include common features and unique features. The original features of the local graph data include user age and occupation.

[0051] The upload module is used to update the local model based on the updated node features and the local graph data, and upload the updated latent factors and their linear combination coefficients to the server for aggregation and update.

[0052] The update module is used to receive the aggregated and updated model parameters from the server, perform the next round of local graph neural network model update, and iterate the training until the preset stopping condition is met. Each client outputs its target personalized federated graph learning model, which is then applied to the field of distributed graph data processing to perform node classification, link prediction, or recommendation tasks on the distributed graph data. The aggregated and updated model parameters are a weighted sum of latent factors and their linear combination coefficients. The latent factors are predefined and shared by the server or globally.

[0053] The beneficial effects of this invention are:

[0054] 1. This invention optimizes the feature and topological bias problem between clients by introducing a personalized latent factor collaborative aggregation method, thereby improving the cross-client collaborative training effect;

[0055] 2. This invention reduces the impact of graph heterogeneity by decoupling graph neural network parameters and using implicit feature cross-generation to generate higher-order feature representations, while better capturing the personalized features of the client.

[0056] 3. The aggregation method used in this invention transmits only relevant information, rather than raw data or sensitive features, which significantly reduces the risk of data leakage and enhances data privacy protection;

[0057] 4. This invention promotes parameter consistency among clients by introducing a global symbiotic space to coordinate personalized client representations, thereby optimizing the performance and training effect of the global model;

[0058] 5. Based on the decoupling of graph neural network parameters and implicit feature cross-interaction, this invention integrates personalized aggregation design, overcoming the two major shortcomings of existing technologies: lack of synergistic consideration of feature and topological deviation and neglect of client-side personalized feature aggregation, thus significantly improving the effectiveness and applicability of federated graph learning. Attached Figure Description

[0059] Appendix Figure 1 This is a flowchart illustrating a federated graph learning method for distributed graph data processing according to the present invention.

[0060] Appendix Figure 2 This is a schematic diagram of the architecture of a federated graph learning method for distributed graph data processing according to the present invention;

[0061] Appendix Figure 3 This is a block diagram of a federated graph learning system for distributed graph data processing according to the present invention. Detailed Implementation

[0062] As attached Figure 1 Appendix Figure 2As shown, the present invention includes a federated graph learning method for distributed graph data processing, specifically including:

[0063] S1. Input local graph data and initialization parameters of local graph neural network model into the client. The local graph data includes nodes, edges and their feature information, and the initialization parameters include the weight matrix and bias vector of each layer.

[0064] In one embodiment of the federated graph learning method, each client needs to receive local graph data and initialization parameters for the local graph neural network model. These initialization parameters are typically provided by the server or generated through a pre-training method. The local graph data includes nodes, edges, and their feature information, while the initialization parameters include basic information such as the weight matrices and bias vectors for each layer. These parameters serve as the starting point for model training, ensuring that each client has a consistent model framework at the beginning of training.

[0065] S2. Based on the initialization parameters, the parameters of the local graph neural network model are decomposed into latent factors and their linear combination coefficients by decoupling the local graph neural network model parameters;

[0066] In one embodiment, by decoupling the parameters, the complex parameters in the local graph neural network model are deconstructed into multiple latent factors and their linear combination coefficients, implicitly modeling and mitigating topological heterogeneity. The decoupling process uses mathematical methods to decompose the originally tightly connected parameters into multiple independently processable sub-components, making the graph neural network more flexible and efficient in subsequent training.

[0067] S21. Based on the initialization parameters, the parameters of the local graph neural network model are decomposed using basis decomposition techniques to obtain multiple latent factors and their corresponding linear combination coefficients.

[0068] The calculation formula for decoupling the parameters of the local graph neural network is as follows:

[0069]

[0070] in,

[0071] Where θ represents the set of latent factors, e m The coefficients of their linear combination are represented. e represents the tensor product calculation. m,b Let θ represent the coefficient of the b-th latent factor under client m. b Let B represent the b-th latent factor, and let R be a positive integer. B Represents a vector of length B from the real number field;

[0072] In one embodiment, in federated graph learning, latent factors represent certain latent features of nodes or edges, typically an abstract representation of the elements in the graph. Through decomposition, the local graph neural network model extracts important relationships or features from the low-dimensional space. Linear combination coefficients represent how these latent factors are combined into the actual model parameters, defining the weight of each factor in reconstructing the local graph neural network model parameters. These linear combination coefficients can be seen as a personalized representation of the client. Through decomposition, the graph neural network dynamically adjusts the influence of each latent factor during training, optimizing the graph's expressive power and learning performance.

[0073] Personalized representation is based on the unique characteristics of client-side local graph data. The model can learn the personalized information of each client's data and retain this personalized information in the model. Since each client's data may have different characteristics, personalized model representation helps to preserve these personalized characteristics.

[0074] S22. Apply sparsity constraints to the linear combination coefficients, allowing some latent factors to participate in the reconstruction of the local graph neural network model parameters, thus obtaining the latent factors and their linear combination coefficients.

[0075] The calculation formula that enables some latent factors to participate in the parameter reconstruction of the local graph neural network model is as follows:

[0076] L sp (e m )=||e m ||1

[0077] Where ||·||1 represents the L1 norm.

[0078] In one embodiment, when reconstructing parameters, the local graph neural network model prioritizes a subset of potential factors that are highly relevant to the local graph data and highlight personalized features, thereby improving the model's performance and personalization capabilities. This invention introduces sparsity constraints; the L1 norm ensures that the combination coefficients are as sparse as possible, further optimizing the linear combination coefficients to make the model emphasize local personalized features.

[0079] S3. Using implicit feature cross-validation, the original features of the local graph data are enhanced by feature interaction to generate higher-order feature representations. The higher-order feature representations are then input into the local graph neural network model to obtain updated node features. The higher-order feature representations include common features and unique features. The original features of the local graph data include user age and occupation.

[0080] In one embodiment, implicit feature cross-validation enhances the input features of a graph neural network. Implicit feature cross-validation is a feature enhancement technique that generates higher-order feature representations by performing cross-validation operations on the original data. These higher-order feature representations capture deeper relationships between the original features, providing richer input information to the graph neural network and improving the model's expressive and learning capabilities. Through implicit feature cross-validation, the graph neural network model obtains new features containing more relational information, which helps improve the performance of the graph neural network in complex tasks. Implicit feature cross-validation enhances common and unique features in the higher-order feature representations through high-order interactive enhancement, significantly improving the ability to capture complex feature relationships. This achieves synergistic optimization of node features and graph structure, alleviating the model heterogeneity problem caused by differences in features and topology between clients.

[0081] Among them, the original features of the local graph data are the features of the original data, which are some simple attributes that describe the attributes and categories of nodes, such as the user's age, occupation, etc.

[0082] S31. By utilizing implicit feature cross-multiplication, local common features and unique features are multiplied separately to obtain higher-order interaction information;

[0083] In graph neural networks and federated learning, higher-order interaction information refers to information represented by higher-order features, where data typically includes multiple types of features. By multiplying features, the original features are crossed and combined to obtain more complex information that captures deeper-level relationships.

[0084] Common features refer to features shared across all clients, exhibiting similarities across different clients. Unique features, on the other hand, are features unique to each client, reflecting the individualized data of each client. By performing feature products on both common and unique features, the model can extract the interaction information between different features. This calculation not only captures the interdependencies between features but also helps the model learn more complex, higher-order feature representations, thereby enhancing the model's expressive power.

[0085] The formulas for calculating the feature product of local common features and local unique features are as follows:

[0086]

[0087] Where m represents the order of the clients, Represents common feature interactions, X m This indicates the existence of its original characteristics, Ψ u Ψ represents a learnable parameter. m s This represents another learnable parameter. This indicates the interaction of characteristics and features.

[0088] S32. The Hilbert-Schmidt independence criterion is used to impose independence constraints on the interactions of common features and the interactions of unique features;

[0089] In one embodiment, the Hilbert-Schmidt independence criterion is a statistical method for measuring the independence between random variables. It is commonly used to ensure that interactions between features in high-dimensional data do not interfere with each other due to excessive interaction. Independence constraints ensure that the effects of common feature interactions and unique feature interactions are independent and do not interfere with each other, reducing model complexity and guaranteeing the independence of each type of feature in the final model. Independence constraints prevent the model from over-relying on a particular type of feature or interaction, improving the model's robustness and accuracy.

[0090] The calculation formula for applying the Hilbert-Schmidt independence criterion to constrain the independence of common feature interactions and unique feature interactions is as follows;

[0091]

[0092] Where HSIC(·,·) represents the Hilbert-Schmidt independence loss function.

[0093] S33. Based on the results of the independence constraint, the higher-order interaction information is concatenated with the original features as the input to the graph neural network to obtain the updated node features.

[0094] In this process, concatenation integrates features from different sources and uses them as input to the neural network for subsequent graph structure learning. This allows for a unified modeling of the relationships and dependencies between different features, while simultaneously improving the network's sensitivity to various types of information and its learning ability.

[0095] The formula for calculating its splicing features is as follows:

[0096]

[0097] Here, || represents the concatenation operation. This represents the input features of a graph neural network.

[0098] In one embodiment, the higher-order feature representation is input into a graph neural network for message propagation and feature updating to obtain updated node features. Message propagation enables learning of graph data through information exchange between nodes. Nodes update their own state by receiving information from their neighbors, capturing spatial relationships within the graph data. Feature updating, after message propagation, involves the graph neural network updating its own features based on the received neighbor information, optimizing the representation of each node and gradually improving model performance. Through this method, the graph neural network learns the implicit relationships between nodes in the graph structure and optimizes the feature representation of each node through continuous information exchange.

[0099] S4. Based on the updated node features and the local graph data, perform local model updates, and upload the updated latent factors and their linear combination coefficients to the server for aggregation and updates.

[0100] In one embodiment, local model updates are performed by combining updated node features with local graph data. After local training on each client, latent factors and their linear combination coefficients are uploaded to the server as personalized representations for each client. A weighted aggregation update mechanism is designed based on these representations, fully utilizing the personalized features of different clients to obtain a global model. This invention overcomes the limitations of traditional averaging or simple weighted aggregation, achieving personalized adjustments during the aggregation process. This ensures the generalization ability of the global model while reflecting the uniqueness of each client, effectively mitigating the performance degradation caused by non-personalized aggregation. The aggregation method used only transmits relevant information, rather than raw data or sensitive features, significantly reducing the risk of data leakage and enhancing data privacy protection.

[0101] S41. Based on the updated node features and the local graph data, update the local model, upload the updated latent factors and their linear combination coefficients to the server, and use them as a personalized representation of the local graph neural network model.

[0102] In one embodiment, latent factors and their linear combination coefficients represent the weights of each latent factor in the model, reflecting the personalized characteristics of the local graph neural network model. The client uploads the latent factors and their linear combination coefficients to the server. These coefficients represent the features of the local graph neural network model and are used to further aggregate and update the global model. Then, through the uploaded latent factors and their linear combination coefficients, the server can understand the unique characteristics or preferences of each client for personalized modeling. In other words, the client uploads latent factors and their linear combination coefficients containing its personalized characteristics to the server, which uses this information to aggregate them, thereby forming a global model containing personalized information.

[0103] S42. Based on the personalized representation of the local graph neural network model, perform aggregate updates.

[0104] In one embodiment, on the server side, all latent factors and their linear combination coefficients uploaded by clients are aggregated. Weights are applied based on the correlation between clients to ensure that each client's personalized information is effectively integrated into the global model. Through aggregation updates, the server integrates the personalized representations of all clients into a single global model, improving the overall model's performance. This allows the global model to not only reflect the personalized characteristics of each client but also to synthesize information from all clients, thus enhancing overall performance. The weights and linear combination coefficients corresponding to each latent factor are maintained as a shared global symbiotic space. Laplace regularization is applied to the latent factor coefficients based on the client correlation matrix to promote consistency and coordination of model parameters across different clients.

[0105] Specifically, taking client m as an example, the server will receive the locally trained parameter set in round t.

[0106]

[0107] in, This represents the feature interaction parameters uploaded by client m in round t. Represents common feature interaction parameters. Indicates local latent factors, To characterize the combination coefficients of latent factors.

[0108] Then, in order to perform personalized aggregation for different parameters, the client-side relevance measurement operation is executed as follows:

[0109] (1) Due to implicit protection considerations, access to the graph data of each client is strictly prohibited. Therefore, a random probing strategy is proposed to reveal the correlations between clients. The server generates random probe vectors that follow a Gaussian distribution. The input to the feature cross-module in each client is then used to obtain the correlation between different clients through the metric function g(·,·)=max(0,cos(·,·)).

[0110] in, Indicates from Gaussian distribution The random probe vector sampled from the sample, g(·,·)=max(0,cos(·,·)) represents the correlation measurement function, and cos(·,·) represents the cosine similarity measurement function.

[0111] Among them, the correlation R between client m and client n mn The calculation formula is as follows:

[0112]

[0113] Where g(·,·) represents the correlation measurement function. ⊙ represents the expectation of the intermediate variable, and ⊙ represents element-wise product.

[0114] (2) After obtaining the correlation matrix R, the client model can perform weighted aggregation based on the correlation coefficients. The update formula for the unique feature interaction parameters of client m is as follows:

[0115]

[0116] Among them, R mn This represents the correlation between client m and client n. This represents the summation symbol that iterates through all clients. This represents the feature interaction parameters after the (t+1)th round update.

[0117] (3) For each client's latent factor coefficient e m =E m,: For m = 1, 2, ..., M, we treat them as client representations and construct the Laplace matrix L = IR between clients based on the correlation matrix R. We then use Laplace regularization to constrain the smoothness of the latent factors, calculated as follows:

[0118] L server (E)=tr(E T LE)

[0119] Among them, L server (E) denotes the latent factor smoothness, E represents the matrix composed of latent factor coefficients, T represents the matrix transpose operation, tr(·) represents the trace of the matrix, and I represents the identity matrix.

[0120] (4) The shared feature interactions and latent factors of different clients need to be consistent. The former ensures that the feature interactions learned by each client are shared, while the latter maintains the co-existence space of the graph neural network across clients. Therefore, its updates are weighted according to the prior distribution of data for each client.

[0121] For example, given a client m, its prior data distribution is: The calculation formula is as follows:

[0122]

[0123] in, ω represents the number of nodes in the client m set. m This indicates the weight of client m in the total participation of the federated training, [·] t+1This indicates the (t+1)th round.

[0124] S5. Receive the aggregated and updated model parameters from the server, perform the next round of local graph neural network model update, iterate the training until the preset stopping condition is met, and each client outputs its target personalized federated graph learning model. Apply the target personalized federated graph learning model to the field of distributed graph data processing to perform node classification, link prediction or recommendation tasks on distributed graph data. The aggregated and updated model parameters are a weighted sum of latent factors and their linear combination coefficients.

[0125] In one embodiment, the server performs aggregation updates based on the latent factors and linear combination coefficients uploaded by the clients. The aggregated model parameters are obtained through weighted summation, with each client's contribution adjusted according to its relevance or importance to other clients. The server returns the aggregated parameters to each client so they can update their local graph neural network models. After this round of updates, the clients continue local training based on the updated model parameters, entering the next round of iterative optimization. This iterative training continues until a preset stopping condition is met, at which point each client outputs its target personalized federated graph learning model.

[0126] The preset stopping conditions include calculating a loss function based on the updated node features and local graph data, determining when the loss function value reaches a set range, and satisfying corresponding constraints when the set range is reached.

[0127] The loss function measures the difference between the current model's prediction and the actual label. During training, the loss function guides model updates, ensuring that the prediction is as close as possible to the true label. For graph neural networks, loss functions include node classification loss, graph structure loss, etc.

[0128] In one embodiment, each client trains its local neural network model based on its own local map data and calculates a loss function to optimize the local model. Based on the calculated loss function, the client updates the model parameters using backpropagation. The updated parameters improve the model's performance in the next training iteration, optimizing the local training process. The local training process is accomplished using gradient descent or other optimization algorithms to minimize the loss function and improve the model's performance on its local map data.

[0129] S51. The receiving server uses the latent factors and their linear combination coefficients after weighted aggregation based on the relevance measurement results as aggregation parameters, wherein the relevance measurement results include weighting factors, cosine similarity, and cosine similarity matrix.

[0130] In one embodiment, to better reflect the similarities and differences between clients and ensure that the global model maintains global consistency while preserving personalized features, the server performs a weighted aggregation of the latent factors and linear combination coefficients uploaded by all clients based on the correlation measurement results between clients. The aggregated parameters are derived from the personalized feature information of each client and represent the update of the global model.

[0131] Latent factors are predefined and shared by the server or globally, forming a fixed set. Each client uses the same set of latent factors during training. The client does not modify the latent factors themselves, but only updates the linear combination coefficients to reflect the personalized characteristics of the local graph data. The client only uploads the latent factors and their linear combination coefficients for server-side aggregation. The complete model after server-side aggregation combines the above coefficients with the fixed set of latent factors to obtain the complete aggregated model parameters. Therefore, the updated model parameters after aggregation are a weighted summary of the latent factors and their linear combination coefficients.

[0132] The relevance measurement results primarily reflect the similarity between different clients.

[0133] Obtaining the correlation measurement results includes:

[0134] (i) Calculate the cosine similarity between different clients by generating virtual feature vectors that conform to Gaussian noise as input for each client;

[0135] In one embodiment, virtual feature vectors conforming to Gaussian noise are generated, providing each client with an independent input for comparing similarity between different clients. These virtual features are not directly derived from the clients' original data but are generated through random noise. This adds randomness to the measurement of client similarity, preventing the model from relying too heavily on noise or bias from the real data.

[0136] (ii) Construct a similarity matrix between clients based on the calculated cosine similarity values;

[0137] In one embodiment, the similarity between clients is calculated based on the generated virtual feature vectors. The similarity is measured using cosine similarity, which reflects the angular relationship between two vectors in vector space. The closer the value is to 1, the more similar the features of the two clients are.

[0138] (iii) Based on the cosine similarity matrix, the correlation between clients is quantified, a weighting factor for aggregation is generated, and the correlation information is transmitted to the server. This correlation information is used for aggregation on the server side.

[0139] In one embodiment, a cosine similarity matrix is ​​constructed between clients, which is a two-dimensional array where each element represents the similarity between two clients. Each row and column in the matrix represents a client, and the value at each position represents the cosine similarity value between the corresponding clients. The cosine similarity matrix comprehensively reflects the similarity relationships between different clients, quantifying the correlation between clients. The similarity values ​​are then transformed into a concrete metric that can be used for model aggregation.

[0140] The generated weighting factors reflect each client's contribution to the overall model update, typically assigning higher weights to clients with higher similarity. These weighting factors and relevance information are transmitted to the server, which uses the weighting factors to perform weighted aggregation of models from different clients based on the received relevance information. This ensures that model fusion across different clients better reflects the unique characteristics of each client.

[0141] S52. Update the local graph neural network model according to the aggregation parameters, complete the preparation for the next round of local training, and iterate the training until the preset stopping condition is met. Each client outputs its target personalized federated graph learning model and applies the target personalized federated graph learning model to the field of distributed graph data processing to perform node classification, link prediction or recommendation tasks on distributed graph data.

[0142] In one embodiment, the aggregation parameters returned by the server include latent factors and their linear combination coefficients, reflecting the correlation between clients and the weighted aggregation results. Therefore, the client uses these aggregation parameters to update its local graph neural network model. The update process involves adjusting the local graph neural network model based on global information from the server, allowing it to better integrate with the global optimization direction. Local training is performed based on the updated graph neural network model, further improving the training effect. After the model update is complete, the client prepares for the next round of local training based on the new model parameters. This iterative training continues until a preset stopping condition is met. Each client outputs its target personalized federated graph learning model, which is applied in the field of distributed graph data processing to perform node classification, link prediction, or recommendation tasks on distributed graph data.

[0143] After completing multiple rounds of updates to the local graph neural network model and reaching the preset stopping condition, the latest decoupled potential factors and their linear combination coefficients are periodically uploaded to the server for collaborative optimization of the global model.

[0144] In one embodiment, each client trains locally for multiple rounds to achieve a personalized target federated graph learning model based on its own data. The training progress of different clients may differ. The parameters of their target personalized federated graph learning models are uploaded to the server. The server weights and aggregates the received client model parameters to form a dynamically evolving global model. After updating the local graph neural network model, the latest decoupled latent factors and their linear combination coefficients are periodically uploaded to the server for collaborative optimization and updating of the global model. This global model continuously integrates the personalized knowledge of each client, achieving global optimization and performance improvement in federated collaborative learning.

[0145] During the collaborative optimization process of the global model, a global symbiotic space is constructed to align and coordinate the personalized representations of each client. The personalized representations are then mapped to a shared representation space, and parameter consistency constraints are applied within this space to achieve parameter consistency among clients.

[0146] In one embodiment, to achieve parameter consistency across different clients and improve the training performance of the global model, a global symbiotic space is constructed. After each client completes multiple rounds of training of its local graph neural network model, its personalized representation is extracted from the model parameters. These personalized representations are then input into a predefined mapping function, transforming the representation vectors of different clients into a global symbiotic space of the same dimension. Parameter consistency constraints are imposed on the representations of each client, and the constrained representations are fed back to the server-side aggregation process. This ensures that the aggregated latent factors and their linear combination coefficients maintain both global consistency and local personalization, thus constructing the global symbiotic space and achieving better global model performance. The personalized representation, a vector form composed of latent factors and linear combination coefficients, characterizes the unique features and common information of that client.

[0147] This invention optimizes the feature and topological bias issues between clients by introducing a personalized latent factor collaborative aggregation method, thereby improving the cross-client collaborative training effect. By decoupling graph neural network parameters and using implicit feature cross-generation to generate higher-order feature representations, it reduces the impact of graph heterogeneity and better captures the personalized features of clients. By integrating personalized aggregation design based on graph neural network parameter decoupling and implicit feature cross-generation, it overcomes the two major shortcomings of existing technologies: lack of consideration for the collaborative relationship between features and topological bias and neglect of client-specific feature aggregation, significantly improving the effect and applicability of federated graph learning.

[0148] Appendix Figure 3 This is a block diagram of a federated graph learning system for distributed graph data processing according to the present invention.

[0149] Based on the same concept, the present invention also provides a federated graph learning system for distributed graph data processing, comprising:

[0150] The receiving module 31 is used to input local graph data and initialization parameters of the local graph neural network model into the client. The local graph data includes nodes, edges and their feature information, and the initialization parameters include the weight matrix and bias vector of each layer.

[0151] The decoupling module 32 is used to decompose the parameters of the local graph neural network model into latent factors and their linear combination coefficients based on the initialization parameters by decoupling the parameters of the local graph neural network model.

[0152] Cross module 33 is used to enhance the original features of the local graph data by implicit feature cross-interaction, generate higher-order feature representations, and input the higher-order feature representations into the local graph neural network model to obtain updated node features. The higher-order feature representations include common features and unique features. The original features of the local graph data include user age and occupation.

[0153] Upload module 34 is used to update the local model based on the updated node features and the local graph data, and upload the updated latent factors and their linear combination coefficients to the server for aggregation and update.

[0154] The update module 35 is used to receive the aggregated and updated model parameters through the server, perform the next round of local graph neural network model update, iterate training until the preset stopping condition is met, and each client outputs its target personalized federated graph learning model. The target personalized federated graph learning model is applied to the field of distributed graph data processing to perform node classification, link prediction or recommendation tasks on distributed graph data. The aggregated and updated model parameters are a weighted sum of latent factors and their linear combination coefficients. The latent factors are predefined and shared by the server or globally.

Claims

1. A federated graph learning method for distributed graph data processing, characterized in that, include: The client inputs local graph data and initialization parameters of the local graph neural network model. The local graph data includes nodes, edges and their feature information, and the initialization parameters include the weight matrix and bias vector of each layer. Based on the initialization parameters, the parameters of the local graph neural network model are decomposed into latent factors and their linear combination coefficients by decoupling the local graph neural network model parameters. By utilizing implicit feature cross-referencing, the original features of the local graph data are enhanced through feature interaction to generate higher-order feature representations. These higher-order feature representations are then input into the local graph neural network model to obtain updated node features. The higher-order feature representations include common features and unique features. The original features of the local graph data include user age and occupation. Based on the updated node features and the local graph data, the local model is updated, and the updated latent factors and their linear combination coefficients are uploaded to the server for aggregation and update. The server receives the aggregated and updated model parameters and performs the next round of local graph neural network model updates. The training is iterated until a preset stopping condition is met. Each client outputs its target personalized federated graph learning model and applies it to the field of distributed graph data processing to perform node classification, link prediction, or recommendation tasks on distributed graph data. The aggregated and updated model parameters are a weighted sum of latent factors and their linear combination coefficients. The latent factors are predefined and shared by the server or globally.

2. The federated graph learning method as described in claim 1, characterized in that, The step of decomposing the local graph neural network model parameters into latent factors and their linear combination coefficients based on the initialization parameters includes: Based on the initialization parameters, the parameters of the local graph neural network model are decomposed using basis decomposition techniques to obtain multiple latent factors and their corresponding linear combination coefficients. The calculation formula for decoupling the parameters of the local graph neural network is as follows: Among them, e m =[e m,1 ,e m,2 ,…,e m,B ]∈R B Where θ represents the set of latent factors, e m This represents the coefficients of its linear combination. e represents the tensor product calculation. m,b Let θ represent the coefficient of the b-th latent factor under client m. b Let B represent the b-th latent factor, and let R be a positive integer. B Represents a vector of length B from the real number field; By imposing sparsity constraints on the linear combination coefficients, some latent factors are allowed to participate in the parameter reconstruction of the local graph neural network model, thus obtaining the latent factors and their linear combination coefficients. The formula for calculating the sparse constraint loss is as follows: THE sp (And m )=||e m ||1 Where ||·||1 represents the L1 norm.

3. The federated graph learning method as described in claim 1, characterized in that, The method of using implicit feature cross-validation to enhance the original features of the local graph data generates higher-order feature representations, which are then input into the local graph neural network model to obtain updated node features, including: By utilizing implicit feature cross-multiplication, local common features and unique features are multiplied separately to obtain higher-order interaction information; The formulas for calculating the feature product of local common features and local unique features are as follows: Where m represents the order of the clients, Represents common feature interactions, X m This indicates the existence of its original characteristics, Ψ u This represents a learnable parameter. This represents another learnable parameter. Indicates interaction of characteristics and features; The Hilbert-Schmidt independence criterion is used to impose independence constraints on interactions of common features and interactions of unique features. The calculation formula for applying the Hilbert-Schmidt independence criterion to constrain the independence of common feature interactions and unique feature interactions is as follows; Where HSIC(·,·) represents the Hilbert-Schmidt independence loss function; Based on the independence constraint, the higher-order interaction information is concatenated with the original features and used as input to the graph neural network to obtain the updated node features. The formula for calculating its splicing features is as follows: Here, ‖ represents the splicing operation. This represents the input features of a graph neural network.

4. The federated graph learning method as described in claim 1, characterized in that, The step of updating the local model based on the updated node features and the local graph data, and uploading the updated latent factors and their linear combination coefficients to the server for aggregation and updating, includes: Based on the updated node features and the local graph data, the local model is updated, and the updated latent factors and their linear combination coefficients are uploaded to the server and used as a personalized representation of the local graph neural network model. Aggregate updates are performed based on personalized representations derived from local graph neural network models.

5. The federated graph learning method as described in claim 1, characterized in that, The process involves receiving the aggregated and updated model parameters from the server, performing the next round of local graph neural network model updates, and iteratively training until a preset stopping condition is met. Each client outputs its target personalized federated graph learning model, which is applied in the field of distributed graph data processing and used to perform node classification, link prediction, or recommendation tasks on distributed graph data, including: The receiving server uses the latent factors and their linear combination coefficients after weighted aggregation based on the relevance measurement results as aggregation parameters. The relevance measurement results include weighting factors, cosine similarity, and cosine similarity matrix. The local graph neural network model is updated based on the aggregation parameters to prepare for the next round of local training. The training is iterated until the preset stopping condition is met. Each client outputs its target personalized federated graph learning model and applies the target personalized federated graph learning model to the field of distributed graph data processing to perform node classification, link prediction or recommendation tasks on distributed graph data.

6. The federated graph learning method as described in claim 5, characterized in that, Obtaining the correlation measurement results includes: By generating virtual feature vectors that conform to Gaussian noise as input for each client, the cosine similarity between different clients is calculated. Based on the calculated cosine similarity values, construct a similarity matrix between clients; Based on the cosine similarity matrix, the correlation between clients is quantified, a weighting factor for aggregation is generated, and this correlation information is transmitted to the server for server-side aggregation.

7. The federated graph learning method as described in claim 1, characterized in that, The preset stopping conditions include calculating a loss function based on the updated node features and local graph data, determining when the loss function value reaches a set range, and the corresponding constraints that must be met when the set range is reached.

8. The federated graph learning method as described in claim 1, characterized in that, After completing multiple rounds of updates to the local graph neural network model and reaching the preset stopping condition, the latest decoupled potential factors and their linear combination coefficients are periodically uploaded to the server for collaborative optimization of the global model.

9. The federated graph learning method as described in claim 8, characterized in that, In the collaborative optimization process of the global model, a global symbiotic space is constructed to align and coordinate the personalized representations of each client. The personalized representations are then mapped to a shared representation space, and parameter consistency constraints are applied within this space to achieve parameter consistency among clients.

10. A federated graph learning system for distributed graph data processing, characterized in that, include: The receiving module (31) is used to input local graph data and initialization parameters of the local graph neural network model into the client. The local graph data includes nodes, edges and their feature information, and the initialization parameters include the weight matrix and bias vector of each layer. The decoupling module (32) is used to decompose the parameters of the local graph neural network model into latent factors and their linear combination coefficients by decoupling the parameters of the local graph neural network model based on the initialization parameters. The cross module (33) is used to enhance the original features of the local graph data by using implicit feature cross to generate higher-order feature representations and input the higher-order feature representations into the local graph neural network model to obtain updated node features. The higher-order feature representations include common features and unique features. The original features of the local graph data include user age and occupation. The upload module (34) is used to update the local model based on the updated node features and the local graph data, and upload the updated potential factors and their linear combination coefficients to the server for aggregation and update. The update module (35) is used to receive the aggregated updated model parameters through the server, perform the next round of local graph neural network model update, iterate training until the preset stopping condition is met, and each client outputs its target personalized federated graph learning model. The target personalized federated graph learning model is applied to the field of distributed graph data processing to perform node classification, link prediction or recommendation tasks on distributed graph data. The aggregated updated model parameters are a weighted sum of latent factors and their linear combination coefficients. The latent factors are predefined and shared by the server or globally.

Images