Federated graph learning method and system based on feature space construction and sharing

Abstract

This invention discloses a federated graph learning method and system based on feature space construction and sharing, designed to address the issues of feature space heterogeneity, harmful knowledge integration, and insufficient personalized performance in federated graph learning in a cross-domain Internet of Things environment. The method includes: constructing a globally consistent feature space and achieving node feature alignment through random walk feature generation and standardization; designing a client-side hybrid feature encoder (CHFE) to combine global features (MLP encoding) and local features (GCN encoding) to generate a hybrid representation; proposing an independent adaptive aggregation strategy (IAA) to dynamically calculate aggregation weights based on domain similarity, model similarity, and gradient similarity; and introducing a personalized regularization mechanism (PPT) to balance global collaboration and local optimization. This method significantly improves the model's convergence speed and generalization ability under heterogeneous data, reduces communication overhead, and is suitable for privacy-sensitive scenarios such as healthcare and industry.

Description

Technical Field

[0001] The present invention belongs to the technical field of machine learning, and specifically relates to a federated graph learning method and system based on feature space construction and sharing. Background Art

[0002] In recent years, the widespread application of the Internet of Things (IoT) in industrial manufacturing, smart healthcare, intelligent transportation, and other fields has generated massive amounts of heterogeneous data from multiple sources. Graph data, with its ability to express structured relationships, has become a core vehicle for describing complex systems such as device interactions and social networks. However, due to data privacy regulations (such as GDPR) and communication bandwidth limitations, traditional centralized machine learning struggles to support cross-domain collaborative modeling. Federated Learning (FL) provides a privacy-safe collaborative training solution for distributed environments by leveraging a mechanism of "local data storage and shared model parameters."

[0003] Federated Graph Learning (FGL) has emerged as an emerging research direction for processing graph data in IoT scenarios. Existing FGL technologies face significant limitations in practical cross-domain IoT applications. Their core assumption requires consistent feature spaces and homogeneous graph structures across all clients. However, in real-world scenarios such as healthcare and industry, the severe heterogeneity of feature spaces (e.g., mismatches in node attribute dimensions and magnitudes) caused by differences in device types, industry standards, and regions forces traditional aggregation strategies based on parameter averaging (e.g., FedAvg) to integrate incompatible knowledge, leading to the risk of harmful knowledge integration—manifested by degraded global model generalization performance, reduced local model discriminability, and even collaborative failure. Furthermore, the inherent non-IID nature of IoT data and the cross-domain semantic gap further exacerbate the conflict between global collaborative optimization and local personalized needs. Furthermore, inefficient aggregation mechanisms in heterogeneous environments lead to slow training convergence and increased communication overhead, hindering the practical deployment of resource-constrained IoT devices. Current technologies lack systematic solutions for dynamic feature space alignment, knowledge compatibility determination, and the personalization-generalization trade-off.

[0004] Existing solutions lack systematic mechanisms for dynamically aligning heterogeneous feature spaces, filtering harmful knowledge, and balancing personalization and generalization. This results in insufficient stability and practicality for cross-domain IoT federated graph learning. Therefore, a new federated graph learning framework is urgently needed that can achieve feature space sharing, differentiated aggregation, and local adaptive optimization. Summary of the Invention

[0005] The main purpose of the present invention is to overcome the shortcomings and deficiencies of the existing technology and provide a federated graph learning method based on feature space construction and sharing. By constructing a globally consistent feature space, designing a differentiated aggregation mechanism and personalized optimization strategy, efficient collaborative modeling and personalized performance improvement are achieved in complex heterogeneous environments.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions:

[0007] In a first aspect, the present invention provides a federated graph learning method based on feature space construction and sharing, comprising the following steps:

[0008] For each client's graph data, a normalized random walk matrix is ​​generated based on the adjacency matrix and degree matrix. Multi-order random walk features are extracted to form a unified global feature representation. Feature normalization is then used to eliminate feature distribution differences between different clients and construct a shared global feature space.

[0009] For each node, the global feature and the original local feature are input into the global feature encoder and the local feature encoder respectively, and the hybrid feature is generated by feature splicing;

[0010] Calculate the domain similarity matrix, model similarity matrix, and update direction similarity matrix between clients, construct a global feature relationship matrix based on the domain similarity matrix and the model similarity matrix, and construct a local feature relationship matrix based on the model similarity matrix and the update direction similarity matrix. Perform independent aggregation on the global feature relationship matrix and the local feature relationship matrix, transform the aggregation weight optimization into a constrained quadratic programming problem, and dynamically update the aggregation weight based on the similarity.

[0011] A proximal regularization term is added to the loss function of local client training to constrain the degree of deviation between the local model parameters and the aggregated model parameters sent by the server. Based on the objective function containing the regularization term, the gradient descent method is used to update the local model parameters. The regularization strength is dynamically adjusted based on the heterogeneity of client data to achieve a balance between collaboration and personalization.

[0012] As a preferred technical solution, the global feature space is constructed as follows:

[0013] Based on the adjacency matrix A i Sum degree matrix D i Compute the normalized random walk matrix ;

[0014] Select the random walk matrix k The global features of the nodes are constructed by random walk diagonal elements. The global features are generated using a unified rule on all clients to ensure that different clients can be mapped to a consistent global feature space even if their original feature spaces are different.

[0015] The generated global feature vector is normalized to eliminate the feature distribution differences between different clients.

[0016] As a preferred technical solution, in the client-side hybrid feature encoding:

[0017] The global feature encoder adopts a multi-layer perceptron (MLP) structure to extract the deep semantic representation of global features layer by layer. The update formula of the global feature encoder is:

[0018] ;

[0019] in, represents the updated global features, is the global feature before updating, is a nonlinear activation function, is the trainable weight matrix; is the bias term;

[0020] The local feature encoder uses a graph convolutional network (GCN) to mine the local structural relationship and feature interaction between a node and its neighbors. The update formula of the local feature encoder is as follows:

[0021] ;

[0022] in, represents the updated global features; Represents neighbor nodes u In the l Mixed features of layers; Represents the mixed features of the current node v itself in the lth layer, and Together they are used for feature aggregation to calculate the node v In the l +1 layer updated features ; Representation node v The set of neighbor nodes of and is the trainable parameter of the GCN layer, is a nonlinear activation function;

[0023] After completing multi-layer encoding, the final node hybrid feature representation is obtained.

[0024] As a preferred technical solution, in the step of independent adaptive aggregation:

[0025] The domain similarity matrix measures the data structure similarity between clients and is calculated based on the total variation distance of the node degree distribution;

[0026] The model similarity matrix measures the similarity of global feature encoder parameters and is calculated using cosine similarity;

[0027] The update direction similarity matrix measures the consistency of the gradient update direction of the local feature encoder and is calculated using cosine similarity.

[0028] As a preferred technical solution, in the step of independent adaptive aggregation:

[0029] The global feature relationship matrix is ​​used to aggregate the global feature encoder and is expressed as follows:

[0030] ;

[0031] in, G represents the global feature encoding matrix to be optimized, It means solving the optimal feature encoding matrix that minimizes the overall weighted mean square error. Controls the balance between domain similarity and model similarity. MSE(.,.) represents the mean square error loss function.

[0032] The local feature relationship matrix is ​​used to aggregate the local feature encoder and is expressed as follows:

[0033] ;

[0034] in, L represents the local coding matrix to be optimized, Indicates solving the optimal local feature encoding matrix so that it can be used in the target task T With reference structure C The comprehensive error between Controlling the trade-off between domain structure and gradient update direction.

[0035] As a preferred technical solution, the constrained quadratic programming problem is specifically:

[0036] Aggregate weight vector for each client W i , the optimization problem is expressed as follows:

[0037] ;

[0038] constraint:

[0039] ;

[0040] in, Indicates that among all feasible weight vectors W i In the objective function Get the optimal solution with the minimum value; is a customer i End aggregation weight vector W iThe transpose of ; Q represents the similarity matrix in the feature space, which measures the similarity relationship between each feature or node; b Represents the weight bias term vector, which imposes guided adjustments on the aggregation weights in combination with specific task requirements; Q and b Generated by combining similarity matrix and weight coefficient; i Indicates the client number currently being optimized; j Indicates that on the client i The referenced feature or neighbor index is used to assign feature contribution weights one by one.

[0041] As a preferred technical solution, the aggregation weight is dynamically updated based on similarity, specifically:

[0042] Each client i The personalized model is updated as follows:

[0043] ;

[0044] in, Indicates the client logo i The personalized model parameters of the final model after fusion and update; K Indicates the number of optional reference clients or reference features; L ik Represents the client learned by the local feature encoder i With reference client k The local aggregation weight reflects the local similarity relationship; G ik Represents the client learned by the global feature encoder i With reference client k The global aggregation weight reflects the global structural consistency; Indicates the reference client k The high similarity model parameters are only shared under the condition of high similarity of local features; Reference Client k Model parameters in the global feature space of ;

[0045] Local feature encoders are only shared between clients with high similarity to prevent heterogeneous feature contamination;

[0046] The global feature encoder optimizes aggregation based on domain and model similarity to improve synergy.

[0047] As a preferred technical solution, the optimization goals of client local training are as follows:

[0048] ;

[0049] in: Represents the client i Based on local dataset G i The task loss function of i i Represents the current local model parameters; Represents the personalized aggregation model parameters distributed by the server; l represents the regularization strength hyperparameter, which controls the balance between local adaptation and global coordination; represents the Frobenius norm, which is used to measure parameter differences.

[0050] As a preferred technical solution, the gradient descent method is used to update the local model parameters, and the regularization strength is dynamically adjusted based on the heterogeneity of client data. Specifically:

[0051] Based on the optimization objective, the gradient descent method is used to update the local model:

[0052] ;

[0053] in, Expressed as the gradient of the general task loss; Represents the proximal regularization guide term, which forces the model parameters to converge near the personalized aggregation results and avoid model divergence;

[0054] Dynamically adjust the regularization coefficient l , the adjustment strategy is as follows:

[0055] ;

[0056] in: l 0 is the basic regularization coefficient; d i Represents the client i Heterogeneity metrics.

[0057] In a second aspect, the present invention provides a federated graph learning system based on feature space construction and sharing, which is applied to the aforementioned federated graph learning method based on feature space construction and sharing, and includes a global feature space construction module, a client hybrid feature encoding module, an adaptive aggregation module, and a personalized model update module;

[0058] The global feature space construction module is used to generate a normalized random walk matrix based on the adjacency matrix and the degree matrix for the graph data of each client; extract multi-order random walk features to form a unified global feature representation, and eliminate the feature distribution differences between different clients through feature normalization to construct a shared global feature space;

[0059] The client hybrid feature encoding module is used to input the global feature and the original local feature into the global feature encoder and the local feature encoder respectively for each node, and generate a hybrid feature by feature splicing;

[0060] The adaptive aggregation module is used to calculate the domain similarity matrix, model similarity matrix and update direction similarity matrix between clients, construct a global feature relationship matrix based on the domain similarity matrix and the model similarity matrix, and construct a local feature relationship matrix based on the model similarity matrix and the update direction similarity matrix. The global feature relationship matrix and the local feature relationship matrix are independently aggregated, and the aggregation weight optimization is converted into a constrained quadratic programming problem. The aggregation weight is dynamically updated according to the similarity.

[0061] The personalized model update module is used to add a proximal regularization term to the loss function of local client training to constrain the degree of deviation between the local model parameters and the aggregated model parameters sent by the server. Based on the objective function containing the regularization term, the local model parameters are updated using the gradient descent method, and the regularization strength is dynamically adjusted based on the heterogeneity of client data to achieve a balance between collaboration and personalization.

[0062] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0063] By constructing a globally consistent feature space, designing differentiated adaptive aggregation strategies, and introducing a personalized regularization mechanism, the present invention optimally solves the technical problems faced by federated graph learning in a cross-domain Internet of Things environment, such as heterogeneous feature space, invalid knowledge integration, and insufficient personalized performance.

[0064] Furthermore, the global feature space construction method proposed in the present invention is based on the normalized random walk mechanism. Without accessing the original data, it realizes the alignment and sharing of features across client nodes, breaking the limitations of traditional reliance on manual alignment or feature mapping, and improving the compatibility and scalability of feature representation.

[0065] In addition, the present invention innovatively introduces an independent adaptive aggregation mechanism, establishing similarity metrics and optimization strategies for the global encoder and local encoder respectively, which significantly avoids the "harmful knowledge integration" phenomenon commonly seen in model aggregation and improves the robustness and generalization performance of the global model.

[0066] At the same time, the present invention combines personalized regularization terms with dynamic adjustment of client heterogeneity to ensure the personalized performance and rapid convergence ability of the model under complex data distribution, effectively alleviating the problem of "cooperation-individuality" conflict in federated learning.

[0067] In summary, the present invention not only improves the collaborative efficiency and personalized adaptability of the model under heterogeneous and non-IID data, significantly accelerates the model convergence speed, and reduces communication and computing overhead, but also has good module decoupling and system deployability. It is suitable for various privacy-sensitive IoT application scenarios such as medical, industrial, and financial, and has outstanding data privacy protection advantages and broad engineering application prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0069] Figure 1 This is a flowchart of a federated graph learning method based on feature space construction and sharing according to an embodiment of the present invention;

[0070] Figure 2 This is a block diagram of a federated graph learning system based on feature space construction and sharing in an embodiment of the present invention. DETAILED DESCRIPTION

[0071] In order to enable those skilled in the art to better understand the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0072] References to "embodiments" in this application mean that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute an independent or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described in this application may be combined with other embodiments.

[0073] In a cross-domain IoT environment, different clients (such as medical institutions, industrial enterprises, social platforms, etc.) have significant heterogeneity in their node feature spaces due to differences in data collection equipment, standards, and domains, which directly affects the model synergy in federated learning. To address this issue, this paper proposes a unified node feature initialization method to build a globally consistent feature space shared by all clients. Figure 1 , specifically including the following steps:

[0074] S1: Global feature space construction, including: generating a normalized random walk matrix based on the adjacency matrix and degree matrix; extracting multi-order random walk features to form a unified global node feature representation; ensuring consistent feature distribution across different clients through feature normalization operations, building a shared global feature space, and avoiding the impact of feature heterogeneity on model collaboration.

[0075] In a cross-domain Internet of Things (IoT) environment, different clients (such as medical institutions, industrial enterprises, and social platforms) experience significant heterogeneity in their node feature spaces due to differences in data collection equipment, standards, and domains. This directly impacts the effectiveness of model collaboration in federated learning. To address this issue, this paper proposes a unified node feature initialization method to construct a globally consistent feature space shared by all clients. Specifically, the method includes the following steps:

[0076] S101: Construction of adjacency matrix and degree matrix: For each client's graph data, a normalized random walk matrix is ​​generated based on the adjacency matrix and degree matrix;

[0077] For the graph data held by each client , first calculate the basic matrix of its graph structure:

[0078] Adjacency Matrix : Used to describe the connection relationship between nodes;

[0079] Degree Matrix : is a diagonal matrix representing the degree of each node.

[0080] The degree matrix is ​​expressed as follows:

[0081] ;

[0082] Among them, the degree matrix D i Middle v Rank v The value of the column, representing the node v The degree of the node is the number of edges it is connected to. This value is obtained by comparing the edges in the adjacency matrix with the nodes v The sum of all connected edges.

[0083] Step S101 can ensure that the topological structure of the graph is expressed in a standardized manner, facilitating the subsequent construction of unified global features.

[0084] S202: Random Walk Positional Encoding;

[0085] In order to eliminate the differences in the original feature spaces of each client, the present invention introduces random walk features as a unified node description method, as follows:

[0086] Compute the normalized random walk matrix R i : ;

[0087] in, R i ( v , u ) represents a slave node v Start a random walk to the node u probability.

[0088] Furthermore, in order to enhance the distinguishability of nodes in the graph structure, the diagonal elements of the random walk are selected to construct the global feature vector of the node:

[0089] ;

[0090] in: Indicates progress k The transition matrix after the random walk; S is a feature dimension hyperparameter that controls the expressiveness of global features.

[0091] The feature vector A unified generation rule is used on all clients to ensure that different clients can be mapped to a consistent global feature space even if their original feature spaces are different.

[0092] S203: Feature standardization and alignment;

[0093] In order to avoid the influence of different graph scales or node density differences on feature distribution, the generated global features are further Perform normalization:

[0094] ;

[0095] in, m and s are the mean and standard deviation of each dimension respectively. Global statistics or local statistics can be used to improve the consistency of feature distribution.

[0096] S2: Design a dual-path feature encoding architecture on the client side, specifically including: using a multi-layer perceptron (MLP) as a global feature encoder to extract globally consistent features; using a graph convolutional network (GCN) as a local feature encoder to retain local difference information; and fusing global and local features through a feature splicing mechanism to generate a hybrid feature representation that takes into account both collaborative optimization and local adaptability.

[0097] In the heterogeneous environment of federated graph learning, relying solely on local feature encoders can easily lead to models falling into local optima or experiencing feature bias, failing to fully utilize globally shared features. Relying solely on global features ignores the unique characteristics of each client, reducing the model's discriminative power and personalization. To address this, this paper proposes a Client-side Hybrid Feature Encoder (CHFE) that jointly encodes global and local features to achieve an effective fusion of cross-domain collaboration and local optimization.

[0098] Furthermore, step S2 specifically includes:

[0099] S201: Input feature preparation;

[0100] For the client i Each node v , the input features include two parts: global features: (Generated by the unified feature space construction method); local features: (Originally collected node attributes, such as device parameters, business indicators, etc.)

[0101] Initialize two types of feature embeddings for nodes:

[0102] ;

[0103] S202: Global Feature Encoder (GFE);

[0104] The global feature encoder adopts a multi-layer perceptron (MLP) structure to extract the deep semantic representation of global features layer by layer, avoiding the over-smoothing problem caused by excessive diffusion of graph convolution.

[0105] No. l The global feature update formula of the layer is:

[0106] ;

[0107] in, represents the updated global features, is the global feature before updating, is a nonlinear activation function, is the trainable weight matrix; is the bias term.

[0108] S203: Local Feature Encoder (LFE);

[0109] The local feature encoder uses a graph convolutional network (GCN) to mine the local structural relationship and feature interaction between a node and its neighbors, maintaining the discriminability of the node in the local domain.

[0110] No. l The local feature update formula of the layer is:

[0111] ;

[0112] in, represents the updated global features; Represents neighbor nodes u In the l Mixed features of layers; Represents the mixed features of the current node v itself in the lth layer, and Together they are used for feature aggregation to calculate the node v In the l +1 layer updated features ; Representation node v The set of neighbor nodes of and is the trainable parameter of the GCN layer, is a nonlinear activation function;

[0113] S204: Hybrid feature fusion mechanism;

[0114] In order to fully combine the advantages of global features and local features, this paper designs a fusion strategy based on feature concatenation:

[0115] ;

[0116] in: Represents vector concatenation operation; For the l Mixed feature representation of layers.

[0117] The fusion method in the present invention is simple and efficient, and can preserve global coordination and local personalized information while ensuring feature integrity.

[0118] S205: feature aggregation and output;

[0119] After completing multi-layer encoding, obtain the final node mixed representation , for graph-level tasks, pooling operations are used to generate graph-level features:

[0120] ;

[0121] Among them, the Readout operation can use average pooling, maximum pooling or self-attention pooling mechanism.

[0122] Finally, combined with task requirements (such as classification, prediction, etc.), it is input into the classifier or regressor for output.

[0123] S3: Construct a multi-dimensional similarity measurement system and design a differentiated aggregation mechanism, specifically including: calculating domain similarity, model parameter similarity, and gradient update similarity matrices; generating global feature relationship matrices and local feature relationship matrices respectively for independent aggregation; transforming aggregation weight optimization into a constrained quadratic programming problem to ensure stable and effective aggregation; dynamically adjusting aggregation weights based on similarity to avoid invalid knowledge integration and improve model generalization and personalization performance.

[0124] In traditional federated learning (FL) frameworks, model aggregation typically uses a unified averaging strategy (such as FedAvg), assuming that all clients have consistent data distribution, feature spaces, and model contributions. However, in real-world cross-domain IoT scenarios, clients have significantly heterogeneous feature spaces and data differences. Blind aggregation can lead to: 1) harmful knowledge integration; 2) degradation of client model performance; and 3) reduced generalization ability or even failure of the aggregated results. Therefore, this paper proposes an Independent Adaptive Aggregation (IAA) strategy, designing differentiated aggregation mechanisms for different encoders (global feature encoders and local feature encoders) to achieve precise, dynamic, and controllable model fusion.

[0125] Furthermore, step S3 is specifically as follows:

[0126] S301: Similarity measurement matrix construction;

[0127] (1) Domain Similarity Matrix T ;

[0128] The field similarity matrix T The data structure similarity between clients is measured based on the total variation distance (TVD) of the node degree distribution. The calculation formula is as follows:

[0129] ;

[0130] in, D i Represents the client i The node degree distribution of .

[0131] (2) Model similarity matrixS ;

[0132] Model similarity matrix S To measure the similarity of the global feature encoder parameters, the cosine similarity is calculated:

[0133] ;

[0134] in, For the client i The global feature encoder parameters of For the client j The global feature encoder parameters of .

[0135] (3) Update the direction similarity matrix C ;

[0136] Update the direction similarity matrix C Measure the consistency of the gradient update direction of the local feature encoder, also calculated using cosine similarity:

[0137] ;

[0138] Here, For the client i The local feature encoder gradient at the current round, For the client j Local feature encoder gradients at the current epoch.

[0139] S302: Modeling of weight optimization problem;

[0140] Aggregate the model weight matrix The solution is transformed into the following quadratic optimization problem:

[0141] ;

[0142] Constraints:

[0143] ;

[0144] in, S i Represents different similarity matrices, c i is the corresponding weight coefficient.

[0145] S303: Differentiated aggregation strategy design;

[0146] (1) Global feature relationship matrix G —For aggregating global feature encoders:

[0147] ;

[0148] in, G represents the global feature relationship matrix (or encoding matrix) to be optimized, Represents solving the optimal feature encoding matrix that minimizes the overall weighted mean square error (MSE), Controls the balance between domain similarity and model similarity. MSE(. , .) represents the mean squared error loss function.

[0149] (2) Local feature relationship matrix 𝐿 — used to aggregate local feature encoders:

[0150] ;

[0151] in, L represents the local feature relationship matrix (or local encoding matrix) to be optimized, Indicates solving the optimal local feature encoding matrix so that it can be used in the target task T With reference structure C The comprehensive error between Controlling the trade-off between domain structure and gradient update direction.

[0152] S304: Quadratic programming optimization solution;

[0153] For each row W i (i.e., the aggregate weight vector of each client), the optimization problem can be simplified to:

[0154] ;

[0155] constraint:

[0156] ;

[0157] in, Indicates that among all feasible weight vectors W i In the objective function Get the optimal solution with the minimum value; is a customer i End aggregation weight vector W i The transpose of ; Q represents the similarity matrix in the feature space, which measures the similarity relationship between each feature or node; b Represents the weight bias term vector, which imposes guided adjustments on the aggregation weights in combination with specific task requirements; Q and b Generated by combining similarity matrix and weight coefficient; i Indicates the client number currently being optimized; j Indicates that on the client iThe referenced feature or neighbor index is used to assign feature contribution weights one by one.

[0158] S305: Personalized aggregation update;

[0159] Finally, the personalized model of each client 𝑖 is updated as:

[0160] ;

[0161] in, Indicates the client logo i The personalized model parameters of the final model after fusion and update; K Indicates the number of optional reference clients or reference features (number of neighbors or total number of aggregated objects); L ik Represents the client learned by the local feature encoder i With reference client k The local aggregation weight reflects the local similarity relationship; G ik Represents the client learned by the global feature encoder i With reference client k The global aggregation weight reflects the global structural consistency Indicates the reference client k The high similarity model parameters are shared only under the condition of high similarity of local features (h stands for "highsimilarity"); Indicates the reference client k Model parameters in the global feature space of (e stands for “global encoding” or “entire space”);

[0162] Local feature encoder: shared only between clients with high similarity to prevent heterogeneous feature contamination;

[0163] Global feature encoder: optimizes aggregation based on domain and model similarity to improve synergy.

[0164] S4: Introducing personalized regularization terms during the client-side local training process, specifically including: adding proximal regularization terms to the local loss function to constrain model parameter offsets; dynamically adjusting the regularization strength based on client data heterogeneity to achieve a balance between synergy and personalization; and improving model stability, convergence speed, and personalized performance in non-IID environments.

[0165] In a federated learning (FL) environment, especially in highly heterogeneous IoT scenarios, different clients have significant differences in data distribution, feature space, and task requirements (Non-IID problem). Traditional global model synchronization strategies are prone to the following problems:

[0166] (1) Model Drift: Directly overwriting the local model with the server-side aggregated model will cause the model to deviate from the client's actual data distribution and reduce personalization performance.

[0167] (2) Conflict between generalization and personalization: Global collaborative optimization emphasizes universality, while each client is more concerned with the accuracy of its own task. It is difficult to balance the two.

[0168] Therefore, this paper introduces a personalized regularization mechanism (PPT). During local training, this regularization term guides the model to fully adapt to local data characteristics while maintaining global knowledge synergy, achieving a "synergy-personalization" balance. By adding a proximal regularization term to the local objective function, the deviation between local model parameters and aggregate model parameters is limited, preventing the model from overfitting to local data or being completely controlled by the global model.

[0169] S401: Local optimization objective function design;

[0170] For the client i , in each round of local training, the optimization objective is:

[0171] ;

[0172] in: Represents the client i Based on local dataset G i Task loss function (such as cross entropy loss); i i Represents the current local model parameters; Indicates the personalized aggregation model parameters distributed by the server (generated by IAA policy); l represents the regularization strength hyperparameter, which controls the balance between local adaptation and global coordination; represents the Frobenius norm, which is used to measure parameter differences.

[0173] S402: Gradient update rule;

[0174] Based on the above optimization objectives, the gradient descent method (or optimizer such as Adam) is used to update the local model:

[0175] ;

[0176] Among them, among them, Expressed as the gradient of the general task loss; It is expressed as a proximal regularization guide term, which forces the model parameters to converge near the personalized aggregation results and avoid model divergence.

[0177] S403: Dynamically adjust the regularization coefficient λ;

[0178] Taking into account the different degrees of data heterogeneity of different clients, the present invention supports dynamic adjustment of the regularization coefficient based on the following strategy: l , the adjustment strategy is as follows:

[0179] ;

[0180] in: l 0 is the basic regularization coefficient; d i Represents the client i Heterogeneity metrics (such as performance difference with the global model, data distribution offset, etc.).

[0181] S404: Optimize convergence assurance;

[0182] After introducing personalized regularization, the local optimization problem remains strongly convex (when l >0, which can ensure the convergence stability of each round of local training process and prevent overfitting or oscillation. Through theoretical derivation, the present invention can still achieve the following convergence constraints under non-convex conditions:

[0183] ;

[0184] Where N is the number of local training steps.

[0185] This paper eliminates feature heterogeneity in cross-domain IoT by constructing a globally consistent feature space based on random walk features, designs a dual-channel hybrid encoder (MLP+GCN) to jointly generate node representations that take into account both collaboration and personalization, and innovatively proposes an independent adaptive aggregation strategy - dynamically assigning weights based on domain, model and gradient similarity to achieve differentiated aggregation of global and local encoders to avoid harmful knowledge integration, while introducing a proximal regularization mechanism in local training to dynamically balance personalization and generalization requirements; this scheme significantly improves model robustness, communication efficiency and convergence speed in cross-domain scenarios, and simultaneously guarantees local task accuracy and global collaborative stability under highly Non-IID data, breaking through the heterogeneous collaboration bottleneck of traditional federated graph learning.

[0186] It should be noted that, for the sake of convenience, the aforementioned method embodiments are all expressed as a series of action combinations, but those skilled in the art should know that the present invention is not limited to the described order of actions, because according to the present invention, certain steps can be performed in other orders or simultaneously.

[0187] Based on the same concept as the federated graph learning method based on feature space construction and sharing in the above-mentioned embodiment, the present invention also provides a federated graph learning system based on feature space construction and sharing, which can be used to execute the above-mentioned federated graph learning method based on feature space construction and sharing. For ease of explanation, the structural diagram of the embodiment of the federated graph learning system based on feature space construction and sharing only shows the parts related to the embodiment of the present invention. Those skilled in the art will understand that the illustrated structure does not constitute a limitation of the device, and may include more or fewer components than shown in the diagram, or combine certain components, or arrange the components differently.

[0188] See also Figure 2 In another embodiment of the present application, a federated graph learning system 100 based on feature space construction and sharing is provided, which includes a global feature space construction module 101, a client hybrid feature encoding module 102, an adaptive aggregation module 103, and a personalized model updating module 104;

[0189] The global feature space construction module 101 is used to generate a normalized random walk matrix based on the adjacency matrix and the degree matrix for the graph data of each client; extract multi-order random walk features to form a unified global feature representation, and eliminate the feature distribution differences between different clients through feature normalization to construct a shared global feature space;

[0190] The client hybrid feature encoding module 102 is used to input the global feature and the original local feature into the global feature encoder and the local feature encoder respectively for each node, and generate a hybrid feature by feature splicing;

[0191] The adaptive aggregation module 103 is used to calculate the domain similarity matrix, model similarity matrix and update direction similarity matrix between clients, construct a global feature relationship matrix based on the domain similarity matrix and the model similarity matrix, and construct a local feature relationship matrix based on the model similarity matrix and the update direction similarity matrix. The global feature relationship matrix and the local feature relationship matrix are independently aggregated, and the aggregation weight optimization is converted into a constrained quadratic programming problem. The aggregation weight is dynamically updated according to the similarity.

[0192] The personalized model update module 104 is used to add a proximal regularization term to the loss function of local training on the client to constrain the degree of deviation between the local model parameters and the aggregated model parameters sent by the server. Based on the objective function containing the regularization term, the local model parameters are updated using the gradient descent method, and the regularization strength is dynamically adjusted based on the heterogeneity of the client data to achieve a balance between collaboration and personalization.

[0193] It should be noted that the federated graph learning system based on feature space construction and sharing of the present invention corresponds one-to-one to the federated graph learning method based on feature space construction and sharing of the present invention. The technical features and beneficial effects described in the above-mentioned embodiment of the federated graph learning method based on feature space construction and sharing are applicable to the embodiment of the federated graph learning based on feature space construction and sharing. For specific contents, please refer to the description in the embodiment of the method of the present invention. No further details will be given here. This is hereby declared.

[0194] In addition, in the implementation of the federated graph learning system based on feature space construction and sharing in the above-mentioned embodiment, the logical division of each program module is only an example. In actual applications, the above-mentioned functions can be assigned to different program modules as needed, for example, for the convenience of corresponding hardware configuration requirements or software implementation. That is, the internal structure of the federated graph learning system based on feature space construction and sharing is divided into different program modules to complete all or part of the functions described above.

[0195] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program. The program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0196] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0197] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be considered as equivalent replacement methods and are included in the scope of protection of the present invention.

Claims

1. A federated graph learning method based on feature space construction and sharing, characterized by: The steps include: For each client's graph data, a normalized random walk matrix is ​​generated based on the adjacency matrix and the degree matrix; multi-order random walk features are extracted to form a unified global feature representation, and feature normalization is used to eliminate feature distribution differences between different clients to construct a shared global feature space; the clients include medical institution clients, industrial enterprise clients, or social platform clients; For each node, the global feature and the original local feature are input into the global feature encoder and the local feature encoder respectively, and the hybrid feature is generated by feature splicing; Calculate the domain similarity matrix, model similarity matrix, and update direction similarity matrix between clients, construct a global feature relationship matrix based on the domain similarity matrix and the model similarity matrix, and construct a local feature relationship matrix based on the model similarity matrix and the update direction similarity matrix. Perform independent aggregation on the global feature relationship matrix and the local feature relationship matrix, transform the aggregation weight optimization into a constrained quadratic programming problem, and dynamically update the aggregation weight based on the similarity. In the independent polymerization step: The domain similarity matrix measures the data structure similarity between clients and is calculated based on the total variation distance of the node degree distribution; The model similarity matrix measures the similarity of global feature encoder parameters and is calculated using cosine similarity; The update direction similarity matrix measures the consistency of the gradient update direction of the local feature encoder and is calculated using cosine similarity; The constrained quadratic programming problem is specifically: Aggregate weight vector for each client W i , the optimization problem is expressed as follows: ； constraint: ； in, Indicates that among all feasible weight vectors W i In the objective function Get the optimal solution with the minimum value; is a customer i End aggregation weight vector W i The transpose of ; Q represents the similarity matrix in the feature space, which measures the similarity relationship between each feature or node; b Represents the weight bias term vector, which imposes guided adjustments on the aggregation weights in combination with specific task requirements; Q and b Generated by combining similarity matrix and weight coefficient; i Indicates the client number currently being optimized; j Indicates that on the client i The referenced feature or neighbor index is used to assign feature contribution weights one by one; A proximal regularization term is added to the loss function of local client training to constrain the degree of deviation between the local model parameters and the aggregated model parameters sent by the server. Based on the objective function containing the regularization term, the gradient descent method is used to update the local model parameters. The regularization strength is dynamically adjusted based on the heterogeneity of client data to achieve a balance between collaboration and personalization.

2. The federated graph learning method based on feature space construction and sharing according to claim 1 is characterized in that: The global feature space is constructed as follows: Based on the adjacency matrix A i Sum degree matrix D i Compute the normalized random walk matrix ; Select the random walk matrix k The global features of the nodes are constructed by random walk diagonal elements. The global features are generated using a unified rule on all clients to ensure that different clients can be mapped to a consistent global feature space even if their original feature spaces are different. The generated global feature vector is normalized to eliminate the feature distribution differences between different clients.

3. The federated graph learning method based on feature space construction and sharing according to claim 1 is characterized in that: In client-side hybrid feature encoding: The global feature encoder adopts a multi-layer perceptron (MLP) structure to extract the deep semantic representation of global features layer by layer. The update formula of the global feature encoder is: ； in, represents the updated global features, is the global feature before updating, is a nonlinear activation function, is the trainable weight matrix; is the bias term; The local feature encoder uses a graph convolutional network (GCN) to mine the local structural relationship and feature interaction between a node and its neighbors. The update formula of the local feature encoder is as follows: ； in, represents the updated global features; Represents neighbor nodes u In the l Mixed features of layers; Represents the mixed features of the current node v itself in the lth layer, and Together they are used for feature aggregation to calculate the node v In the l +1 layer updated features ; Representation node v The set of neighbor nodes of and is the trainable parameter of the GCN layer, is a nonlinear activation function; After completing multi-layer encoding, the final node hybrid feature representation is obtained.

4. The federated graph learning method based on feature space construction and sharing according to claim 1 is characterized in that: In the independent polymerization step: The global feature relationship matrix is ​​used to aggregate the global feature encoder and is expressed as follows: ； in, G represents the global feature encoding matrix to be optimized, It means solving the optimal feature encoding matrix that minimizes the overall weighted mean square error. Controls the balance between domain similarity and model similarity. MSE(. , .) represents the mean square error loss function. The local feature relationship matrix is ​​used to aggregate the local feature encoder and is expressed as follows: ； in, L represents the local coding matrix to be optimized, Indicates solving the optimal local feature encoding matrix so that it can achieve the best performance in the target task T With reference structure C The comprehensive error between Controlling the trade-off between domain structure and gradient update direction.

5. The federated graph learning method based on feature space construction and sharing according to claim 1 is characterized in that: The aggregation weight is dynamically updated based on similarity, specifically: The personalized model for each client 𝑖 is updated as: ； in, Indicates the client logo i The personalized model parameters of the final model after fusion and update; K Indicates the number of optional reference clients or reference features; L ik Represents the client learned by the local feature encoder i With reference client k The local aggregation weight reflects the local similarity relationship; G ik Represents the client learned by the global feature encoder i With reference client k The global aggregation weight reflects the global structural consistency; Indicates the reference client k The high similarity model parameters are only shared under the condition of high similarity of local features; Reference Client k Model parameters in the global feature space of ; Local feature encoders are only shared between clients with high similarity to prevent heterogeneous feature contamination; The global feature encoder optimizes aggregation based on domain and model similarity to improve synergy.

6. The federated graph learning method based on feature space construction and sharing according to claim 1 is characterized in that: The optimization goals of client local training are as follows: ； in: Represents the client i Based on local dataset G i The task loss function of θ i Represents the current local model parameters; Represents the personalized aggregation model parameters distributed by the server; λ represents the regularization strength hyperparameter, which controls the balance between local adaptation and global coordination; represents the Frobenius norm, which is used to measure parameter differences.

7. The federated graph learning method based on feature space construction and sharing according to claim 1 is characterized in that: The gradient descent method is used to update the local model parameters, and the regularization strength is dynamically adjusted based on the heterogeneity of client data. Specifically: Based on the optimization objective, the gradient descent method is used to update the local model: ； in, Expressed as the gradient of the general task loss; Represents the proximal regularization guide term, which forces the model parameters to converge near the personalized aggregation results and avoid model divergence; Dynamically adjust the regularization coefficient λ , the adjustment strategy is as follows: ； in: λ 0 is the basic regularization coefficient; δ i Represents the client i Heterogeneity metrics.

8. A federated graph learning system based on feature space construction and sharing, characterized by: A federated graph learning method based on feature space construction and sharing applied to any one of claims 1-7, comprising a global feature space construction module, a client hybrid feature encoding module, an adaptive aggregation module, and a personalized model update module; The global feature space construction module is used to generate a normalized random walk matrix based on the adjacency matrix and the degree matrix for the graph data of each client; extract multi-order random walk features to form a unified global feature representation, and eliminate the feature distribution differences between different clients through feature normalization to construct a shared global feature space; The client hybrid feature encoding module is used to input the global feature and the original local feature into the global feature encoder and the local feature encoder respectively for each node, and generate a hybrid feature by feature splicing; The adaptive aggregation module is used to calculate the domain similarity matrix, model similarity matrix and update direction similarity matrix between clients, construct a global feature relationship matrix based on the domain similarity matrix and the model similarity matrix, and construct a local feature relationship matrix based on the model similarity matrix and the update direction similarity matrix. The global feature relationship matrix and the local feature relationship matrix are independently aggregated, and the aggregation weight optimization is converted into a constrained quadratic programming problem. The aggregation weight is dynamically updated according to the similarity. The personalized model update module is used to add a proximal regularization term to the loss function of local client training to constrain the degree of deviation between the local model parameters and the aggregated model parameters sent by the server. Based on the objective function containing the regularization term, the local model parameters are updated using the gradient descent method, and the regularization strength is dynamically adjusted based on the heterogeneity of client data to achieve a balance between collaboration and personalization.

Images