A method and apparatus for federated heterogeneous graph reinforcement client selection
By constructing client state and action spaces, and quantifying data activity using node centrality, information entropy, and information density, combined with active learning and reinforcement learning, the challenge of client selection in federated heterogeneous graph learning is solved, improving model performance and privacy protection capabilities.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- BEIJING UNIV OF POSTS & TELECOMM
- Filing Date
- 2025-02-17
- Publication Date
- 2026-07-14
Smart Images
Figure CN120297434B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of federated learning technology, and in particular to a method and apparatus for enhancing the selection of clients in federated heterogeneous graphs. Background Technology
[0002] Federated Heterogeneous Graph Learning (FHGL), a combination of Federated Learning (FL) and Heterogeneous Graph Neural Networks (HGNN), aims to address the limitations of traditional HGNNs, which rely on massive amounts of data and are constrained by hardware resources and data privacy. Its core idea is to train a global model collaboratively through distributed clients while maintaining the privacy of local Heterogeneous Information Network (HIN) data. HINs consist of various node and edge types with complex topologies, requiring high-order feature extraction via HGNNs. FHGL allows each client to process this heterogeneous data locally and achieves global model aggregation through a federated mechanism. In this scenario, client selection becomes crucial: an efficient client selection mechanism must balance geographical distribution differences, HIN structural heterogeneity, and resource constraints to optimize global model performance, reduce communication overhead, and enhance privacy protection. However, existing technologies face significant challenges: First, traditional client selection algorithms are mostly designed for Euclidean data (such as images and text), making it difficult to adapt to the diversity of node / edge types and complex topological structures in HINs, leading to biases in the evaluation of client contributions. Second, the heterogeneity of HIN data (large differences in client distribution and complex topological combinations) exacerbates the contradiction between convergence and generalization in federated learning. Existing methods lack robustness in balancing these two aspects, and model performance may fluctuate due to dynamic combinations of node / edge types. Furthermore, in resource-constrained scenarios (such as edge devices), existing algorithms struggle to efficiently select clients that can provide rich topological information while reducing communication costs, further limiting practical deployment effectiveness. These problems stem from the mismatch between the graph structure characteristics of HGNs and the traditional federated optimization objectives. There is an urgent need to develop client selection strategies that take into account the dynamic characteristics of graph neural networks, accurate contribution evaluation metrics, and heterogeneity robustness to achieve a dual improvement in privacy protection and model performance. Summary of the Invention
[0003] In view of this, embodiments of the present invention provide an enhanced method and apparatus for selecting clients in federated heterogeneous graphs, in order to eliminate or improve one or more defects existing in the prior art, and to solve the problem of client and data selection in the context of federated learning of heterogeneous graph neural networks.
[0004] This invention provides an enhanced method for selecting clients in a federated heterogeneous graph. The method is executed collaboratively by multiple clients and a global server. Each client maintains its own independent heterogeneous graph data locally and performs federated learning in conjunction with the global server. The method includes the following steps:
[0005] The client loads an active learning agent to construct a client state space. Local active states are constructed using the node centrality, information entropy, and information density of each node in the local independent heterogeneous graph data. A client action space is constructed, and data samples are collected from the independent heterogeneous graph data using a fixed batch size. The activity scores of these data samples are calculated, and a first set number of data samples with the highest activity scores are selected for local training. Statistical features of the activity scores of the training data samples are calculated. The local training process uses the selected data samples to train the heterogeneous graph neural network in conjunction with a node classifier. The node classification results are combined with the labels in the data samples to establish a first loss. The local data prototype is calculated using the hidden layer representation of the local independent heterogeneous graph data using the heterogeneous graph neural network.
[0006] The global server is used to aggregate and update the global model by the parameters of the heterogeneous graph neural network and the node classifier trained by each client; obtain the local data prototype of the currently participating client to obtain a global prototype set, and redistribute the global prototype set to the currently participating client; at the same time, obtain the statistical features corresponding to the currently participating client and combine them as a global state to construct a global state space; and construct a global action space by selecting the client to participate in the next round of training as an action.
[0007] The client identifies the target client that is closest to its local data prototype, introduces a regularization term by minimizing the distance between the target client and the local data prototype, and adds the first loss function to update the parameters of the client's local training.
[0008] A first reward is calculated by combining the local training performance metrics of the client and the global model performance metrics of the global server; a second reward is calculated by combining the global model performance metrics and the target accuracy; based on reinforcement learning, the client and its local training data samples are selected for the next round of training with the optimization direction of maximizing the first reward and the second reward.
[0009] In some embodiments, the method further includes calculating the contribution of different types of edge pairs to the centrality based on degree centrality, quantifying the node centrality, and the calculation formula is:
[0010] C D (v)=∑ t∈T ω t ·deg t (v);
[0011] Among them, C D (v) represents the degree centrality of node v; T denotes the set of edge types; deg t(v) represents the degree of node v on edge type t; ω t Indicates the weight of the edge type.
[0012] In some embodiments, the method further includes:
[0013] The information entropy corresponding to the classification probability of a node is quantified using the node classifier.
[0014] The information density is quantified by the norm, sparsity index, or diversity index of the hidden layer representation after feature extraction by the heterogeneous graph neural network for each node; wherein, the sparsity index is the proportion of zero elements, and the diversity index is the Simpson diversity index;
[0015] Based on parameter n q The active score of the first Each percentile is used as the corresponding statistical feature.
[0016] In some embodiments, the first reward is calculated by combining the local training performance metrics of the client and the global model performance metrics of the global server, and the calculation formula is as follows:
[0017]
[0018] in, B represents the first reward. L Indicates the base of the first exponent. The global model performance metric represents the classification accuracy based on the global test dataset at time step t. The local training performance metric represents the classification accuracy based on the local test dataset at time step t.
[0019] In some embodiments, the second reward value is calculated by combining the global model performance index and the target accuracy, and the calculation formula is as follows:
[0020]
[0021] Where, r global B represents the second reward. G Indicates the base of the second exponent. The global model performance metric represents the classification accuracy (Acc) at time step t. T Indicates the target precision.
[0022] In some embodiments, the local data prototype is calculated using the hidden layer representation of the local independent heterogeneous graph data through the heterogeneous graph neural network, and the calculation formula is:
[0023]
[0024] Among them, C k f represents the local data prototype of the k-th client. k D represents the heterogeneous graph neural network local to the k-th client. k This represents the independent heterogeneous graph data located locally on the k-th client.
[0025] In some embodiments, the client identifies a target client that is closest to its local data prototype, and introduces a regularization term by minimizing the distance between the target client and its local data prototype. The regularization term is calculated as follows:
[0026]
[0027] in, This represents the local data prototype of the client that is closest to the local data prototype of the k-th client.
[0028] On the other hand, the present invention also provides an enhanced federal heterogeneous graph client selection device, comprising multiple clients and a global server, wherein the clients and the global server implement the steps of the above method when the computer program / instruction is executed.
[0029] On the other hand, the present invention also provides a computer-readable storage medium having a computer program / instructions stored thereon, which, when executed by a processor, implement the steps of the above-described method.
[0030] On the other hand, the present invention also provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the above-described method.
[0031] The beneficial effects of the present invention are at least as follows:
[0032] The enhanced federated heterogeneous graph client selection method and apparatus of this invention addresses the balance between data privacy and model performance in heterogeneous graphs through collaborative optimization between the client and the global server. On the client side, a local state space quantified based on node centrality, information entropy, and information density is first constructed. An active learning agent is used to select heterogeneous graph samples with high activity scores to train the local model, generating a local prototype representing the data distribution characteristics. Simultaneously, a cross-client prototype alignment mechanism is introduced, optimizing the training loss by minimizing the regularization term that minimizes the distance to the target client prototype, thus mitigating data heterogeneity. On the server side, client model parameters are aggregated to update the global model, and a global prototype library is constructed by integrating client prototypes. A reinforcement learning framework is designed based on the global state space combining client statistical features and the action space based on client selection strategies. An optimization mechanism is established by combining a first reward related to the client's local training accuracy and a second reward related to the global model's convergence progress. By maximizing the dual reward values, high-contribution clients and their data samples are dynamically selected for the next round, achieving multi-objective optimization of communication efficiency, topological information richness, and privacy protection. This method combines heterogeneous graph feature quantization, prototype alignment, and reinforcement decision-making to improve the robustness of federated learning in complex topological scenarios.
[0033] Additional advantages, objects, and features of the invention will be set forth in part in the description which follows, and will also become apparent in part to those skilled in the art upon studying the description, or may be learned by practice of the invention. The objects and other advantages of the invention can be realized and obtained by means of the structures specifically pointed out in the description and drawings.
[0034] Those skilled in the art will understand that the objectives and advantages achievable with the present invention are not limited to those specifically described above, and that the above and other objectives achievable with the present invention will become clearer from the following detailed description. Attached Figure Description
[0035] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, are not intended to limit the scope of the invention. In the drawings:
[0036] Figure 1 This is a logical structure diagram of the enhanced federated heterogeneous graph client selection method according to an embodiment of the present invention.
[0037] Figure 2 This is a structural diagram of the enhanced federated heterogeneous graph client selection device according to an embodiment of the present invention. Detailed Implementation
[0038] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the embodiments and accompanying drawings. Here, the illustrative embodiments and descriptions of this invention are used to explain the invention, but are not intended to limit the invention.
[0039] It should also be noted that, in order to avoid obscuring the invention with unnecessary details, only the structures and / or processing steps closely related to the solution according to the invention are shown in the accompanying drawings, while other details that are not closely related to the invention are omitted.
[0040] It should be emphasized that the term "including / comprises" as used herein refers to the presence of a feature, element, step, or component, but does not exclude the presence or addition of one or more other features, elements, steps, or components.
[0041] First, we will explain several concepts involved in this invention.
[0042] Federated learning is a distributed machine learning framework that allows multiple devices (such as smartphones and edge devices) to collaboratively train a model while maintaining local data privacy. The model is trained locally on each client, and after local updates, the model parameters are transmitted to a central server for aggregation in an encrypted manner, avoiding direct data transmission.
[0043] Federated graphs are a graph data structure extended within the federated learning framework, designed to handle training tasks with distributed graph data. In this scenario, graph data is stored across multiple different clients, each possessing its own local graph data. Through federated learning, clients can collaboratively train graph models, leveraging graph structure information for effective knowledge sharing and model improvement, without needing to share the original graph data.
[0044] Federated heterogeneous graphs refer to graph data used in federated learning where the nodes, edges, or attributes are of different types (i.e., "heterogeneous"). In this scenario, each client's data may be a different type of graph with different nodes, edges, and attributes. Federated heterogeneous graph learning faces challenges due to data heterogeneity and inconsistent node and edge types, requiring efficient training and aggregation while ensuring privacy.
[0045] Reinforcement learning is a machine learning method that learns optimal decision-making strategies through interaction with the environment. An agent observes the state of the environment, selects actions, and receives rewards or penalties based on those actions, thereby adjusting its strategy to maximize long-term rewards.
[0046] Active learning is a machine learning method in which the learning system improves learning efficiency by selecting the most valuable training samples, rather than relying entirely on randomly selected samples. In active learning, the model proactively selects samples that it is uncertain about or that contain a lot of information for labeling, thereby reducing the number of samples that need to be labeled and improving model performance.
[0047] Similar to other neural networks, massive amounts of data can significantly improve the performance of Heterogeneous Graph Neural Networks (HGNNs). However, in practical applications, collecting large amounts of data for training is often a challenging task due to hardware limitations or privacy concerns from multiple data sources. Federated learning (FL), as an effective method for collaboratively training models, does not require exchanging raw data and is particularly suitable for training machine learning models on distributed devices. Therefore, Federated Graph Learning (FGL) has gradually become a research hotspot, and Federated Heterogeneous Graph Learning (FHGL) allows local clients to maintain Heterogeneous Information Network (HIN) data and can handle various node and edge types.
[0048] In federated learning, client data is typically distributed across multiple geographical locations and exhibits heterogeneity. Client selection mechanisms can significantly impact the training efficiency and performance of the global model. This is particularly true in federated heterogeneous graph learning scenarios, where client data not only suffers from geographical heterogeneity but also involves the complex structure of heterogeneous information networks (HINs)—the combination of different node and edge types exacerbates this heterogeneity. Effective client selection prioritizes clients that contribute significantly to the global model, thereby improving training efficiency and reducing communication overhead and computational resource waste. The complex and diverse topologies and node / edge types in heterogeneous information networks make traditional client selection methods ill-suited to handle this complex information. In federated heterogeneous graph learning, fully considering the heterogeneity of nodes and edges during client selection, ensuring that selected clients effectively provide rich topological information to the global model, is crucial for improving model performance. Therefore, researching a client selection method that adapts to the characteristics of heterogeneous graphs can help select clients containing more useful information, thus improving the model's generalization ability and accuracy. A core advantage of federated learning is data privacy protection; client-local data is not directly transmitted to the server. Effective client selection can reduce unnecessary communication overhead by selecting only clients that can contribute significantly to the model for global aggregation. This not only improves communication efficiency but also avoids redundant data transmission, further enhancing the system's privacy protection capabilities. In many practical applications, federated learning faces challenges such as limited hardware resources and network bandwidth constraints. For example, in resource-constrained environments like smartphones and edge computing devices, training a global model requires an efficient client selection mechanism to ensure efficient model training within limited resources. Furthermore, with the increasing severity of data privacy and security issues, how to ensure effective data privacy protection through reasonable client selection while simultaneously optimizing model performance has become an important research need in federated heterogeneous graph learning.
[0049] While existing client selection (CS) algorithms have been extensively studied in Euclidean data scenarios, their application to federated heterogeneous graph learning (FHGL) has not received sufficient attention. Furthermore, the unique characteristics of heterogeneous information networks and the complexity of graph neural networks make accurately evaluating client contributions and selecting appropriate clients for aggregation exceptionally difficult in FHGL scenarios.
[0050] In the FHGL scenario, client selection faces two major challenges. First, the diverse node and edge types in heterogeneous networks introduce complex topological information, necessitating more advanced graph neural network architectures to handle more complex tasks. Existing client selection algorithms rely on model information and performance metrics, but this evaluation method is prone to bias, making it difficult to accurately select the client that contributes the most to global model aggregation. Second, due to data heterogeneity, existing client selection algorithms often face a trade-off between convergence and generalization. The complexity and diversity of heterogeneous networks further exacerbate data heterogeneity; the data distribution among clients may vary significantly, and complex node and edge combinations create difficult-to-handle topological information, challenging the robustness of federated learning in managing data heterogeneity. Traditional client selection algorithms may exacerbate the conflict between convergence and accuracy in this scenario.
[0051] Existing dynamic client selection methods in federated learning primarily determine which clients participate in the training of the global model by evaluating factors such as client performance, data distribution, computational power, communication costs, and contributions. Common strategies include selection based on client local training progress or data representativeness, dynamically adjusting client selection through reinforcement learning to adapt to different training needs, optimizing client selection based on communication overhead to reduce bandwidth consumption, and selecting representative groups after grouping clients using clustering algorithms. Furthermore, game theory has been incorporated into client selection strategies to optimize global training performance by promoting cooperation or competition among clients. However, the diverse node and edge types in heterogeneous information networks introduce complex topological information, requiring more advanced graph neural network architectures to handle more complex tasks. Existing client selection algorithms rely on model information and performance metrics, but this evaluation method is prone to bias and struggles to accurately select clients that contribute the most to the global model aggregation.
[0052] In view of this, the present invention provides an enhanced federated heterogeneous graph client selection method, wherein the method is executed collaboratively by multiple clients and a global server, each client maintaining local independent heterogeneous graph data and jointly performing federated learning with the global server, the method comprising the following steps S101 to S104:
[0053] Step S101: The client loads the active learning agent, constructs the client state space, and constructs the local active state based on the node centrality, information entropy, and information density of each node in the local independent heterogeneous graph data; constructs the client action space, collects data samples in the independent heterogeneous graph data using a fixed batch size and calculates the activity score of the data samples, selects the first set number of data samples with the highest activity scores to participate in local training, and calculates the statistical features of the activity scores of the data samples participating in the training; during the local training process, the selected data samples are used to train the heterogeneous graph neural network in combination with the node classifier, and the node classification results are combined with the labels in the data samples to establish the first loss; the local data prototype is calculated using the hidden layer representation of the local independent heterogeneous graph data using the heterogeneous graph neural network.
[0054] Step S102: The global server aggregates and updates the global model by collecting the parameters of the heterogeneous graph neural network and node classifier trained by each client; obtains the local data prototypes of the currently participating clients to obtain the global prototype set, and redistributes the global prototype set to the currently participating clients; at the same time, it obtains the statistical features corresponding to the currently participating clients and combines them as the global state to construct the global state space; and constructs the global action space by using the selection of clients to participate in the next round of training as actions.
[0055] Step S103: The client identifies the target client that is closest to its local data prototype, introduces a regularization term by minimizing the distance between the target client and its local data prototype, and adds the first loss function to update the parameters of the client's local training.
[0056] Step S104: Calculate the first reward by combining the local training performance metrics of the client and the global model performance metrics of the global server; calculate the second reward value by combining the global model performance metrics and the target accuracy; based on the form of reinforcement learning, with the optimization direction of maximizing the first and second reward values, select the client and its local training data samples to participate in the next round of training.
[0057] This invention addresses the challenges of client selection and data heterogeneity in Federated Heterogeneous Graph Learning (FHGL) through three core logics: client-side local optimization, global collaborative aggregation, and reinforcement learning dynamic decision-making.
[0058] Based on the federated learning structure, both the client and global server utilize independent heterogeneous graph data distributed across various clients for local learning, and aggregate parameters through the global server. On the client side, an active learning agent constructs local activity states based on node centrality, information entropy, and information density, quantifying the importance and information density of nodes in the heterogeneous graph data. High-activity score samples are selected for training, prioritizing the use of information-rich local topological data. Hidden layer representations are extracted using a heterogeneous graph neural network (HGNN) to generate local data prototypes characterizing the local data distribution, providing a foundation for subsequent cross-client alignment. By identifying the target client closest to the local prototype, a cross-client prototype distance minimization constraint (regularization term) is introduced into the loss function to reduce feature bias caused by data heterogeneity and improve model generalization.
[0059] During the global collaborative aggregation process, the global server aggregates the HGNN and node classifier parameters from each client, updates the global model, and achieves the basic collaboration of federated learning. It integrates the local prototypes of all clients to form a global prototype set, redistributes it to the clients, provides a global data distribution view, and assists in local training alignment. Finally, it combines the statistical features of each client (such as the distribution of activity scores) into a global state, dynamically characterizing the overall training state of the federated system and providing a basis for reinforcement learning decisions.
[0060] Dynamic reinforcement learning decision-making is introduced simultaneously on both the client and global server sides, selecting participating clients and data nodes during each training cycle. On the client side, high-value samples are dynamically selected based on activity scores to reduce redundant computation. A first reward is introduced, based on the client's local training performance (e.g., classification accuracy) and global model performance (e.g., aggregated accuracy), incentivizing the client to improve its local training quality. On the global server side, based on the global state, clients participating in the next round of training are selected through reinforcement learning strategies (e.g., Q-learning or policy gradient). A second reward is introduced, dynamically adjusted based on the gap between the global model performance and the target accuracy, guiding the system towards efficient convergence.
[0061] This scheme deeply integrates the characteristics of heterogeneous graph data, federated collaboration mechanisms, and reinforcement learning strategies through a closed-loop logic of local feature quantization, global prototype sharing, and enhanced dynamic decision-making. Under the premise of protecting privacy, it solves the core challenges of client selection and heterogeneity management in FHGL, and ultimately achieves efficient and robust global model training.
[0062] In step S101, in order to introduce reinforcement learning and active learning to select high-quality samples, it is first necessary to construct a state space and an action space, and select data samples with better performance. In this invention, the local active state is constructed using the node centrality, information entropy, and information density of each node in the local independent heterogeneous graph data.
[0063] The method calculates the contribution of different types of edges to the centrality based on degree centrality, quantifies node centrality, and the calculation formula is as follows:
[0064] C D (v)=∑ t∈T ω t ·deg t (v);
[0065] Among them, C D (v) represents the degree centrality of node v; T denotes the set of edge types; deg t (v) represents the degree of node v on edge type t; ω t Indicates the weight of the edge type.
[0066] In some embodiments, the method further includes: quantifying the information entropy corresponding to the classification probability of a node using a node classifier; quantifying the information density using the norm, sparsity index, or diversity index of the hidden layer representation of each node after feature extraction through a heterogeneous graph neural network; wherein the sparsity index is the proportion of zero elements, and the diversity index is the Simpson diversity index. Based on parameter n q The first active score Each percentile is used as the corresponding statistical characteristic.
[0067] Furthermore, the activity score of the data sample is obtained by weighting and summing the node centrality, information entropy, and information density of each node in the data sample.
[0068] In step S102, based on the aggregation of federated learning parameters, a global state space and a global action space are constructed to perform client selection. The global server aggregates the heterogeneous graph neural network (HGNN) and node classifier parameters trained by each client (e.g., using the FedAvg algorithm) and updates the global model. By fusing the local knowledge of multiple clients, the generalization ability of the global model is improved, avoiding model bias caused by data limitations of a single client. The global model is updated after each federated round to ensure that all clients perform the next round of local training based on a unified knowledge benchmark.
[0069] Collect local data prototypes (i.e., the center points of hidden layer feature representations) from each client to form a global prototype set. The global prototype library characterizes the commonalities in data distribution among all clients in the federated system, providing a reference for subsequent feature alignment between clients. The global prototype library is distributed to the clients participating in training, guiding them to align with the global distribution during local training through regularization terms (step S103), reducing feature shifts caused by data heterogeneity.
[0070] The global server aggregates the activity score statistics (such as the 25th, 50th, and 75th percentiles) reported by each client to construct a global state space. This quantifies the overall data distribution characteristics of the federated system; for example, the concentration of high percentiles (such as P75) reflects that most clients possess high-value samples, while the dispersion of low percentiles exposes insufficient data quality from some clients.
[0071] In this process, only model parameters and prototypes (rather than raw data) are transmitted, reducing communication overhead while protecting privacy. Through global state analysis, subsequent reinforcement learning can prioritize clients with high data quality (such as P90 activity scores) to avoid inefficient participation.
[0072] In some embodiments, the local data prototype is calculated using the hidden layer representation of the local independent heterogeneous graph data using a heterogeneous graph neural network, and the calculation formula is as follows:
[0073]
[0074] Among them, C k f represents the local data prototype of the k-th client. k Let D represent the heterogeneous graph neural network local to the k-th client. k This represents the independent heterogeneous graph data located locally on the k-th client.
[0075] In step S103, the client identifies the target client that is closest to its local data prototype. A regularization term is introduced by minimizing the distance between the target client and its local data prototype. The regularization term is calculated as follows:
[0076]
[0077] in, This represents the local data prototype of the client that is closest to the local data prototype of the k-th client.
[0078] By minimizing the distance between the local data prototype and the target client data prototype, the regularization term constrains the training direction of the client, preventing it from becoming overly reliant on its local data and getting trapped in local optima. This constraint mechanism helps enhance collaboration among clients, making the training results of each client closer to the expectations of the global model, thereby improving the overall performance and generalization ability of the entire federated learning system. Simultaneously, the regularization term can also reduce overfitting during client training to some extent, ensuring the adaptability of the local model to the global data distribution.
[0079] In step S104, the first reward is calculated by combining the client's local training performance metrics and the global model performance metrics on the global server side. The calculation formula is as follows:
[0080]
[0081] in, Indicates the first reward, B L Indicates the base of the first exponent. This represents the global model performance metric at time step t, which is the classification accuracy based on the global test dataset. This represents the local training performance metric at time step t, which is the classification accuracy based on the local test dataset.
[0082] In some embodiments, the second reward value is calculated by combining the global model performance index and the target accuracy, and the calculation formula is as follows:
[0083]
[0084] Where, r global Indicates the second reward, B G Indicates the base of the second exponent. This represents the global model performance metric at time step t, where Acc is the classification accuracy based on the global test dataset. T Indicates the target precision.
[0085] On the other hand, the present invention also provides an enhanced federal heterogeneous graph client selection device, comprising multiple clients and a global server, wherein the clients and the global server implement the steps of the above method when the computer program / instruction is executed.
[0086] On the other hand, the present invention also provides a computer-readable storage medium having a computer program / instructions stored thereon, which, when executed by a processor, implement the steps of the above-described method.
[0087] On the other hand, the present invention also provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the above-described method.
[0088] The present invention will now be described with reference to a specific embodiment:
[0089] This embodiment provides a Federated Heterogeneous Graph Client Selection (RAFHGL) method based on reinforcement learning and active learning. This method combines the advantages of reinforcement learning (RL) and active learning (AL) to improve client selection performance in federated heterogeneous graph learning. The RAFHGL algorithm can dynamically adapt to changes in the client scenario, providing an effective client selection method that ensures convergence in the short term while contributing to the optimization of the global model in the long term. By leveraging the ability of reinforcement learning to handle delayed rewards, RAFHGL enables clients to make intelligent decisions, thereby optimizing not only the local model but also improving the performance of the entire federated learning framework.
[0090] The structure of this embodiment is as follows: Figure 1 As shown, the quality of client data is evaluated through active learning, selecting samples that are more informative for the model. Using the distribution of effective samples as an indicator of client quality ensures data privacy while reducing interference from other factors. The proposed architecture relies primarily on two different agents: a local active learning agent (ALA) and a global client selection agent (CSA), which help the server accurately evaluate client information and select the most informative clients to promote the convergence of the global model. However, the active selection by these two agents may introduce bias into the training data; therefore, a data prototype-based correction method is proposed to mitigate the over-personalization of local models.
[0091] In a federated environment with K clients and one server, the k-th client maintains its own independent heterogeneous graph data D. k =(G k ,X k ,Y k ), where X k and Y k These are the feature matrix and the label matrix, respectively, while G k =(V k E k A k ,R k ) represents a heterogeneous graph data.
[0092] Client-side Local Active Learning Agent (ALA): On the local client, the ALA calculates the activity score of samples based on the local state. It prioritizes nodes with higher scores for training the local heterogeneous graph neural network. Simultaneously, the statistical features of the activity scores are transmitted to the server. Since the results of local active learning are a crucial input to the global reinforcement learning state, generating stable and accurate active learning results is essential. The reinforcement algorithm can adaptively adjust the sample selection strategy to cope with different use cases and the evolving global model. To achieve this, the active learning process on the client is instantiated as a Markov decision process. The ALA on the k-th client can be represented as a tuple M. k =(S k U k ,r k ), where S k U represents the local state set. k This represents the action set, specifically the probability distribution of output choices, while r... k This indicates the reward associated with the selected action.
[0093] Status. Local active status S k It encompasses three key aspects: node centrality, reflecting the importance of nodes in the data network, which is efficiently calculated using the PageRank algorithm; information entropy, measuring the ease with which a classifier accurately classifies node embeddings; high entropy aids in learning ambiguous information, while low entropy reinforces existing knowledge; this invention uses the classification probability of the local classifier as information entropy; and information density involves active learning on the client side, selecting samples with the highest training priority instead of creating a core set for labeling. Because traditional methods relying on core set distance calculations are difficult to apply, this invention utilizes the hidden layer representation of nodes to evaluate the information density of sample embeddings.
[0094] Action. Based on the status of the local nodes, ALA calculates the activity score for all local samples using a fixed batch size. The top B samples with the highest activity scores are selected. s Each sample participates in local training. Simultaneously, statistical characteristics of the activity score are calculated.
[0095] Rewards. The goal of local AI can be summarized as improving model performance, specifically reflected in local training performance (Acc). L And global model performance Acc G Two aspects. It's important to note that improving global performance is the primary task; therefore, changes in global performance will be calculated and factored in by an exponential base B. L Exponential scaling is applied. To ensure that samples with low or even negative local performance changes, but which may contribute to improving global performance, are not ignored, local performance is given linear weights.
[0096]
[0097] in, Indicates the first reward, B L Indicates the base of the first exponent. This represents the global model performance metric at time step t, which is the classification accuracy based on the global test dataset. This represents the local training performance metric at time step t, which is the classification accuracy based on the local test dataset.
[0098] Server-Client Agent Selection (CSA): On the global server side, CSA constructs a global state derived from aggregated statistics of all client activity scores. The server then identifies the clients most likely to make significant contributions to the global model for model aggregation. Therefore, in each round of federated communication, K is selected using CSA. s The equipment involved in the training is as follows. The specific design of the CSA agent is as follows:
[0099] State. The global client's selection state can be represented as a vector s. G ={u1,u2,...,u Ks}, where u k This represents the statistical characteristics of the activity score result for the k-th client. For a given parameter n... q The statistical characteristics correspond to the evaluation results of the first... Percentiles.
[0100] Actions. The action space is defined as {1,2,...,K}, where a = k represents selecting the k-th device to participate in federated training.
[0101] Rewards. The reward system is designed to incentivize the agent to approach the target accuracy (Acc). T This achieves higher accuracy, and the specific formula is as follows:
[0102]
[0103] Where, r global Indicates the second reward, B G Indicates the base of the second exponent. This represents the global model performance metric at time step t, where Acc is the classification accuracy based on the global test dataset. T Indicates the target precision.
[0104] Data Drift Correction Based on Data Prototypes: Due to the data selection process on both the client and server sides, the global model in each aggregation round is only influenced by a subset of characteristic data. This can lead to the global model converging to a local optimum and exacerbate client drift. To address this issue, this invention proposes a mitigation strategy based on data prototypes. This strategy aims to limit local training, thereby preventing outliers from having an excessive impact on the model and ensuring training stability. Specifically, each client initially performs data selection based on its local dataset D. k The hidden layer represents the computation of the local data prototype C. k .
[0105]
[0106] Among them, C k f represents the local data prototype of the k-th client. k Let D represent the heterogeneous graph neural network local to the k-th client. k This represents the independent heterogeneous graph data local to the k-th client. Where f k It is a local heterogeneous graph neural network model.
[0107] Subsequently, the central server collects all client prototypes to obtain a global prototype set P = {C1, C2, ..., C}. K The server then distributes these prototypes to the client. During this process, instead of aggregating these prototypes, the server introduces a regularization term. This is to punish nodes whose hidden layer features deviate from the overall federated system.
[0108]
[0109] in, This represents the local data prototype of the client that is closest to the local data prototype of the k-th client.
[0110] The purpose of adding regularization is to mitigate client drift caused by repeated active choices between the client and server. However, excessive penalty can limit the client's exploration speed in the solution space, leading to slower convergence. Therefore, we intuitively choose the local prototype with the smallest deviation from the respective local client for correction, rather than calculating the global prototype.
[0111] Corresponding to the above method, the present invention also provides an apparatus / system including a computer device, the computer device including a processor and a memory, the memory storing computer instructions, the processor executing the computer instructions stored in the memory, and when the computer instructions are executed by the processor, the apparatus / system performs the steps of the method as described above.
[0112] This invention also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the aforementioned edge computing server deployment method. The computer-readable storage medium can be a tangible storage medium, such as random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, floppy disks, hard disks, removable storage disks, CD-ROMs, or any other form of storage medium known in the art.
[0113] In summary, the enhanced federated heterogeneous graph client selection method and apparatus of this invention addresses the balance between data privacy and model performance in heterogeneous graphs through collaborative optimization between the client and the global server. On the client side, a local state space quantifying data activity based on node centrality, information entropy, and information density is first constructed. An active learning agent is used to select heterogeneous graph samples with high activity scores to train the local model, generating a local prototype representing the data distribution characteristics. Simultaneously, a cross-client prototype alignment mechanism is introduced, optimizing the training loss by minimizing the regularization term relative to the target client prototype to alleviate data heterogeneity. On the server side, client model parameters are aggregated to update the global model, and a global prototype library is constructed by integrating client prototypes. A reinforcement learning framework is designed based on the global state space combining client statistical features and the action space based on client selection strategies. An optimization mechanism is established by combining a first reward related to the client's local training accuracy and a second reward related to the global model's convergence progress. By maximizing the dual reward values, high-contribution clients and their data samples are dynamically selected for the next round, achieving multi-objective optimization of communication efficiency, topological information richness, and privacy protection. This method combines heterogeneous graph feature quantization, prototype alignment, and reinforcement decision-making to improve the robustness of federated learning in complex topological scenarios.
[0114] Those skilled in the art will understand that the exemplary components, systems, and methods described in conjunction with the embodiments disclosed herein can be implemented in hardware, software, or a combination of both. Whether implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this invention. When implemented in hardware, it can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this invention are programs or code segments used to perform the desired tasks. The programs or code segments can be stored in a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried in a carrier wave.
[0115] It should be clarified that the present invention is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present invention is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of the present invention.
[0116] In this invention, features described and / or illustrated for one embodiment may be used in the same or similar manner in one or more other embodiments, and / or combined with or in place of features of other embodiments.
[0117] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, various modifications and variations of the embodiments of the present invention are possible. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.
Claims
1. A method for enhancing client selection in a federated heterogeneous graph, characterized in that, The method is executed collaboratively by multiple clients and a global server. Each client maintains its own independent heterogeneous graph data locally and performs federated learning in conjunction with the global server. The method includes the following steps: The client loads an active learning agent to construct a client state space. Local active states are constructed using the node centrality, information entropy, and information density of each node in the local independent heterogeneous graph data. A client action space is constructed, and data samples are collected from the independent heterogeneous graph data using a fixed batch size. The activity scores of these data samples are calculated, and a first set number of data samples with the highest activity scores are selected for local training. Statistical features of the activity scores of the training data samples are calculated. The local training process uses the selected data samples to train the heterogeneous graph neural network in conjunction with a node classifier. The node classification results are combined with the labels in the data samples to establish a first loss. The local data prototype is calculated using the hidden layer representation of the local independent heterogeneous graph data using the heterogeneous graph neural network. The global server is used to aggregate and update the global model by the parameters of the heterogeneous graph neural network and the node classifier trained by each client; obtain the local data prototype of the currently participating client to obtain a global prototype set, and redistribute the global prototype set to the currently participating client; at the same time, obtain the statistical features corresponding to the currently participating client and combine them as a global state to construct a global state space; and construct a global action space by selecting the client to participate in the next round of training as an action. The client identifies the target client that is closest to its local data prototype, introduces a regularization term by minimizing the distance between the target client and the local data prototype, and adds the first loss function to update the parameters of the client's local training. A first reward is calculated by combining the local training performance metrics of the client and the global model performance metrics of the global server; a second reward is calculated by combining the global model performance metrics and the target accuracy; based on reinforcement learning, the client and its local training data samples are selected for the next round of training with the optimization direction of maximizing the first reward and the second reward.
2. The enhanced federated heterogeneous graph client selection method according to claim 1, characterized in that, The method further includes calculating the contribution of different types of edges to the centrality based on degree-based centrality, quantifying the node centrality, and the calculation formula is: C D (v)=∑ t∈T ω t ·deg t (v); Among them, C D (v) represents the degree centrality of node v; T denotes the set of edge types; deg t (v) represents the degree of node v on edge type t; ω t Indicates the weight of the edge type.
3. The enhanced federated heterogeneous graph client selection method according to claim 1, characterized in that, The method further includes: The information entropy corresponding to the classification probability of a node is quantified using the node classifier. The information density is quantified by the norm, sparsity index, or diversity index of the hidden layer representation after feature extraction by the heterogeneous graph neural network for each node; wherein, the sparsity index is the proportion of zero elements, and the diversity index is the Simpson diversity index; Based on parameter n q The active score of the first Each percentile is used as the corresponding statistical feature.
4. The enhanced federated heterogeneous graph client selection method according to claim 1, characterized in that, The first reward is calculated by combining the local training performance metrics of the client and the global model performance metrics of the global server. The calculation formula is as follows: in, B represents the first reward. L Indicates the base of the first exponent. The global model performance metric represents the classification accuracy based on the global test dataset at time step t. The local training performance metric represents the classification accuracy based on the local test dataset at time step t.
5. The enhanced federated heterogeneous graph client selection method according to claim 1, characterized in that, The second reward value is calculated by combining the global model performance metrics and the target accuracy, using the following formula: Where, r global B represents the second reward. G Indicates the base of the second exponent. The global model performance metric represents the classification accuracy (Acc) at time step t. T Indicates the target precision.
6. The enhanced federated heterogeneous graph client selection method according to claim 1, characterized in that, The local data prototype is calculated using the hidden layer representation of the local independent heterogeneous graph data through the heterogeneous graph neural network, and the calculation formula is as follows: Among them, C k f represents the local data prototype of the k-th client. k D represents the heterogeneous graph neural network local to the k-th client. k This represents the independent heterogeneous graph data located locally on the k-th client.
7. The enhanced federated heterogeneous graph client selection method according to claim 6, characterized in that, The client identifies the target client that is closest to its local data prototype, and introduces a regularization term by minimizing the distance between the target client and its local data prototype. The regularization term is calculated as follows: in, This represents the local data prototype of the client that is closest to the local data prototype of the k-th client.
8. A device for enhancing the selection of clients in a federated heterogeneous graph, characterized in that, It includes multiple clients and a global server, the clients and the global server being used to perform the steps of the method according to any one of claims 1 to 7.
9. A computer-readable storage medium having a computer program / instructions stored thereon, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method as described in any one of claims 1 to 7.
10. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method according to any one of claims 1 to 7.