Dual-channel federal map learning method, device and system with privacy protection

Abstract

The invention provides a dual-channel federal map learning method, device and system with privacy protection, and belongs to the technical field of federal learning. The method comprises the following steps: a client performs feature aggregation on a locally held sub-graph, acquires aggregated node feature information, and sends the aggregated node feature information to a server; the server randomly performs cross-client feature aggregation between the two client sub-graphs by using the aggregated node feature information; the client interacts with the server according to the cross-client sub-graph information, and global meta-model parameters are obtained in a meta-learning mode; the client updates local model parameters according to the parameters of the global meta-model; and the client updates local model parameters according to the support set of the local sub-graph nodes and uploads the local model parameters to the server, and the server performs fine adjustment on global meta-model parameters by adopting a federated average algorithm to obtain a global model. According to the invention, while the learning effect is ensured, the privacy protection capability of the data is greatly enhanced, and the overall efficiency and effect of federal learning are improved.

Description

Technical Field

[0001] The present invention relates to a dual-channel federated graph learning method, device and system with privacy protection, belonging to the technical field of federated learning. Background Art

[0002] Distributed graph learning faces many challenges in the current fields of cloud computing, system security, and federated learning technology. With the strict implementation of data protection regulations (such as GDPR), how to achieve efficient model training while protecting data privacy has become an urgent problem to be solved. Traditional distributed graph learning methods often have information missing problems between clients, resulting in unsatisfactory model classification results, and weak data privacy and model security during training.

[0003] Specifically, existing graph learning methods have difficulty effectively handling the problem of non-independent and identically distributed (Non-IID) label distribution offset and incomplete cross-client data when aggregating data across clients. These problems will seriously affect the training effect and robustness of the model. In addition, there is a risk of privacy leakage during the transmission and aggregation of client data. The server may infer the client's private data through the gradient information uploaded by the client, thereby causing privacy leakage problems.

[0004] To overcome these challenges, researchers have begun to explore federated learning technologies with privacy protection capabilities. However, existing federated learning technologies often have difficulty balancing data privacy protection, model training efficiency, and model robustness when processing graph data. In particular, when facing untargeted attacks from malicious clients, existing methods often have difficulty ensuring the stability and reliability of the system.

[0005] Therefore, there is an urgent need for a federated graph learning method that can simultaneously solve the problems of data privacy protection, model training efficiency, and model robustness. This method needs to be able to overcome the data island problem while protecting the original privacy data of the client and the data privacy during the federated learning training process. It can also have good classification results for non-independent and identically distributed subgraph-level federated learning data. At the same time, it can well ensure the robustness of the system in the face of untargeted attacks from malicious clients, thereby improving classification accuracy and efficiency. Summary of the invention

[0006] The purpose of the present invention is to provide a dual-channel federated graph learning method, device and system with privacy protection to solve the problems existing in the prior art and realize efficient, secure and robust federated graph learning.

[0007] In order to achieve the above object, the present invention is implemented through the following technical solutions: The method comprises: the client performs feature aggregation on a subgraph held locally, obtains aggregated node feature information, and sends the aggregated node feature information to the server; The server uses the aggregated node feature information to randomly perform cross-client feature aggregation between two client subgraphs to obtain cross-client subgraph information; The client interacts with the server based on the cross-client subgraph information and obtains the global meta-model parameters through meta-learning. The client divides the subgraph nodes in the subgraph information into a support set and a query set, updates the local model parameters on the support set, and calculates the loss gradient on the query set. The gradient is uploaded to the server, and the server aggregates the gradients from all clients and updates the global meta-model parameters through the gradient descent algorithm. The client updates the local model parameters according to the parameters of the global meta-model, and the global meta-model and the local model adopt a graph convolutional neural network with a dual-channel attention feature fusion structure; The client updates the local model parameters according to the support set of the local subgraph nodes and uploads it to the server. The server uses the federated averaging algorithm to fine-tune the global meta-model parameters and obtain the global model.

[0008] Preferably, the graph convolutional neural network of the dual-channel attention feature fusion structure includes: a linear layer, a ReLU layer, a Dropout layer, a dual-channel convolution module, a Dropout+ReLU layer, an attention feature fusion module and a Log_Softmax activation function; the dual-channel convolution module uses mean aggregation and truncated mean aggregation as the feature aggregation method for each convolution channel, respectively.

[0009] Preferably, the node features output by the Dropout layer are Node features are obtained through mean aggregation , node features are obtained through truncated mean aggregation , the node features and After the Dropout+ReLU layer, the attention feature fusion module is used to obtain the node features. , the formula is as follows: , in, Represented by node features The obtained attention weights.

[0010] Preferably, the truncated mean aggregation is obtained by the following formula: , in, In order to discard the proportion of extreme values, Indicates that it is a node The set of neighbor nodes of Representation Node In the The feature representation of the layer, Representation Node The neighbor nodes of Representation Node The neighbor nodes of Representation Node In the Aggregate value of the layer.

[0011] Preferably, the client interacts with the server according to the cross-client subgraph information and obtains the global meta-model parameters by meta-learning, specifically in the following manner: Divide the subgraph nodes of the cross-client subgraph information into a support set and a query set; The client updates the local model parameters through the support set and sets the labels corresponding to the support set nodes. and the output of the current node of the support set The cross entropy loss is used as the loss function for updating the support set of the local model; Get the local gradient through the query set and upload it to the server. Get the local gradient through the following formula : , in, represents the gradient of the model parameters, Represents the loss on the query set, which is the cross entropy loss of the label corresponding to the query set node and the output of the current node in the query set; According to the gradients uploaded by each client, the server sums the gradients and performs gradient descent to obtain the global meta-model parameters.

[0012] Preferably, the client updates the local model parameters according to the support set of the local subgraph nodes , the specific formula is as follows: , in, represents the learning rate, Represents the loss function of this stage, and the calculation formula is as follows: , in, and is a hyperparameter, express The l2 norm of is the regularization term, Indicates Metamodel parameters of the wheel, Indicates The feature vector on the client support set.

[0013] Preferably, the specific method of subgraph cross-client feature aggregation is as follows: feature information of neighboring nodes around the aggregation node is updated through the feature information of the current node.

[0014] Preferably, the data interaction process between the client and the server is encrypted using a multi-key fully homomorphic encryption algorithm, and each client uses a separate encryption key.

[0015] A dual-channel federated graph learning device with privacy protection includes a processor and a memory storing program instructions, wherein the processor is configured to execute the dual-channel federated graph learning method with privacy protection when running the program instructions.

[0016] A dual-channel federated graph learning system with privacy protection, comprising: The feature aggregation module is configured to perform feature aggregation on the subgraph held locally by the client, send the aggregated node feature information on the subgraph aggregated locally by the client to the server, randomly perform cross-client feature aggregation between two client subgraphs, and obtain cross-client subgraph information.

[0017] The dual-channel attention fusion module is configured to aggregate the features of each convolution channel through mean aggregation and truncated mean aggregation; The multi-key fully homomorphic encryption privacy protection module is configured to encrypt the interactive data between the client and the server through a multi-key fully homomorphic encryption algorithm.

[0018] The federated learning module is configured to enable the client and server to interact with each other through meta-learning to obtain global meta-model parameters; and to fine-tune the global meta-model parameters to obtain a global model.

[0019] The advantages of the present invention are as follows: the dual-channel federated graph learning method of the present invention realizes isolated transmission and processing of data by introducing two independent communication channels, namely the data channel and the model channel, and effectively reduces the risk of data leakage. Users do not need to directly share original data, but only need to interact through encrypted or anonymized data or model parameters, thereby greatly enhancing the data privacy protection capability while ensuring the learning effect.

[0020] This method uses the characteristics of graph-structured data to build a graph neural network model, which can capture the complex relationships between data and improve the model's ability to understand and represent data. At the same time, the dual-channel design allows data updates and model updates to be performed in parallel, reducing communication delays and waiting time, and further improving the overall efficiency and effectiveness of federated learning.

[0021] By implementing privacy-preserving federated learning, the present invention breaks the data silo phenomenon and promotes data collaboration and resource sharing among different organizations or institutions. All parties can jointly participate in the training and optimization of the model without leaking their sensitive data, thereby accelerating technological progress and the development of innovative applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention.

[0023] Figure 1 It is a schematic diagram of the process of the present invention; Figure 2 This is the dual-channel graph attention fusion network framework diagram of the present invention; Figure 3 The overall flow chart of robust federated learning training between client and server; Figure 4 This is a diagram of the multi-key fully homomorphic encryption process; Figure 5 is the average accuracy score of the Cora dataset at different feature perturbation levels; Figure 6 is the average accuracy score of the CiteSeer dataset at different feature perturbation levels; Figure 7 is the average accuracy score of the PubMed dataset at different feature perturbation levels; Figure 8 It is the average accuracy score under different feature mask ratios of the Cora dataset; Fig. 9 is the average accuracy score under different feature mask ratios of the CiteSeer dataset; Fig.10 The average accuracy scores under different feature mask ratios of the PubMed dataset. DETAILED DESCRIPTION

[0024] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0025] Example 1 like Figure 1 As shown, a dual-channel federated graph learning method with privacy protection includes the following steps: S1: The client performs feature aggregation on the subgraph held locally, obtains the aggregated node feature information, and sends it to the server.

[0026] This process involves using the connection relationship between nodes in the graph to aggregate the feature information of neighboring nodes around the node and update the feature representation of the current node. Through this process, the client obtains the aggregated node feature information.

[0027] S2: The server uses the aggregated node feature information to randomly perform cross-client feature aggregation between two client subgraphs to obtain cross-client subgraph information.

[0028] The purpose of this step is to further improve the information of the global graph and solve the problem of missing information between clients. The subgraphs of each client after cross-client feature aggregation will be used as the initial subgraphs used in the subsequent federated learning training process.

[0029] S3: The client interacts with the server based on the cross-client subgraph information and obtains the global meta-model parameters through meta-learning. The client divides the sub-graph nodes in the sub-graph information into a support set and a query set, updates the local model parameters on the support set, and calculates the loss gradient on the query set. The gradient is uploaded to the server, and the server aggregates the gradients from all clients and updates the global meta-model parameters through the gradient descent algorithm.

[0030] S4: The client updates the local model parameters according to the parameters of the global meta-model, and the global meta-model and the local model adopt a graph convolutional neural network with a dual-channel attention feature fusion structure.

[0031] S5: The client updates the local model parameters according to the support set of the local subgraph nodes and uploads it to the server. The server uses the federated averaging algorithm to fine-tune the global meta-model parameters and obtain the global model.

[0032] As a refinement of the above embodiment, Figure 2 As shown, the graph convolutional neural network of the dual-channel attention feature fusion structure includes: a linear layer, a ReLU layer, a Dropout layer, a dual-channel convolution module, a Dropout+ReLU layer, an attention feature fusion module and a Log_Softmax activation function; the dual-channel convolution module uses mean aggregation and truncated mean aggregation as the feature aggregation method of each convolution channel respectively.

[0033] The graph convolutional neural network feature processing process of the dual-channel attention feature fusion structure is as follows: The input node features are sequentially passed through the linear layer, ReLU layer, and Dropout layer. The node features output by the Dropout layer Node features are obtained through mean aggregation , node features are obtained through truncated mean aggregation , the node features and After the Dropout+ReLU layer, the attention feature fusion module is used to obtain the node features. , the formula is as follows: , in, Represented by node features The obtained attention weights.

[0034] In this embodiment, the traditional single-channel graph convolution model is improved by increasing the number of channels from a single channel to dual channels for parallel processing; The graph convolution process is obtained by the following formula Neighbor aggregation of layers: , in, Is a node and The normalized edge weights between is the node feature representation of the next layer, Representation Node Neighbor nodes The neighbor set of is obtained by the following formula: ; As a refinement of the above embodiment, the truncated mean aggregation is obtained by the following formula: , in, In order to discard the proportion of extreme values, Indicates that it is Layer Node The aggregate value of Indicates that it is a node The set of neighbor nodes of No. The updated representation vector of the layer, node neighbor nodes.

[0035] In the federated learning system, the data held by the client exists in the form of subgraphs and is divided into honest clients and malicious clients according to the nature of their behavior. Honest clients directly use their original subgraph data to participate in cross-client feature aggregation and federated learning processes, while malicious clients may perform untargeted attacks on their local subgraphs, including adding or removing nodes, adding or removing edges, or disrupting node features, etc. These behaviors are intended to disrupt model training and threaten its robustness. Among them, the disrupted node features are operated by the following formula: , in, is the noise level, From the normal distribution Generate and A random tensor of the same shape.

[0036] To address the above problems, especially the interference of malicious clients on feature aggregation, a graph convolutional neural network (GCN) with a dual-channel attention feature fusion structure is proposed. This network design enhances the model's robustness to noise and outliers through an innovative feature aggregation strategy.

[0037] In graph convolutional networks, feature aggregation is a key step in training and updating node features. The traditional mean aggregation method averages the feature values ​​of all neighboring nodes indiscriminately, but this strategy is susceptible to node feature perturbations when there are malicious client attacks.

[0038] Mean aggregation indiscriminately averages all feature values ​​from neighboring nodes. When the client performs non-targeted attack behavior, it is easily affected by the feature disturbance of some nodes. Truncated mean aggregation discards some extremely abnormal feature values ​​and only averages the remaining feature values, thereby alleviating the fluctuations caused by malicious node feature disturbances to a certain extent. This method aims to balance the impact of outliers and normal data and improve the robustness of the model.

[0039] This embodiment uses a graph convolutional neural network with a dual-channel attention feature fusion structure to effectively respond to untargeted attacks from malicious clients while protecting client privacy, thereby improving the overall performance and robustness of the federated learning system.

[0040] As a refinement of the above embodiment, in step S3, the client interacts with the server according to the cross-client subgraph information and obtains the global meta-model parameters by meta-learning, and the specific method is as follows: S301: In each round, server S first sets the global model parameters Sent to each client, in the first round, the global model parameters Initialized randomly; divide the subgraph nodes of cross-client subgraph information into support set and query set.

[0041] S302: Client Receive global model parameters As local model parameters, in the support set Perform multiple gradient descents on it to update the model parameters. The formula is as follows: , in, For the On the client The local model parameters of the wheel, represents the learning rate, Update the loss function of the support set for the local model. The specific formula is as follows: , in, To support labels corresponding to set nodes, To support the output of the current node, is the cross entropy loss.

[0042] S303: Obtain the local loss gradient through the query set and upload it to the server. Obtain the local gradient through the following formula : , in, represents the gradient of the model parameters, Represents the loss on the query set. The specific formula is as follows: , in, is the output of the current node of the query set, The label corresponding to the query set node.

[0043] S304: According to the gradients uploaded by each client, the server sums the gradients and performs gradient descent to obtain the global meta-model parameters. The formula is as follows: , in, No. The global metamodel parameters obtained by round 1 are: represents the learning rate, Indicates the number of clients.

[0044] As a refinement of the above embodiment, in step S5, the client updates the local model parameters according to the support set of the local subgraph node and uploads it to the server. The server uses the federated average algorithm to fine-tune the global metamodel parameters to obtain the global model. The specific method is as follows: S501: Client Receive parameters from the server As local model parameters , each client supports the set The local model parameters are updated and uploaded to the server. The local model parameter update is obtained through the following formula: , in, represents the learning rate, represents the gradient of the model parameters, Represents the loss function of this stage, and the formula is as follows: , in, and is a hyperparameter, express The l2 norm of Indicates Metamodel parameters of the wheel, Indicates The client supports the node feature vector, represents the cross entropy loss, is a regularization term used to constrain the difference between the local model and the global model while minimizing the local model loss.

[0045] S502: Based on the model parameters uploaded by each client, the server uses the federated average algorithm to fine-tune and update the global model parameters, and obtains the global model update through the following formula: , As a refinement of the above embodiment, the specific method of subgraph cross-client feature aggregation is as follows: feature information of neighboring nodes around the aggregation node is obtained by the following formula: in, Indicates that it is Layer Node The aggregate value of Representation Node The neighbor nodes of Indicates that it is a node The set of neighbor nodes of Indicates Updated representation vector.

[0046] The current node information is updated using the information of the neighbors around the aggregated node, which is obtained through the following formula: , in: It is Layer Node The aggregate value of Layer Node All neighbor nodes of The representation vector is obtained, It is The updated representation vector of the layer.

[0047] As a refinement of the above embodiment, throughout the entire process, the client's local data and the model parameters during the training process are strictly protected. The data is aggregated and processed locally on the client and does not leave the local environment. At the same time, a multi-key fully homomorphic encryption method is used to protect the data transmitted between the client and the server to ensure the security and privacy of the data.

[0048] The data interaction process between the client and the server is encrypted using a multi-key fully homomorphic encryption algorithm, and each client uses a separate encryption key.

[0049] The multi-key fully homomorphic encryption method is used to protect the data transmitted during the interaction between the client and the server during the entire federated learning process, which is reflected in the following processes: Process 1: Cross-client node feature aggregation process, this process aims to protect the privacy of client node functions: Step 1: The client uses its encryption key Message aggregation of encrypted node features , and Upload to the server.

[0050] Step 2: The server aggregates the encryption characteristics of each node , and aggregate the features Sent to each client.

[0051] Step 3: Clients on the same node use their own decryption keys Jointly decrypt the aggregated feature results and obtain .

[0052] Process 2: Obtaining global metamodel parameters. This process aims to protect the privacy of the loss of interaction between the meta-parameter training phase and the client: Step 1: Client uses their encryption key Crypto local loss , and upload To the server; Step 2: The server updates the meta parameters using the following formula and sends it to each client: , Step 3: Client accepts and combines decryption meta parameters , get the decrypted meta parameters For subsequent training.

[0053] Process 3: Fine-tune the global model using federated learning. This process aims to protect the privacy of model updates in the interaction between the federated learning fine-tuning phase and the client. Step 1: Client uses their encryption key Encrypting local model parameters , and upload To the server; Step 2: The server updates the model parameters using the following formula and sends it to each client: Step 3: The client accepts and jointly decrypts the updated model parameters , get the decrypted model parameters Used for subsequent local model updates.

[0054] This embodiment can be applied to the product recommendation of the global e-commerce platform. Each client represents a country. The users and transaction information within each country are used as nodes and their attributes on the graph, and the transaction information between countries is used as the cross-client edge. The global main server is responsible for generating a classification of each user's commodity transaction, further distinguishing users according to the classification of transaction commodities, and then making purposeful and directional commodity recommendations to users. For example, a user purchased xx brand anti-dandruff shampoo and xx brand hair growth essence. According to these transaction records, the user's transaction type can be marked as "scalp care". Regarding eliminating the impact of abnormal client behavior, for example, a user buys more snacks in a daily basis, but is mixed with a large amount of phone recharge records, then the user's classification should still be "snacks", and will not be mistakenly classified as "phone recharge". Regarding information security issues, some users may use platform vulnerabilities to steal other users' personal privacy information and transaction records for fraud.

[0055] Take the example of Amazon e-commerce platform. Some countries (clients) may try to disrupt the model training process when uploading local model updates, such as maliciously tampering with local original data (modifying transaction record sets and user information, etc.) to affect model training, stealing model parameters uploaded by other countries to infer their local data, etc. In other words, the clients participating in the training are malicious. The ultimate goal of malicious clients may be to disrupt the model training process so that the final trained model cannot recognize its malicious data, such as poisoning attacks, or to steal other clients' personal information for illegal resale, such as reasoning attacks, etc.

[0056] The present invention can overcome the data island problem while protecting the client's original privacy data and the data privacy during the federated learning training process. It can also have good classification results for sub-graph-level federated learning data that are not independent and identically distributed. At the same time, in the face of untargeted attacks from malicious clients, the present invention can well ensure the robustness of the system, thereby improving the classification accuracy and efficiency.

[0057] Example 2 A dual-channel federated graph learning system with privacy protection, comprising: The feature aggregation module is configured to perform feature aggregation on the subgraph held locally by the client, send the aggregated node feature information on the subgraph aggregated locally by the client to the server, randomly perform cross-client feature aggregation between two client subgraphs, and obtain cross-client subgraph information.

[0058] The dual-channel attention fusion module is configured to aggregate the features of each convolution channel through mean aggregation and truncated mean aggregation; The multi-key fully homomorphic encryption privacy protection module is configured to encrypt the interactive data between the client and the server through a multi-key fully homomorphic encryption algorithm.

[0059] The federated learning module is configured to enable the client and server to interact with each other through meta-learning to obtain global meta-model parameters; and to fine-tune the global meta-model parameters to obtain a global model.

[0060] The disclosed embodiment also provides a dual-channel federated graph learning device with privacy protection, including a processor and a memory. Optionally, the device may also include a communication interface and a bus. Among them, the processor, the communication interface, and the memory can communicate with each other through the bus. The communication interface can be used for information transmission. The processor can call the logic instructions in the memory to execute the dual-channel federated graph learning method with privacy protection of the above embodiment.

[0061] In addition, the logic instructions in the above-mentioned memory can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product.

[0062] The memory, as a computer-readable storage medium, can be used to store software programs and computer executable programs, such as program instructions / modules corresponding to the method in the embodiment of the present disclosure. The processor executes the functional application and data processing by running the program instructions / modules stored in the memory, that is, the dual-channel federated graph learning method with privacy protection in the above embodiment is implemented.

[0063] The memory may include a program storage area and a data storage area, wherein the program storage area may store an operating system and an application required for at least one function; the data storage area may store data created according to the use of the terminal device, etc. In addition, the memory may include a high-speed random access memory and may also include a non-volatile memory.

[0064] In order to verify the effectiveness of a dual-channel federated graph learning method with privacy protection in an embodiment of the present invention, in a specific embodiment, the Cora dataset, CiteSeer dataset and PubMed dataset are selected to evaluate the above method.

[0065] The above datasets are usually used for node classification tasks. Cora is a citation network dataset for machine learning, containing 2708 nodes (papers), 5429 edges (citation relationships), and 7 category labels (case-based, genetic algorithm, neural network, probabilistic method, reinforcement learning, rule learning, theory); CiteSeer is a citation network dataset that records citation relationships between papers, containing 3312 nodes, 4723 edges, and 6 label categories (Agents, AI, DE, IR, ML, HCI); PubMed is a citation network dataset for diabetes scientific publications, containing 19717 nodes, 44338 edges, and 3 label categories. Table 1 provides detailed information for each dataset.

[0066] It should be noted that the experimental setting of this embodiment adopts the basic architecture of 2-layer GCN, with 16 hidden units, and uses mean aggregation and truncated mean aggregation methods. In order to simulate the training scenario with fewer samples, we randomly select 40 instances from each class in the dataset as the support set, and use a total of 140 samples as the query set. These samples are evenly distributed among all clients, and the number of clients is set to 20; in order to evaluate the convergence speed of the model, 20 global training sets, 40 local training sets and 10 fine-tuning sets are set; the learning rate is set to is set to 0.01, the meta-learning rate is set to 0.02, and the regularization coefficient Set to 2, Set to 0.5.

[0067] Specifically, this embodiment conducts experiments on six methods including the method proposed in the present invention based on two different experimental scenarios. It should be noted that the experiments in each scenario are conducted independently. To ensure the accuracy of the experiments, each experimental data comes from the average value of five repeated experiments, and the experimental results between them do not affect each other.

[0068] Furthermore, the results of experiments on each method under different Dirichlet data distributions in the Cora, CiteSeer and PubMed datasets are used to evaluate the performance of the example method of the present invention relative to other methods. =10000, 1000 means independent and identically distributed or =100, 10, 1 represents non-IID label distribution deviation. The results of the six methods under five label distribution deviation scenarios on Cora, CiteSeer and PubMed datasets are shown in Table 1 Table 1 Average accuracy scores of six experimental methods on three datasets Table 1 shows the accuracy scores of the six methods on the Cora, CiteSeer and PubMed datasets in five label distribution offset scenarios. The bold data represents the optimal accuracy in the non-overlapping scenario. It can be seen that on the Cora dataset, the method proposed in the present invention is superior to other methods in terms of accuracy; when the data is not independent and identically distributed, the method of adding and aggregating features and using meta-learning proposed in the present invention has significantly improved the average accuracy compared with other methods, which is about 2 times on the Cora dataset, about 1.75 times on the CiteSeer dataset, and about 1.48 times on the PubMed dataset. When using the truncated mean aggregation method, the performance on the Cora dataset is improved compared to the mean aggregation, and the improvement on the PubMed dataset is more obvious, up to 4.81%; the accuracy on the CiteSeer dataset changes less by ±0.5%; in addition, compared with the single truncated mean aggregation, the dual-channel attention aggregation has an average improvement of 1.384±0.05% on Cora and Citeseer, and a more significant improvement on PubMed, averaging 2.786%, especially under non-IID conditions.

[0069] In order to verify the robustness of the method in this embodiment during the federated learning training process, two typical untargeted node attacks are used to simulate real scenarios where the data may not be completely clean: Node feature perturbation: This attack adds random noise to the node features in the graph, making the features slightly different from the original values; Node feature masking. This attack masks a certain percentage of the features of each node in the graph, effectively hiding some of the information; Furthermore, experiments were conducted on the mean aggregation, truncated mean aggregation, and dual-channel attention aggregation methods in the Cora, CiteSeer, and PubMed datasets to evaluate the average accuracy scores of the example method of the present invention relative to the performance of other methods in the untargeted attack scenario. The average accuracy scores under different feature perturbation levels are shown in Figure 2. Figure 5-Figure 7 As shown in Figure 2, the average accuracy scores under different feature mask ratios are Figure 8-Figure 10 shown.

[0070] like Figure 3 As shown, compared with the mean and truncated mean aggregation methods on all three datasets, the dual-channel aggregation of the embodiment of the present invention has a higher accuracy score. It is worth noting that as the noise level increases, the improvement in accuracy becomes more obvious, highlighting the robustness of the method of the embodiment of the present invention against untargeted client attacks. This effect is particularly evident on smaller datasets such as Cora; exist Figure 4 In the , the average accuracy score decreases significantly with the increase of the node feature mask ratio, and the single-channel method shows greater fluctuations on the PubMed dataset. However, the method proposed in the embodiment of the present invention maintains a higher accuracy than other single-channel aggregation methods.

[0071] Furthermore, the stability score is used as an indicator of model robustness, and the stability score is defined as the absolute value of the average accuracy change under non-targeted attacks compared to clean data. The stability score of the embodiment of the present invention was experimentally evaluated on three aggregation methods. The accuracy and stability scores of the model on three data sets under node feature perturbation are shown in Table 2, and the accuracy and stability scores of the model on three data sets under node feature masking are shown in Table 3. A lower stability score indicates that the aggregation method is more stable and more robust to non-targeted attacks.

[0072] Table 2 Accuracy and stability scores of the model on three datasets under node feature perturbation Table 3 Accuracy and stability scores of the model under node feature mask on three datasets As shown in Table 2, under the node feature perturbation, the dual-channel method provided by the embodiment of the present invention has the best stability score, which is improved by 10 times on CiteSeer and even 100 times on Cora, which shows that the dual-channel aggregation is more robust than the single aggregation method. However, on large datasets such as PubMed, although the accuracy score significantly exceeds that of other aggregation methods, the stability score of the dual-channel aggregation cannot be stably maintained at a lower value. When the perturbation level is 0.5, the stability score of the dual-channel aggregation is worse than that of other methods; As shown in Table 3, compared with other methods, the dual-channel aggregation method provided by the embodiment of the present invention shows a higher stability score under the node feature mask. Although the stability under the node feature mask is not as low as the stability under the node feature perturbation, the dual-channel aggregation shows consistency on all three datasets. Specifically, it achieves an average improvement of 2.1 times on Cora, 1.37 times on CiteSeer, and 1.73 times on PubMed, which shows that the dual-channel aggregation maintains higher robustness under the feature mask and shows more consistent performance than other methods.

[0073] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A dual-channel federated graph learning method with privacy protection, characterized in that: include: The client performs feature aggregation on the subgraph held locally, obtains the aggregated node feature information, and sends it to the server; The server uses the aggregated node feature information to randomly perform cross-client feature aggregation between two client subgraphs to obtain cross-client subgraph information; The client interacts with the server based on the cross-client subgraph information and obtains the global meta-model parameters through meta-learning; The client divides the subgraph nodes in the subgraph information into a support set and a query set, updates the local model parameters on the support set, and calculates the loss gradient on the query set. The gradient is uploaded to the server, and the server aggregates the gradients from all clients and updates the global meta-model parameters through the gradient descent algorithm. The client updates the local model parameters according to the parameters of the global meta-model, and the global meta-model and the local model adopt a graph convolutional neural network with a dual-channel attention feature fusion structure; The client updates the local model parameters according to the support set of the local subgraph nodes and uploads it to the server. The server uses the federated averaging algorithm to fine-tune the global meta-model parameters and obtain the global model.

2. The dual-channel federated graph learning method with privacy protection according to claim 1, characterized in that: The graph convolutional neural network with the dual-channel attention feature fusion structure includes: a linear layer, a ReLU layer, a Dropout layer, a dual-channel convolution module, a Dropout+ReLU layer, an attention feature fusion module and a Log_Softmax activation function; the dual-channel convolution module uses mean aggregation and truncated mean aggregation as the feature aggregation method of each convolution channel respectively.

3. The dual-channel federated graph learning method with privacy protection according to claim 2, characterized in that: The node features output by the Dropout layer Node features are obtained by mean aggregation , node features are obtained through truncated mean aggregation , the node features and After the Dropout+ReLU layer, the attention feature fusion module is used to obtain the node features. , the formula is as follows: ， in, Represented by node features The obtained attention weights.

4. The dual-channel federated graph learning method with privacy protection according to claim 3, characterized in that: The truncated mean aggregation is obtained by the following formula: ， in, In order to discard the proportion of extreme values, Indicates that it is a node The set of neighbor nodes of Representation Node In the The feature representation of the layer, Representation Node The neighbor nodes of Representation Node The neighbor nodes of Representation Node In the Aggregate value of the layer.

5. The dual-channel federated graph learning method with privacy protection according to claim 2, characterized in that: The client interacts with the server based on the cross-client subgraph information and obtains the global meta-model parameters through meta-learning. The specific method is as follows: Divide the subgraph nodes of the cross-client subgraph information into a support set and a query set; The client updates the local model parameters through the support set and sets the labels corresponding to the support set nodes. and the output of the current node of the support set The cross entropy loss is used as the loss function for updating the support set of the local model; Get the local gradient through the query set and upload it to the server. Get the local gradient through the following formula : ， in, represents the gradient of the model parameters, Represents the loss on the query set, which is the cross entropy loss of the label corresponding to the query set node and the output of the current node in the query set; According to the gradients uploaded by each client, the server sums the gradients and performs gradient descent to obtain the global meta-model parameters.

6. The dual-channel federated graph learning method with privacy protection according to claim 5, characterized in that: The client updates the local model parameters based on the support set of the local subgraph nodes , the specific formula is as follows: ， in, represents the learning rate, Represents the loss function of this stage, and the calculation formula is as follows: ， in, and is a hyperparameter, express The l2 norm of is the regularization term, Indicates Metamodel parameters of the wheel, Indicates The feature vector on the client support set.

7. The dual-channel federated graph learning method with privacy protection according to claim 1, characterized in that: The specific method of subgraph cross-client feature aggregation is as follows: feature information of neighboring nodes around the aggregation node is used to update the current node information.

8. The dual-channel federated graph learning method with privacy protection according to claim 1, characterized in that: The data interaction process between the client and the server is encrypted using a multi-key fully homomorphic encryption algorithm, and each client uses a separate encryption key.

9. A dual-channel federated graph learning device with privacy protection, comprising a processor and a memory storing program instructions, characterized in that: The processor is configured to execute the dual-channel federated graph learning method with privacy protection as described in any one of claims 1-8 when running the program instructions.

10. A dual-channel federated graph learning system with privacy protection for executing any of the methods of claims 1-8, characterized in that: include: The feature aggregation module is configured to perform feature aggregation on the subgraph held locally by the client, send the aggregated node feature information on the subgraph aggregated locally by the client to the server, randomly perform cross-client feature aggregation between two client subgraphs, and obtain cross-client subgraph information. The dual-channel attention fusion module is configured to aggregate the features of each convolution channel through mean aggregation and truncated mean aggregation; The multi-key fully homomorphic encryption privacy protection module is configured to encrypt the interactive data between the client and the server through a multi-key fully homomorphic encryption algorithm. The federated learning module is configured to enable the client and server to interact with each other through meta-learning to obtain global meta-model parameters; and to fine-tune the global meta-model parameters to obtain a global model.

Images