Model aggregation method, system and device for distributed graph learning

Abstract

This invention discloses a model aggregation method, system, and apparatus for distributed graph learning, relating to the field of distributed machine learning. A central server constructs a Harary graph to define the physical communication neighborhood of clients and distributes a unified random graph as a probe to extract the response representation vectors of each client. Similarity weights are then calculated and corrected using Non-IID quantization differences. Each client sparsifies the mask, projects it along the main update direction, and adds noise to generate a purified mask, which is then exchanged with neighbors to recover a pseudo-gradient that retains the main direction but does not reveal the plaintext. Finally, each client performs local weighted aggregation of the pseudo-gradient based on similar neighbors and weights. This invention provides a novel paradigm for local approximate aggregation with low communication overhead, effectively resisting abnormal updates under the premise of zero plaintext leakage, and improving the convergence and robustness of distributed graph models.

Description

Technical Field

[0001] This invention belongs to the field of distributed graph learning, which combines distributed learning with graph neural networks. Specifically, it relates to a model aggregation method, system, and apparatus for distributed graph learning. Background Technology

[0002] With the development of artificial intelligence technology, machine learning methods based on graph-structured data have been widely applied in fields such as social network analysis, recommender systems, knowledge graph mining, and bioinformatics. Graph Neural Networks (GNNs) achieve efficient modeling of complex graph-structured data by iteratively aggregating information about nodes and their neighbors. However, in practical applications, graph data is often distributed across multiple independent institutions or devices (such as different companies, organizations, or terminal devices). Due to reasons such as data privacy protection, data ownership, and legal compliance, it is difficult for the participating parties to directly share the original graph data, thus limiting the application of centralized graph learning models.

[0003] Distributed learning (or federated learning) achieves a collaborative learning mechanism where "data doesn't leave the local machine" by training models locally on multiple clients and aggregating them on the server, providing an effective way to solve data silos and privacy protection issues. However, when combining distributed learning with graph neural networks to form distributed graph learning, the graph data structure, node features, and label distribution held by each client often differ significantly, resulting in non-independent and identically distributed (Non-IID) data problems. This leads to large differences in the model update direction between different clients, severely affecting the convergence and stability of the global model.

[0004] On the other hand, in distributed learning, servers typically need to aggregate model updates or gradient information uploaded by clients. Existing distributed learning methods usually employ techniques such as secure aggregation, homomorphic encryption, or differential privacy to protect model updates. However, these methods face problems of high communication overhead and computational complexity in distributed graph learning scenarios. Furthermore, traditional cryptographically secure aggregation requires global summation for the masks to cancel each other out, making it difficult for the server to obtain the differences between individual clients and thus hindering the effective identification of abnormal or malicious client updates. Simultaneously, the global average aggregation method in traditional distributed learning fails to fully consider the similarity differences in model representations among clients, making it vulnerable to abnormal updates or malicious attacks in non-IID environments, thus reducing the overall robustness of the system. Summary of the Invention

[0005] Purpose of the invention: Unlike existing secure aggregation schemes that require the server to obtain the global accurate sum of model updates, the purpose of this invention is to provide a model aggregation method, system, and device for distributed graph learning. Through a novel distributed graph learning aggregation paradigm, while protecting the privacy of client model updates, the client can recover learnable pseudo-gradients among semantically similar neighbors and perform local personalized aggregation.

[0006] Technical solution: This invention provides a model aggregation method for distributed graph learning, comprising the following steps:

[0007] S1. The central server constructs a Harary communication graph based on the set of clients participating in distributed graph learning, determines the communication neighbor set of each client according to the adjacency relationship of the Harary communication graph, and sends it to the corresponding client.

[0008] S2. Each client trains the local graph neural network model locally based on the local graph data to obtain the local model update;

[0009] S3. The central server sends a unified random graph with the same graph structure and initial feature configuration to each client; calculates the model response similarity between clients based on the client representation vectors extracted after each client inputs the unified random graph; corrects the model response similarity by combining the non-independent and identically distributed (Non-IID) quantization difference parameters between clients to obtain a corrected similarity; and sends a set of similar neighbors to each client after filtering from their communication neighbor set based on the similarity weight.

[0010] S4. Each client generates a private mask and a pair of masks, and updates the mask gradient by combining the local model.

[0011] S5. Each client generates a merged mask vector based on the private mask and the paired mask, and performs sparsification, projection along the local model update direction, and random noise addition on the merged mask vector in sequence to generate a purified mask. The mask gradient and the purified mask are then sent to neighboring clients in their communication neighbor set.

[0012] S6. Each client receives the mask gradient and purification mask sent by the neighboring client, and calculates and recovers the pseudo gradient that retains the main direction of the neighboring client model update through the mask gradient and the purification mask.

[0013] S7. Each client performs a weighted summation of the pseudo gradients of each neighbor client in the similar neighbor set based on the received similar neighbor set and the corresponding similarity weights, and obtains the aggregated update result to update the parameters of the local graph neural network model.

[0014] The present invention also provides a model aggregation system for distributed graph learning, including a central server and multiple clients with communication connections, which collaboratively execute the above aggregation method.

[0015] The present invention also provides a model aggregation device for distributed graph learning, which is deployed on the client side and the central server side respectively, and includes corresponding functional modules for performing the above method steps.

[0016] Beneficial effects: Compared with the prior art, the advantages of the present invention are as follows:

[0017] (1) This invention breaks away from the traditional fully connected communication method of distributed learning. At the physical topology level, a structured communication topology is constructed based on the Harary graph, which strictly restricts the communication relationships between clients to a limited safe topological neighborhood. At the logical level, model similarity is further used to construct a logically similar neighbor set. This dual-graph mechanism design of "physical constraint + logical optimization" significantly reduces the communication complexity between distributed nodes while ensuring the overall network communication connectivity, and significantly improves the stability of the system in the event of client dynamic disconnection or external changes.

[0018] (2) This invention sends a unified random graph to each client as a consistent input stimulus (probe), uses a graph neural network to extract the embedding vectors of intermediate layer nodes, and aggregates them to generate client response representation vectors that represent the functions. By combining this representation with Non-IID quantization difference parameters, a high-precision similarity weight is calculated, enabling the server to accurately characterize the model state and data distribution differences of different clients under Non-IID data without touching the local real data and plaintext model parameters, providing a precise measurement basis for personalized aggregation.

[0019] (3) In view of the strict limitation that traditional cryptographic security aggregation can only offset the mask after global exact summation, this invention generates a purified mask by sparse processing of the merged mask, projecting along the main direction of model update, and adding random noise perturbation. This one-way mathematical dimensionality reduction operation enables the receiver (neighbor client) to recover the usable pseudo gradient that retains the original main direction of update but never leaks the original sensitive information. With minimal privacy protection overhead, it perfectly supports client-side local similarity-based, non-globally exact personalized weighted summation.

[0020] (4) Benefiting from the local recoverability of pseudo-gradients, the aggregation position of this invention is decentralized to the client side. Analysis of variance is used to identify and eliminate abnormal clients. Each client can autonomously select similar neighbor sets from non-abnormal communication neighbors for weighted aggregation, allowing clients with similar data distributions or model characteristics to participate in collaboration first. This not only effectively resists malicious updates and large-biased data poisoning interference, but also perfectly adapts to the local personalized learning needs in Non-IID environments, significantly improving the convergence speed and overall robustness during distributed model training. Combined with the anti-attack experiment verification in Figure 4, this invention maintains significantly higher accuracy under a malicious attack rate of up to 50% compared to the comparison scheme under six dataset tests. Combined with the accuracy curve verification in Figure 5, this invention achieves faster convergence and ultimately higher model prediction accuracy compared to traditional mainstream federated aggregation algorithms under six dataset tests. Attached Figure Description

[0021] Figure 1 This is a sequence diagram of the system interaction process according to an embodiment of the present invention;

[0022] Figure 2 This is a schematic diagram of the system modules according to an embodiment of the present invention;

[0023] Figure 3 This is a schematic diagram of the system flow according to an embodiment of the present invention;

[0024] Figure 4 This demonstrates the effectiveness of the invention against malicious client attacks under six different datasets.

[0025] Figure 5 This is an accuracy curve of the present invention tested on six datasets. Detailed Implementation

[0026] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings, but the scope of protection of the present invention is not limited to the embodiments described.

[0027] Example 1: Model Aggregation Method for Distributed Graph Learning

[0028] This invention provides a model aggregation method for distributed graph learning, with the overall process covering a complete closed loop from communication graph construction to similarity-based weighted aggregation. For example... Figure 1 The system interaction process sequence diagram shown illustrates that this method is mainly completed collaboratively by the central server and multiple clients (such as client u and client v). The specific interaction and execution steps are as follows:

[0029] S1, Central Server Communication Diagram Construction

[0030] like Figure 1As shown, the central server obtains the set of clients participating in distributed learning. ,in, This represents the total number of clients. The central server constructs the connectivity parameter based on this set. The number of nodes is Harary communication diagram : ,in, Represents a set of nodes. Denotes the set of edges. This represents the connectivity parameter.

[0031] The central server will aggregate the clients. Mapped to Harary diagram Node set Based on the adjacency relationship in the communication graph, the central server serves each client (such as client...). Client Determine the set of communication neighbors that can interact with each other, and send the neighbor set down to each neighbor. and Establish structured, constrained communication topology connections, whereby... or This indicates that each client has a unique sequence number. Indicates with the client A set of clients that establish communication relationships.

[0032] S2, Local model training on each client

[0033] like Figure 1 and Figure 3 As shown, each client loads subgraph data locally and initializes its local graph neural network model based on the model parameters synchronized in the previous round. Subsequently, each client independently performs local training, generating model update vectors (e.g., client-side). produce Client produce ).

[0034] Client For example, client Load local subgraph dataset ,in, Indicates client Local subgraph dataset, Indicates the client The local graph structure, Indicates the client Corresponding tag information; client Initialize the local graph neural network model based on the global or synchronous model parameters from the previous round; Client Based on local subgraph dataset Perform forward and backward propagation to obtain the local model update for this round. .

[0035] S3. Model Similarity Calculation and Similar Neighbor Set Generation

[0036] 3.1 Sending a unified random graph: The central server generates and sends a unified random graph to all clients. The random graph is constructed using a probabilistic graphical model with a cluster structure. It is used to perform unified functional response detection on the local graph neural network models of each client. It has a consistent number of nodes, edge connections, and initial node features as consistent detection stimuli.

[0037] 3.2 Upload Embedded Vector: Each client will receive the embedded vector. In the input local graph neural network model, the output of the penultimate fully connected layer and / or the second fully connected layer from the input is extracted as the intermediate layer node embedding vector. In order to transform the node-level features into graph-level features, each client performs pooling aggregation on the intermediate layer node embedding vector (aggregation methods include, but are not limited to, average pooling or max pooling) to obtain the client representation vector used to characterize the client model response features. (such as client) Generate and return Client return Each client sends its client representation vector to the central server. This process corresponds to... Figure 2 The client transmits the "embedded vector" unidirectionally to the server via a dashed path.

[0038] 3.3 Calculate similarity: After receiving the client representation vector, the central server calculates the cosine similarity between the clients.

[0039] Specifically, the central server uses the client representation vector and client representation vector Computing Client With the client Cosine similarity between ,satisfy: In addition to cosine similarity, similarity can also be determined by at least one of Euclidean distance transformation similarity, Pearson correlation coefficient, kernel function similarity, or attention scoring function.

[0040] Subsequently, the central server adjusts the model response similarity by incorporating the non-independent identically distributed (Non-IID) quantization differences between clients. The adjustment formula is as follows: ,in, This parameter represents the quantification parameter for data distribution differences. It is determined by combining at least one of the following: local label distribution statistical differences, training loss differences, or node feature distribution differences. To protect privacy, these differences are not obtained by uploading the original data or original statistical details to the client, but are generated locally by the client after performing privacy-preserving processing based on data characteristics. The privacy-preserving processing includes at least one of the following: interval quantization, hierarchical quantization, truncation, normalization, bucket quantization, hash encoding, random projection, dimensionality reduction, secure aggregation, or differential privacy noise enhancement. For example: regarding the difference in local label distribution statistics, each client locally performs statistics on the number of label categories, the proportion of samples in each category, or the label distribution entropy, and then performs intervalization, gradation, or differential privacy noise processing on the statistical results to generate label distribution difference indication information; regarding the difference in training loss, each client locally performs statistics on the mean loss, variance loss, rate of decrease loss, or convergence state of loss within a preset training round, and then performs smoothing, truncation, quantization, or noise processing on the statistical results to generate training state difference indication information; regarding the difference in node feature distribution, each client locally performs statistics on the mean, variance, quantiles, norm distribution, or feature embedding statistics of node features, and then performs dimensionality reduction, bucketing, hash encoding, random projection, or differential privacy perturbation processing on the statistical results to generate node feature distribution difference indication information; the central server determines the Non-IID quantization parameters between clients based on at least one of the label distribution difference indication information, training state difference indication information, and node feature distribution difference indication information.

[0041] The central server uses scaling parameters Normalization yields similarity weights. This is used for subsequent pseudo-gradient weighted aggregation. ,in, This represents the similarity scaling parameter. Simultaneously, the central server performs anomaly identification based on analysis of variance, generating a set of similar neighbors for each client after removing anomalies. .

[0042] S4. Each client generates a double mask (mask gradient).

[0043] Client Generate random seed Generate private mask based on pseudo-random number generator and generate key pairs ,in, Indicates the client private key, Indicates the client The public key. Similarly, the client... generate , and key pair Client With Neighbor Client Key negotiation is performed based on each party's public and private keys to generate a symmetric shared key. and based on shared keys Generate pairwise masks ,in .

[0044] Client Update based on local model Private mask and paired masks Generate mask gradient :

[0045]

[0046] S5, each client generates a purification mask.

[0047] While generating the mask gradient, the client also uses a private mask. and paired masks Generate merge mask vector : ,in, Indicates the client The corresponding merge mask vector.

[0048] Client For merged mask vectors Perform sparsification to obtain a sparse mask vector. Update based on local model Calculate the unit direction vector : ,in, Indicates the client The corresponding model updates the unit direction vector. Indicates model update The norm of .

[0049] Client sparse mask vector Projected onto unit direction vector The projection mask vector is obtained from the above. : ,in, This represents the vector inner product. By projecting along the local model update direction, it can preferentially retain the mask components related to the main model update direction to support subsequent pseudo-gradient recovery.

[0050] Finally, add random noise. Generate a purification mask Purification mask For merged mask vectors The mask vector obtained after sparsification, directional projection, and random perturbation satisfies:

[0051]

[0052] in, Indicates the client The corresponding random noise vector is used to further suppress the possibility of deriving the complete mask vector or updating the original model from the purified mask; Client mask gradient and purification mask Send to The neighbor client in the game.

[0053] S6. Each client communicates and transmits the mask gradient and purification mask, and generates a pseudo gradient.

[0054] Client and client They send mask gradients to each other and purification mask This interaction process corresponds to Figure 2 A solid line connection is used between the clients to transmit "pseudo-gradient and purification mask". Each client verifies the mask gradient and purification mask, including data integrity verification, message source verification, and / or format validity verification, to ensure the reliability of the subsequent pseudo-gradient recovery process.

[0055] The client uses the mask gradient. and purification mask The corresponding pseudo gradient is calculated and recovered: ,in, Indicates the client The corresponding pseudo-gradient. This pseudo-gradient is an approximate model update used for aggregation. It retains the main direction information of the neighboring client's local model update, but through sparsification and noise addition mechanisms, the receiver cannot accurately recover the true local model update plaintext of the neighboring client, and it is not equivalent to the original model update plaintext of the neighboring client. Each client assigns the pseudo-gradient corresponding to each neighboring client. As the input for subsequent aggregation, since each client recovers the pseudo gradient after purification masking rather than the original local model update, it can reduce the risk of neighboring clients directly obtaining sensitive update information while supporting similarity aggregation.

[0056] S7. Aggregate clients based on similarity weighting.

[0057] The central server will use the similarity weights calculated in the previous stage. For each client, a set of similar neighbors is selected and constructed from its set of communicating neighbors. This set, along with its corresponding similarity weights, is then distributed to the respective clients. Each client, locally, performs a weighted summation of the locally recovered pseudo-gradients based on the received set of similar neighbors and their weights to obtain the neighbor aggregation update. Based on this update, the parameters of the local graph neural network model are then updated. This completes a single round of distributed secure training.

[0058] Specifically, the central server uses similarity weights. A client similarity set is constructed, grouped, and inter-group significance is calculated using analysis of variance (ANOVA). Abnormal clients are then identified based on the significance test results and average similarity. Clients with an average similarity below a preset threshold are considered abnormal. By identifying and excluding abnormal clients, the interference of abnormal or low-similarity updates on the aggregation results can be reduced. The central server then selects clients from the non-abnormal clients... Before choosing Find the number of similar neighbors to obtain the set of similar neighbors. ,in, This represents the threshold number of neighbors participating in each round of aggregation, and That is, the set of similar neighbors is further filtered within the communication reach range; the central server will then process the set of similar neighbors. and similarity weight set Send to the client Client Based on similar neighbor sets Based on the set of similar neighbors returned by the server The pseudo gradients corresponding to each neighboring client and similarity weight Perform a weighted summation to obtain the aggregated update result for client u.

[0059] In practical applications, random graphs Generation method, connectivity parameters Similarity scaling parameters Threshold for the number of neighbors participating in aggregation in each round The threshold for identifying abnormal clients and the intensity of random noise can be set or adjusted according to the number of clients, the degree of data heterogeneity, communication resource limitations, and privacy protection requirements.

[0060] Example 2: Model Aggregation System for Distributed Graph Learning

[0061] like Figure 2 As shown, this embodiment provides a model aggregation system for distributed graph learning. This system consists of a central server and various client-side components (such as...). Figure 2 The model aggregation device of clients u, v, a, b) together constitutes:

[0062] 1. The model aggregation device deployed on the central server side includes the following functional modules:

[0063] The communication graph construction module is used to obtain the set of clients participating in training, construct the Harary communication graph, determine the communication topology, and send the "communication graph" information (i.e., the set of communication neighbors) to the corresponding clients.

[0064] Random graph generation module: Used to generate a unified random graph with the same graph structure and initial feature configuration, and then distribute it as a probe to each client.

[0065] Similarity calculation module: It is used to receive the client representation vector (embedded vector in 2 in the figure) returned by each client through the dashed path, and calculate the similarity weight by combining the non-independent and identically distributed (Non-IID) quantization difference parameter between the clients.

[0066] Anomaly identification module: Used to identify abnormal nodes by combining variance analysis, further filter out similar neighbor sets for aggregation from the communicating neighbors, and feed back the results and similarity weights to the client through thick solid arrows.

[0067] 2. The model aggregation devices deployed on the client side all include the following equivalent functional modules:

[0068] Local training module: Used to load local data, perform forward and backward propagation of the graph neural network, and generate local model update vectors.

[0069] Embedding extraction module: Receives a unified random graph from the server, inputs it into the local model, extracts the embedding vectors of intermediate layer nodes output from the last convolutional layer, aggregates them, generates and sends back client representation vectors to the server (e.g., ...). Figure 2 (As indicated by the dashed arrow).

[0070] Dual Mask Generation Module: Used to complete key negotiation, generate private masks and paired masks, and process local model updates to output mask gradients.

[0071] Purification Mask Generation Module: This module performs sparsification, main direction projection, and random noise addition operations on the merged double mask vector to output a purified mask.

[0072] Pseudo-gradient calculation module: used to calculate gradients by... Figure 2 The solid-line interaction path between clients exchanges mask gradients and purification masks with their communicating neighbors, and performs subtraction and verification calculations to recover pseudo-gradients that can be used for aggregation.

[0073] The aggregation update module receives the set of similar neighbors and similarity weights from the central server, performs a weighted summation based on the results output from the pseudo gradient calculation module, and updates the local model parameters accordingly.

[0074] Through the above system architecture and module division, this invention achieves effective decoupling of control flow (between server and client) and data flow (between clients). The server only grasps the features of the random graph and abstract embedding vectors, and neighbors only exchange pseudo-gradients and masks with mathematical confusion, providing dual security guarantees from both the physical and logical levels of the system.

[0075] Example 3: To verify the effectiveness of the proposed method, six public graph datasets were selected for testing: Cora Citation Network, CiteSeer Citation Network, PubMed Citation Network, Amazon-Computers, Amazon-Photo, and OGBN-Arxiv Citation Network. The experiment simulated a distributed graph learning environment with 100 clients and evaluated the performance of the proposed method in typical scenarios such as Non-IID data distribution and malicious client attacks.

[0076] 1. Analysis of the effectiveness of resisting malicious client attacks

[0077] Referring to Figure 4, this invention evaluated the robustness of the system under attacks from malicious clients with different proportions. The horizontal axis in the figure represents the percentage of malicious clients (increasing from 10% to 50%), and the vertical axis represents the model's test accuracy. The comparison scheme is the existing Federated Personalized Subgraph (FED-PUB) method; the method of this invention is shown as a cyan bar chart.

[0078] Analysis Results: As shown in the six subgraphs a) to f) of Figure 4, the test accuracy of all methods decreases with the increase of the proportion of malicious clients. However, thanks to the "dual-graph mechanism" (Harary physical constraint + similarity logic optimization) and the abnormal client removal mechanism based on ANOVA, the present invention significantly outperforms the FED-PUB method on all six datasets. Especially in extreme cases where the proportion of malicious clients is as high as 25% or even 50% (as indicated by the dashed highlighted lines), the test accuracy of the present invention decreases much less than the comparative scheme, and it can still maintain a relatively high model availability, fully demonstrating that the present invention has extremely strong system robustness in resisting data poisoning and malicious updates.

[0079] 2. Model convergence speed and accuracy analysis

[0080] Referring to Figure 5, this invention compares the convergence curves of multi-round communication training in a Non-IID environment. The horizontal axis represents the number of communication rounds (0 to 100 rounds), and the vertical axis represents accuracy. The baseline methods compared include the classic Federated Avg, Federated Prox, Federated Adaptive+, Federated Personalized Subgraph (FED-PUB), Federated Sampling Aggregation Enhancement (FedSage+), and the GCFL method.

[0081] Analysis Results: As shown in the six subplots a) to f) of Figure 5, the method of this invention (the red dashed line with red dots) exhibits an extremely steep upward trend in the early stages of training (approximately 0-20 epochs), indicating that its convergence speed is significantly faster than all the comparison baselines. After 100 epochs of communication, this invention achieved the highest global test accuracy on six datasets: Cora, CiteSeer, PubMed, Amazon-Computers, Amazon-Photo, and OGBN-arXiv.

[0082] This advantage is primarily attributed to the "unified random graph probe" and "pseudo-gradient local recoverability" mechanisms of this invention. The server can accurately calculate similarity weights in Non-IID environments, enabling clients to prioritize personalized aggregation with neighbors that have similar features. This avoids update direction conflicts caused by traditional global average aggregation, thereby significantly accelerating convergence and improving the model's generalization and prediction accuracy.

[0083] Example 4: Application Scenario of Distributed Graph Learning for Fault Prediction of Industrial IoT Devices

[0084] This embodiment provides a specific application scenario for applying the above-mentioned distributed graph learning method to the fault prediction of Industrial Internet of Things (IIoT) devices.

[0085] In modern intelligent manufacturing, multiple independent entities (such as factories and production workshops in different regions) face an urgent need to eliminate "data silos" in order to jointly train equipment failure prediction models. However, due to the restrictions of trade secrets (such as production cycle time, equipment process parameters, yield rate, etc.), it is impossible to centrally upload the underlying production data to the cloud.

[0086] In this scenario, the central server is a cloud-based central scheduling and diagnostic platform; the client is an edge computing gateway deployed in various factories or production workshops.

[0087] Physical mapping of graph data: Each edge gateway locally aggregates equipment operation data from its corresponding production line in real time. Each machine or core component serves as a graph node (node ​​characteristics include real-time temperature, high-frequency vibration characteristics, current, voltage, operating time, and alarm records collected by sensors); physical connections between devices, the sequence of material flow, or the correlation of control signals serve as graph edges. Thus, each edge gateway locally constructs local industrial equipment graph data representing the unique production process of the factory.

[0088] When applying the method of this invention, the specific execution logic is as follows:

[0089] (1) Structured communication restrictions (corresponding to step S1): The central scheduling platform does not adopt full network interconnection, but constructs a Harary communication graph based on the number of edge gateways participating in the collaboration, restricting the communication of each factory gateway to a limited number of topological neighbors. This prevents the global model from collapsing after a single factory network is controlled by a network attack, thus improving the security of the industrial system from the network physical level.

[0090] (2) Local Graph Neural Network Training (corresponding to step S2): Before each round of distributed collaboration, each edge gateway initializes its local graph neural network model using the model parameters from the previous round. Subsequently, each gateway loads its real-time collected industrial equipment graph data entirely within its local closed loop, using historical fault labels and alarm records as supervisory signals, and performs forward and backward propagation calculations of the model to minimize the loss function for equipment fault prediction, thereby generating a local model update vector that characterizes the current health dynamics of the equipment in the plant area. Throughout the entire process, plaintext data from equipment sensors does not leave the local gateway.

[0091] (3) Feature probe and working condition similarity matching (corresponding to step S3): Due to the different parts processed by different factories and the different levels of equipment age, their graph data exhibits strong Non-IID characteristics. The central scheduling platform sends a "unified random graph" (equivalent to the input stimulus of simulating a baseline virtual production line) to each edge gateway. Each edge gateway inputs the probe into the newly trained local model and returns the response representation vector. At the same time, the edge gateway uploads the Non-IID quantization parameters by differentially adding privacy noise to the local equipment fault category distribution. In this way, the central scheduling platform can accurately match a set of similar factories with "similar processing technology" or "similar equipment wear stage" without touching the real data.

[0092] (4) Protection of process secrets and recovery of pseudo-gradients (corresponding to steps S4-S6): When "Factory A" needs to transmit model updates to a similar "Factory B", in order to prevent "Factory B" from deducing "Factory A's" exclusive process parameters, Gateway A combines its local model update, private mask, and paired mask to generate a mask gradient. In particular, Gateway A performs sparsification on its merged mask, projects it along the main direction of local fault prediction update, and adds random noise to generate a "purified mask". After receiving it, Gateway B performs the calculation and can only recover the pseudo-gradient. This pseudo-gradient retains the main direction law of "how to detect equipment anomalies", but completely destroys the microscopic features that may leak the original sensor values, thus achieving privacy protection at the level of industrial secrets.

[0093] (5) Anti-interference personalized collaboration (corresponding to step S7): The central scheduling platform uses variance analysis to identify abnormal edge gateways that have experienced large-scale sensor failures or malicious data poisoning, and removes them. Each normal edge gateway performs local weighted summation based only on the pseudo gradients of its preferred neighbors with similar operating conditions selected by the platform, to complete the aggregation and update of the fault prediction model parameters for this round.

[0094] Through the systematic collaboration of this embodiment, the present invention successfully deploys deep graph neural networks and cryptographic de-identification technology in industrial IoT physical entities. While ensuring compliance by ensuring that core production data and equipment operating conditions in each factory "absolutely do not leave the local area," it not only significantly improves the accuracy, convergence speed, and generalization ability of fault prediction models for local devices on each edge gateway, but also effectively reduces communication overhead across factory wide area networks, demonstrating strong industrial application value.

[0095] Example 5: Application Scenarios of Distributed Graph Learning for Intelligent Traffic Flow Prediction and Anomaly Event Recognition

[0096] To further demonstrate the universality and application value of the present invention in different physical systems, this embodiment provides a practical scenario for applying the above-mentioned distributed graph learning method to intelligent traffic flow prediction and abnormal event identification.

[0097] In modern city-level intelligent transportation systems, traffic management platforms in different administrative regions or under different road jurisdictions possess massive amounts of sensor data, creating an urgent need for joint training of global traffic prediction models. However, due to the privacy of citizens' travel trajectories, the protection of sensitive geographical locations, and limitations in cross-regional communication bandwidth, directly aggregating raw traffic monitoring videos, radar point clouds, or vehicle-to-everything (V2X) trajectory data not only poses significant security risks but also faces extremely high computational and communication bottlenecks.

[0098] In this scenario, the central server is a city-level traffic cloud platform (such as "Traffic Brain"); the clients are roadside edge computing servers (MEC) or traffic sensing nodes deployed in traffic management centers in various regions.

[0099] Physical mapping of graph data: Each edge server locally aggregates traffic flow data within its jurisdiction in real time. "Road intersections" or "key road segments" in the physical road network are used as graph nodes (node ​​features include real-time traffic flow, average vehicle speed, lane occupancy, etc., collected by cameras, geomagnetic coils, and radar); physical connectivity between roads, traffic flow transfer probabilities, or spatial geographic proximity relationships are used as graph edges. Thus, spatiotemporal traffic map data representing the dynamic evolution of the road network in the region is constructed locally on each edge server.

[0100] When applying the method of this invention in the complex physical space of urban transportation networks, the specific execution logic is as follows:

[0101] (1) Structured communication restrictions to prevent spread (corresponding to step S1): Since roadside facilities are highly vulnerable to network attacks, the cloud platform constructs a Harary communication graph based on the network status and generates a whitelist of communication neighbors between regions. Even if a roadside node is compromised and a false congestion event is injected, its malicious model update will only be limited to a very small topological neighborhood, preventing the spread of "congestion false alarms" and paralyzing the entire city's traffic control system.

[0102] (2) Local training of regional traffic map model (corresponding to step S2): Each roadside edge server initializes its local traffic prediction graph neural network using the model parameters from the previous round of collaboration. Combined with real-time traffic map data collected within its jurisdiction, and using future real-time traffic flow or abnormal traffic accident labels as the supervision target, it independently performs iterative training of the graph neural network within its local MEC, extracting and generating a local model update vector representing the current road network evolution pattern of the region. This process ensures that vehicle trajectories and monitoring images remain safely within each region.

[0103] (3) Traffic flow feature probe and similarity selection (corresponding to step S3): Due to the huge differences in tidal characteristics between different areas (old urban area and new industrial area) (exhibiting highly Non-IID attributes), the cloud platform issues a "unified random map" (simulating a standard traffic flow impact). After the local model of each area inputs the probe, it returns the response representation; at the same time, differential privacy noise is added to the frequency of local abnormal events to generate Non-IID quantization parameters. The cloud platform uses this to accurately calculate the set of areas with "similar traffic flow features" (such as associating multiple new urban residential areas with the same tidal characteristics), breaking the limitation of dividing neighbors solely by geographical location.

[0104] (4) Trajectory privacy protection and lightweight pseudo-gradient (corresponding to steps S4-S6): To prevent communication neighbors from intercepting the sensitive trajectories of specific vehicles, the sending edge server performs sparsification processing after generating a merged mask by combining it with the local model update, and projects and adds noise along the main update direction of congestion feature extraction to generate a "purified mask". The receiving edge server can only recover the pseudo-gradient based on this. This pseudo-gradient contains "macroscopic laws" that improve the accuracy of traffic prediction, but makes the inverse operation of restoring the microscopic traffic flow of a specific road infeasible.

[0105] (5) Interference-resistant regional personalized collaboration (corresponding to step S7): The cloud platform uses variance analysis to eliminate abnormal edge servers that have experienced large-scale sensor failures due to extreme weather. Each normal roadside node performs weighted aggregation locally only based on the pseudo-gradients of its neighbors with similar traffic flow patterns. Thus, this invention significantly improves the model accuracy for road congestion early warning and dynamic traffic light scheduling in each jurisdiction while ensuring traffic data privacy and physical network security.

[0106] Through the above systematic design, this invention, while ensuring the privacy of regional traffic data and the physical security of the road network, enables each traffic sensing node to extract the characteristic patterns of congestion evolution and abnormal events from the "same type" road network throughout the city, which greatly improves the accuracy and model response speed of each jurisdiction in terms of future road congestion, traffic accident early warning, and dynamic scheduling of traffic lights.

[0107] As described above, although the invention has been shown and described with reference to specific preferred embodiments, it should not be construed as limiting the invention itself. Various changes in form and detail may be made without departing from the spirit and scope of the invention as defined in the appended claims.

Claims

1. A model aggregation method for distributed graph learning, characterized in that, Includes the following steps: S1. The central server constructs a Harary communication graph based on the set of clients participating in distributed graph learning, determines the communication neighbor set of each client according to the adjacency relationship of the Harary communication graph, and sends it to the corresponding client. S2. Each client trains the local graph neural network model locally based on the local graph data to obtain the local model update; S3. The central server sends a unified random graph with the same graph structure and initial feature configuration to each client; calculates the model response similarity between clients based on the client representation vectors extracted after each client inputs the unified random graph; corrects the model response similarity by combining the non-independent and identically distributed (Non-IID) quantization difference parameters between clients to obtain a corrected similarity; and sends a set of similar neighbors to each client after filtering from their communication neighbor set based on the similarity weight. S4. Each client generates a private mask and a pair of masks, and updates the mask gradient by combining the local model. S5. Each client generates a merged mask vector based on the private mask and the paired mask, and performs sparsification, projection along the local model update direction, and random noise addition on the merged mask vector in sequence to generate a purified mask. The mask gradient and the purified mask are then sent to neighboring clients in their communication neighbor set. S6. Each client receives the mask gradient and purification mask sent by the neighboring client, and calculates and recovers the pseudo gradient that retains the main direction of the neighboring client model update through the mask gradient and the purification mask. S7. Each client performs a weighted summation of the pseudo gradients of each neighbor client in the similar neighbor set based on the received similar neighbor set and the corresponding similarity weights, and obtains the aggregated update result to update the parameters of the local graph neural network model.

2. The model aggregation method for distributed graph learning according to claim 1, characterized in that, The process by which each client extracts the client representation vector includes: Each client inputs the received unified random graph into a local graph neural network model and extracts the representative layer output as the intermediate layer node embedding vector. The unified random graph has a consistent number of nodes, edge connection relationships, and initial node features. The representative layer output includes the penultimate fully connected layer output and / or the second fully connected layer output from the input end of the local graph neural network model. The intermediate layer node embedding vectors are pooled and aggregated to obtain the client representation vectors used to characterize the client model response features, and then sent to the central server; wherein, the pooling and aggregation process includes average pooling or max pooling.

3. The model aggregation method for distributed graph learning according to claim 1, characterized in that, The process by which the central server calculates similarity weights and filters a set of similar neighbors for each client includes: Assume any client is a client Its corresponding client representation vector is Any neighbor client in the communication neighbor set is a client. The corresponding client representation vector is ; The central server computing client With the client Direct similarity between representation vectors The direct similarity is determined using at least one of cosine similarity, Euclidean distance transformed similarity, Pearson correlation coefficient, kernel function similarity, or attention scoring function. Obtain the data distribution difference quantification parameters generated by each client locally after performing privacy protection processing based on data characteristics. Using the formula The direct similarity is corrected to obtain a corrected similarity, and then scaled based on the similarity scaling parameter. Normalization is performed to obtain the similarity weights, which are then used for subsequent pseudo-gradient weighted aggregation. The central server calculates the significance between groups based on the analysis of variance test, identifies and removes abnormal clients, and selects clients from the non-abnormal clients. The set of similar neighbors is obtained by filtering neighbors within a set threshold number. .

4. The model aggregation method for distributed graph learning according to claim 3, characterized in that, The direct similarity is calculated using cosine similarity. ,satisfy: ; The data distribution difference quantification parameter The privacy protection process is determined by at least one of the following: the difference in local label distribution statistics, the difference in training loss, or the difference in node feature distribution. The privacy protection process includes at least one of the following: intervalization, hierarchicalization, truncation, normalization, bucket quantization, hash encoding, random projection, dimensionality reduction, secure aggregation, or differential privacy noise reduction.

5. The model aggregation method for distributed graph learning according to claim 1, characterized in that, The process of generating mask gradients for each client includes: Assume any client is a client Its corresponding local model is updated to ; Client Generating a private mask based on a pseudo-random number generator and key pairs; Client Neighbor clients in its communication neighbor set Key negotiation is performed based on each key pair to generate a symmetric shared key, and a pairwise mask is generated based on the symmetric shared key. ; Client Based on the following formula: ， Generate the mask gradient .

6. The model aggregation method for distributed graph learning according to claim 1, characterized in that, The process of generating a purification mask for each client includes: Assume any client is a client Client Generate the merge mask vector according to the following formula: ， in, For private mask, For paired masks; Client For the merged mask vector Perform sparsification to obtain a sparse mask vector. , Update according to the local model Calculate the unit direction vector The sparse mask vector Projected onto the unit direction vector The projection mask vector is obtained from the above. ; Add a random noise vector to the projection mask vector Generate the purification mask: 。 7. The model aggregation method for distributed graph learning according to claim 1, characterized in that, The process of calculating and recovering the pseudo gradient of the main direction of the neighbor client model update includes: After receiving the mask gradient and purification mask sent by the neighboring client, each client performs at least one of the following validity checks: data integrity check, message source check, and format validity check. After the verification is passed, the difference between the mask gradient and the purification mask is calculated to obtain the pseudo gradient, which is used as an approximate model update amount for the subsequent weighted aggregation process.

8. A model aggregation system for distributed graph learning, characterized in that, It includes a central server and several clients with communication connections, among which: The central server is used to: obtain a client set and construct a Harary communication graph; determine a communication neighbor set for each client based on edge relationships; send a unified random graph downwards; calculate similarity weights based on the client representation vectors returned by each client; and construct and send a corresponding similar neighbor set for each client. The client is used to: train a local graph neural network model based on local graph data to obtain a local model update; extract embedding vectors from the unified random graph input model to generate the client representation vector; generate mask gradients by processing the local model update based on private masks and paired masks; generate a purified mask by sparsifying, projecting the model update direction, and adding noise to the merged mask vector; exchange mask gradients and purified masks with neighbors in the communication neighbor set to recover pseudo gradients; and perform weighted summation of the pseudo gradients of the corresponding neighbors according to the similar neighbor set and similarity weights to update the local graph neural network model.

9. A model aggregation device for distributed graph learning, deployed on the client side, characterized in that, include: The local training module is used to train a local graph neural network model on a local dataset based on a subgraph dataset, and obtain local model updates. The embedding extraction module is used to receive a unified random graph sent by the central server, input it into the local graph neural network model, extract the embedding vectors of intermediate layer nodes and perform aggregation processing to generate client representation vectors and send them to the central server. The dual-mask generation module is used to complete key negotiation, generate a private mask and a pair of masks, and update the generated mask gradient in combination with the local model. The purification mask generation module is used to generate a merged mask vector based on the private mask and the paired mask, and to perform sparsification, projection along the local model update direction and random noise addition on the merged mask vector in sequence to generate a purification mask, and to send the mask gradient and the purification mask to the neighbor clients in the communication neighbor set. The pseudo gradient calculation module is used to receive the mask gradient and purification mask sent by the neighbor client, and perform subtraction and verification calculations on the mask gradient and the purification mask to recover the pseudo gradient that retains the main direction of the neighbor client model update. The aggregation update module is used to receive the set of similar neighbors and the corresponding similarity weights sent by the central server, perform weighted summation of the pseudo gradients corresponding to each neighbor client in the set of similar neighbors, and update the parameters of the local graph neural network model accordingly.

10. A model aggregation device for distributed graph learning, deployed on a central server side, characterized in that, include: The communication graph construction module is used to obtain the set of clients participating in the training, construct the Harary communication graph, determine the communication neighbor set of each client based on the adjacency relationship of the Harary communication graph, and send it to the corresponding client. The random graph generation module is used to generate a unified random graph with the same structure and initial feature configuration, which is then sent to each client as a probe. The similarity calculation module is used to receive the client representation vector extracted after each client inputs the unified random graph, and calculate the similarity weight by combining the non-independent and identically distributed (Non-IID) quantization difference parameter between the clients. The anomaly identification module is used to identify abnormal clients by combining variance analysis, and to filter similar neighbor sets from the communication neighbor set for each non-abnormal client based on the similarity weight, and to send the similar neighbor sets and corresponding similarity weights to the corresponding clients.

Images