Model training method and device, storage medium and program product

Abstract

The invention relates to the technical field of artificial intelligence, and discloses a model training method, and clients generate different aggregation strategies for each client on the premise of not sharing privacy information by uploading key model parameters and graph-level samples. A representative sample with privacy protection property is constructed at a client side through an adversarial learning mode, and the representative sample is uploaded to a server for subsequent personalized model aggregation. According to the personalized global aggregation strategy, graph-level representation is extracted from uploaded information on a server side, and graph similarity between clients is estimated, so that a personalized aggregation strategy is formulated through collaborative integration of multiple clients, the clustering performance of a local model is enhanced, the relationship between the clients can be accurately captured, and the clustering effect is improved. When personalized recommendation is carried out on a trained clustering model, the accuracy of personalized recommendation is improved. The invention further discloses a model training device, a storage medium and a program product.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, such as a training method and apparatus for a model, a storage medium, and a program product. Background Technology

[0002] Federated Graph Learning (FGL) is a technique that combines federated learning with graph learning. Specifically, it's a distributed machine learning paradigm that allows multiple clients (such as different institutions or devices) to collaboratively train graph neural network models without sharing the original graph data. This allows them to learn a shared graph representation, addressing graph data privacy concerns and achieving a balance between data privacy and model performance improvement. Graph data is a data structure composed of nodes (vertices) and edges (lines connecting nodes). Nodes represent entities, and edges represent relationships between entities. For example, in a social network, users can be considered nodes, while friendships between users are edges. Graph data can naturally represent various complex relationships and structural information.

[0003] In related technologies, federated graph learning can be applied to personalized recommendation scenarios. For example, users and items (such as movies, books, and goods) are used as nodes, and user ratings, browsing, and purchasing behaviors are used as edges to form graph data. By clustering this graph using a neural network model, user groups with similar interests and item groups with similar characteristics can be identified. Based on these clustering results, items favored by groups with similar interests can be recommended to users.

[0004] However, in related technologies, the heterogeneity of graph data caused by structural differences among multiple clients limits the quality of multi-source information negotiation, thus affecting the clustering effect of neural network models on graph data, and consequently affecting the accuracy of personalized recommendations. Summary of the Invention

[0005] To provide a basic understanding of some aspects of the disclosed embodiments, a brief summary is given below. This summary is not intended as a general commentary, nor is it intended to identify key / important components or describe the scope of protection of these embodiments, but rather as a prelude to the detailed description that follows.

[0006] This application provides a model training method, apparatus, storage medium, and program product. During model training, it can more accurately capture the potential correlations between clients, thereby improving the clustering effect of the trained model on graph data, and thus improving the accuracy of personalized recommendations when the trained model performs personalized recommendations.

[0007] In a first aspect, embodiments of this application provide a method for training a model, applied to a server, the server deploying a global clustering model, the server establishing connections with multiple clients, and each client deploying a local clustering model, the method comprising:

[0008] Obtain the graph-level representative samples and model parameters uploaded by each client; the graph-level representative samples are reconstructed by the client based on the graph training data; the model parameters are the parameters of the client's local clustering model;

[0009] The graph-level representative samples uploaded by each client are learned through a global clustering model to obtain graph-level embedding representations;

[0010] Based on graph-level embedding representation, the potential similarity between graph-level representative samples uploaded by each client is evaluated;

[0011] Update the model parameters uploaded by each client based on the potential similarity between the graph-level representative samples uploaded by each client;

[0012] The updated model parameters are distributed to each client respectively, so that each client can train the local clustering model based on the updated model parameters and graph training data;

[0013] The global clustering model is considered complete when its parameters converge; the parameters of the global clustering model are generated based on the model parameters uploaded by each client.

[0014] Secondly, embodiments of this application provide a model training method applied to a client, wherein the client deploys a local clustering model, the client establishes a connection with a server, and the server deploys a global clustering model; the method includes:

[0015] The local clustering model is trained based on the graph training data to obtain the model parameters of the local clustering model;

[0016] Graph-level representative samples are reconstructed from the graph training data;

[0017] The model parameters of the local clustering model and the graph-level representative samples are uploaded to the server so that the server can learn the graph-level representative samples through the global clustering model to obtain graph-level embedding representations. Based on the graph-level embedding representations, the potential similarity between the graph-level representative samples uploaded by each client is evaluated. Based on the potential similarity between the graph-level representative samples uploaded by each client, the model parameters uploaded by each client are updated and the updated model parameters of each client are distributed to each client respectively.

[0018] The local clustering model is trained based on the updated model parameters and graph training data;

[0019] Training is considered complete when the model parameters of the local clustering model converge.

[0020] Optionally, the local clustering model is trained based on the updated model parameters and graph training data, including: filtering the updated model parameters through a local masking mechanism to obtain local model parameters; and training the local clustering model based on the local model parameters and graph training data.

[0021] Thirdly, embodiments of this application provide a model training device integrated into a server. The server deploys a global clustering model, establishes connections with multiple clients, and each client deploys a local clustering model. The device includes:

[0022] The acquisition module is used to acquire the graph-level representative samples and model parameters uploaded by each client; the graph-level representative samples are reconstructed by the client based on the graph training data; the model parameters are the parameters of the client's local clustering model.

[0023] The learning module is used to learn the graph-level representative samples uploaded by each client through a global clustering model in order to obtain graph-level embedding representations;

[0024] The evaluation module is used to evaluate the potential similarity between the graph-level representative samples uploaded by each client based on the graph-level embedding representation;

[0025] The update module is used to update the model parameters uploaded by each client based on the potential similarity between the graph-level representative samples uploaded by each client.

[0026] The distribution module is used to distribute the updated model parameters from each client to each client, so that each client can train the local clustering model based on the updated model parameters and graph training data;

[0027] The determination module is used to determine that the global clustering model training is complete when the parameters of the global clustering model converge; the parameters of the global clustering model are generated based on the model parameters uploaded by each client.

[0028] Fourthly, embodiments of this application provide a model training device integrated into a client. The client deploys a local clustering model, establishes a connection with a server, and the server deploys a global clustering model. The device includes:

[0029] The training module is used to train the local clustering model based on the graph training data to obtain the model parameters of the local clustering model.

[0030] The reconstruction module is used to reconstruct graph-level representative samples based on graph training data.

[0031] The upload module is used to upload the model parameters of the local clustering model and the graph-level representative samples to the server, so that the server can learn the graph-level representative samples through the global clustering model to obtain the graph-level embedding representation. Based on the graph-level embedding representation, the potential similarity between the graph-level representative samples uploaded by each client is evaluated, and the model parameters uploaded by each client are updated according to the potential similarity between the graph-level representative samples uploaded by each client. The updated model parameters of each client are then distributed to each client.

[0032] The training module is also used to train the local clustering model based on the updated model parameters and graph training data;

[0033] The confirmation module is used to determine that training is complete when the model parameters of the local clustering model converge.

[0034] Optionally, when the training module trains the local clustering model based on the updated model parameters and graph training data, it specifically performs the following: filtering the updated model parameters through a local masking mechanism to obtain local model parameters; and training the local clustering model based on the local model parameters and graph training data.

[0035] Fifthly, embodiments of this application provide a server, which is deployed with a global clustering model, establishes connections with multiple clients, and each client deploys a local clustering model; the server includes a processor and a memory storing program instructions, the processor being configured to execute the training method of the model in the first aspect when running the program instructions.

[0036] In a sixth aspect, embodiments of this application provide a client that has a local clustering model deployed on it. The client establishes a connection with a server, and the server has a global clustering model deployed on it. The client includes a processor and a memory storing program instructions. The processor is configured to execute the training method of the model in the second aspect when running the program instructions.

[0037] In a seventh aspect, embodiments of this application provide a storage medium storing program instructions, which, when executed, perform the training method of the model in the first or second aspect.

[0038] Eighthly, embodiments of this application provide a computer program product, including a computer program, which, when executed by a processor, implements a training method for a model of the first or second aspect.

[0039] The training method, apparatus, storage medium, and program product for a model provided in this application embodiment can achieve the following technical effects:

[0040] During the training of the clustering model, the correlations between clients are fully explored, effectively preserving cross-client information beneficial to clustering. Specifically, on the client side, representative samples are first selected and reconstructed after removing privacy information. For these privacy-preserving samples, a min-max optimization strategy is used to reconstruct the graph structure, while node attributes are reconstructed using a Gaussian noise-like technique. Subsequently, these reconstructed samples and model parameters are uploaded to the server. On the server side, the uploaded samples are learned to obtain graph-level embedding representations for estimating the potential affinity between clients. Based on this, the unique advantages of each client are collaboratively integrated using these potential affinities, and different aggregation strategies are customized for the model parameters, which are then transmitted back to the client. In addition, an adaptive local masking mechanism is proposed, which selectively refines the parameters by evaluating the gradient direction, thereby enhancing the clustering performance of the local model. This leads to a personalized global aggregation strategy that accurately captures the relationships between clients, thereby improving the accuracy of personalized recommendations when the trained clustering model is used for personalized recommendations.

[0041] The above general description and the description below are exemplary and illustrative only and are not intended to limit this application. Attached Figure Description

[0042] One or more embodiments are illustrated by way of example with reference to the accompanying drawings. These illustrations and drawings do not constitute a limitation on the embodiments. Elements having the same reference numerals in the drawings are considered similar elements. The drawings do not constitute a limitation of scale, and wherein:

[0043] Figure 1 This is a network architecture diagram applicable to the training method of the model provided in the embodiments of this application;

[0044] Figure 2 This is a schematic diagram of the training method for the model provided in the embodiments of this application;

[0045] Figure 3 This is a performance comparison diagram of the clustering models provided in the embodiments of this application;

[0046] Figure 4 This is a schematic diagram of the results of an ablation experiment provided in an embodiment of this application;

[0047] Figure 5 This is a schematic diagram of a model training device provided in an embodiment of this application;

[0048] Figure 6 This is a schematic diagram of a training device for another model provided in an embodiment of this application. Detailed Implementation

[0049] The terms "first," "second," etc., used in the specification, claims, and drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of this application described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion.

[0050] Unless otherwise stated, the term "multiple" means two or more.

[0051] In this embodiment, the character " / " indicates that the objects before and after it are in an "or" relationship. For example, A / B means: A or B.

[0052] The term "and / or" describes an association between objects, indicating that three relationships can exist. For example, A and / or B means: A or B, or A and B.

[0053] The term "correspondence" can refer to an association or binding relationship. The correspondence between A and B means that there is an association or binding relationship between A and B.

[0054] Federated graph clustering guides client-side local models to group unlabeled graphs through a multi-source information sharing mechanism. However, the heterogeneity of graph-level data caused by structural differences among multiple clients limits the quality of multi-source information collaboration. To address this, Personalized Federated Learning (PFL) is employed for tasks such as computer vision and natural language processing. However, unlike Euclidean data (such as images and text), graph-structured data exhibits diverse and fragile local patterns, further increasing the difficulty of designing personalized aggregation strategies for each client. Specifically:

[0055] In related technologies, federated graph clustering methods can be broadly categorized into two types: 1) Federated node-level clustering: Each client possesses a graph, and node representations are collaboratively learned through federated learning to achieve clustering of unlabeled nodes. 2) Federated graph-level clustering: Each client possesses multiple non-independent and identically distributed (non-IID) graphs, and graph-level representations are collaboratively learned through federated learning to achieve clustering of unlabeled graphs. Since clients may possess a large amount of unlabeled graph data, the learning process between clients may lack clear guidance. To address this, related technologies utilize prior clustering knowledge to reveal potential correlations between clients and further employ personalized aggregation strategies to improve the performance of each local model in graph-level clustering tasks.

[0056] However, the inherent fragility and diversity of graph structures in related technologies lead to significant heterogeneity among clients. This structural heterogeneity makes it difficult to accurately mine correlations and increases the risk of heterogeneous information propagation.

[0057] In view of this, embodiments of this application provide a model training scheme in which clients can generate different aggregation strategies for each client without sharing privacy information by uploading key model parameters and graph-level samples. Specifically, the client first constructs representative samples with privacy protection through adversarial learning and uploads these representative samples to the server for subsequent personalized model aggregation. The server extracts graph-level representations from the uploaded representative samples and estimates the graph similarity between clients, thereby formulating personalized aggregation strategies through multi-client collaborative integration. In addition, the scheme also employs an adaptive local masking mechanism to preserve clustering-friendly model parameters. The model parameters are then optimized with the help of the personalized aggregation strategy to help the local model more clearly delineate cluster boundaries, thereby enhancing the clustering performance of the local model. This, in turn, enables a personalized global aggregation strategy that accurately captures relationships between clients, thereby improving the accuracy of personalized recommendations when the trained clustering model performs personalized recommendations.

[0058] To provide a more detailed understanding of the features and technical content of the embodiments of this application, the implementation of the embodiments of this application will be described in detail below with reference to the accompanying drawings. The accompanying drawings are for illustrative purposes only and are not intended to limit the embodiments of this application. In the following technical description, for ease of explanation, several details are used to provide a full understanding of the disclosed embodiments. However, one or more embodiments may still be implemented without these details. In other cases, well-known structures and devices may be simplified in their depiction to simplify the drawings.

[0059] Combination Figure 1 As shown in the embodiments of this application, a network architecture applicable to a model training method is provided. The framework consists of three core parts: a privacy-preserving sample transformation strategy for generating representative graph-level samples with privacy protection to support subsequent global aggregation; a personalized global aggregation scheme for mining correlations between clients and providing reliable feedback to each client; and an adaptive local masking mechanism for preserving key local model parameters to improve clustering performance. These three core parts correspond to three stages of the scheme: the privacy-preserving sample transformation stage, the personalized global aggregation stage, and the adaptive local masking generation stage.

[0060] The network architecture includes a server and multiple clients, with connections established between the server and the clients. The server deploys a global clustering model, while each client deploys a local clustering model.

[0061] Combination Figure 2 As shown, this application provides a method for training a model. This method applies the network architecture of the aforementioned embodiments. The method is described from the perspective of server-client interaction and specifically includes the following steps:

[0062] S21, the client trains the local clustering model based on the graph training data to obtain the model parameters of the local clustering model.

[0063] S22, the client reconstructs graph-level representative samples based on the graph training data.

[0064] S23, the client uploads the model parameters of the local clustering model and the graph-level representative samples to the server.

[0065] S24, the server learns the graph-level representative samples uploaded by each client through a global clustering model to obtain a graph-level embedding representation.

[0066] S25, the server evaluates the potential similarity between graph-level representative samples uploaded by each client based on graph-level embedding representation.

[0067] S26, the server updates the model parameters uploaded by each client based on the potential similarity between the graph-level representative samples uploaded by each client, and then sends the updated model parameters to each client respectively.

[0068] S27, the client trains the local clustering model based on the updated model parameters and graph training data.

[0069] S28, the client determines that training is complete when the model parameters of the local clustering model converge.

[0070] S29, the server determines that the global clustering model training is complete when the parameters of the global clustering model converge.

[0071] The model training method provided in this application can fully explore the correlations between clients during the clustering model training process and effectively retain cross-client information that is beneficial to clustering. Specifically, on the client side, representative samples are first selected and reconstructed after removing privacy information. For these privacy-preserving samples, a min-max optimization strategy is used to reconstruct the graph structure, while node attributes are reconstructed using a Gaussian noise-like technique. Subsequently, these reconstructed samples and model parameters are uploaded to the server. On the server side, the uploaded samples are learned to obtain graph-level embedding representations for estimating the potential affinity between clients. Based on this, the unique advantages of each client are collaboratively integrated using these potential affinities, and different aggregation strategies are customized for the model parameters, which are then transmitted back to the client. In addition, an adaptive local masking mechanism is proposed. This mechanism selectively refines the parameters by evaluating the gradient direction, thereby enhancing the clustering performance of the local model. This leads to a personalized global aggregation strategy that accurately captures the relationships between clients, thereby improving the accuracy of personalized recommendations when the trained clustering model is used for personalized recommendations.

[0072] The training method of the model in the embodiments of this application will be described in detail below, specifically according to the privacy-preserving sample transformation stage, the personalized global aggregation stage, and the adaptive local mask generation stage in the above embodiments.

[0073] Before describing the model training method provided in the embodiments of this application, the following explanation is given:

[0074] Given a set of undirected graphs for each client, the set of undirected graphs consists of I undirected graphs, denoted as Ii, ... The i-th graph is represented as and These represent the corresponding set of nodes and set of edges, respectively. A node attribute matrix containing structure encoding (such as degree encoding and random walk encoding) is represented as follows: The original adjacency matrix is ​​represented as Where N represents the number of nodes in the graph and d represents the dimension of the node attributes.

[0075] To effectively extract node embeddings, GIN (Graph Isomorphism Network) and its information passing mechanism are used to aggregate information from neighboring nodes. Specifically, in the l-th layer of GIN, the embedding of node v is represented as:

[0076]

[0077] in, Let ω represent the set of neighboring nodes of node v. (l)These are the learnable parameters of the l-th layer, and MLP stands for Multilayer Perceptron. It's important to note the initial embedding of node v. Set as

[0078] On the client side, in order to learn the graph-level embedding representation, a graph containing L is first constructed. l GIN encoder of layer Used to extract node-level representations. For each graph, its node embedding representation is as follows: The READOUT(·) function is a graph embedding aggregation function, which can employ methods such as average pooling or summation pooling. The core objective of this process is to aggregate the embedding representations of all nodes in the graph into a unified vector to support graph-level clustering tasks.

[0079] Subsequently, the graph embedding set Z is input into the K-means clustering algorithm to obtain a set of representative prototypes (i.e., cluster centers). Next, the distance between each graph embedding and all prototypes is calculated, and the top k% of high-confidence samples closest to each prototype are selected. This selected set of graph samples is denoted as Z. The i-th graph sample is represented as Its structure consists of the adjacency matrix A o and node attribute matrix X o As described.

[0080] To optimize the learned node embedding representation H, the following overall objective function is minimized:

[0081]

[0082] The mean squared error loss is defined as follows: Used to measure the original node feature X and the reconstructed feature. The difference between them. KL divergence loss, used to measure the similarity between two distributions, is defined as... Where, q ij ∈Q represents the soft-assignment probability distribution predicted by the model, p ij ∈P represents a predefined target distribution.

[0083] Optionally, a privacy-preserving sample transformation stage: the client collects high-confidence samples... Then, it is uploaded to the server for further personalized knowledge aggregation. However, due to privacy protection restrictions, Direct sharing is not possible. Therefore, a privacy-preserving sample transformation mechanism was designed, which generates a reconstructed structure different from the original structure while maintaining approximate consistency of the encoded result in the latent space. Specifically, a generative adversarial network was constructed, which includes a privacy-preserving structure generator. A privacy-preserving structure discriminator

[0084] In order to generate an adjacency matrix A with the original matrix A o A privacy-preserving structure generator was designed based on different reconstructed adjacency matrices. Its working process is as follows:

[0085] The first step is to calculate the mean μ and the standard deviation σ:

[0086]

[0087] Variational inference effectively extracts the statistical features of the original input information, thereby achieving controllable structural diversity in the latent space.

[0088] The second step, in order to sample from the latent space while maintaining discriminative power, employs a reparameterization mechanism to compute the standard graph-level embedding Z. φ :

[0089]

[0090] Where ∈ represents a standard normal distribution. The random noise term. This mechanism makes Z... φ It can capture key structural information while introducing a certain degree of diversity through variational inference.

[0091] The third step is to put Z φ Decoding to reconstruct the adjacency matrix A φ The decoder g is implemented using a multilayer perceptron (MLP):

[0092]

[0093] Fourth, in order to maintain structural differences while statistically enhancing the consistency between the reconstructed distribution and the original distribution, the following loss function is introduced. Optimize:

[0094]

[0095] Where p(·) represents the prior distribution and q(·) is the posterior distribution. The first KL divergence term ensures that the original structure A is obtained. o and generating structure A φ The resulting representations maintain consistency in the latent space. The second KL term helps prevent overfitting and improves the model's generalization ability by limiting the deviation of the learned distribution from the prior distribution. Overall, This maximizes the likelihood of the clustering objective, thereby ensuring that the learned adjacency matrix contains sufficient information to support effective representation learning.

[0096] The final step is to ensure the reconstruction of the adjacency matrix A. φ To ensure structural legality, symmetry and nonnegativity constraints are introduced:

[0097]

[0098] To evaluate the authenticity of the structure distribution and ensure the consistency of representation learning, a privacy-preserving structure discriminator was designed. Its goal is to determine whether the generated structure is close enough to the real structure, that is, to distinguish between the original structure and the reconstructed structure:

[0099]

[0100] Where W and b are learnable parameters, and δ(·) represents the Sigmoid activation function. Input A φ Its output is A Φ If the input is A o If the probability score is not found, the output is the original structure itself. The probability score of the output determines whether a given adjacency matrix comes from the real graph structure or was generated by the generator. Based on this discrimination mechanism, the optimization objective of the discriminator is defined as:

[0101]

[0102] in This helps enhance the model's discriminative ability and improve the realism of the generated graph structure. After the structure reconstruction is completed, the node attributes are randomly initialized using maximum likelihood estimation. Specifically, Gaussian reconstruction is performed based on the mean and variance of the node features in the original graph:

[0103]

[0104] in This represents the reconstructed attributes of the i-th node. This method effectively avoids privacy leaks while preserving the statistical features of node characteristics. Finally, representative graph-level samples (i.e., A) are used to... φ and Upload to the server.

[0105] In this embodiment, the proposed privacy-preserving sample conversion mechanism has the following two major advantages: 1) The uploaded samples are graph-level samples after structural reconstruction, avoiding direct leakage of the original data; 2) These samples maintain a distribution as similar as possible to the original samples in the latent space, thereby preserving the reliable clustering characteristics within each client, which is helpful for subsequent personalized global knowledge sharing.

[0106] Optionally, in the personalized global aggregation stage: after obtaining representative graph-level samples that differ from the original data on the server side, it is necessary to design aggregation strategies for each client to mitigate the negative impact of structural differences between different clients. To achieve this goal, the key lies in evaluating the potential similarity between the graph data uploaded by each client in the latent space, which mainly includes the following three steps. Specifically, firstly, a global learning model consistent with the structure of the local feature extractor is constructed. The model uses A φ and As input, a graph-level embedding representation is generated. The entire learning process is optimized using formula (2).

[0107] Next, the potential similarity between the learning results of each client is evaluated, calculated as follows: Among them, s ij ∈S are elements in the similarity matrix. and Let represent the graph-level average representations from the i-th and j-th clients, respectively.

[0108] Finally, personalized model parameters are calculated for each client using the similarity matrix S:

[0109]

[0110] Where C represents the total number of clients, α ij It is the normalized similarity weight between the i-th client and the j-th client. These are the personalized model parameters generated for the i-th client.

[0111] In this embodiment, the proposed personalized global aggregation strategy has the following advantages: 1) This strategy effectively utilizes the graph-level embeddings obtained from prior learning, enabling each client to obtain a unique local model aggregation method; 2) Conversely, this personalized aggregation method can also guide clients to actively discover potential relationships, further improving the performance of the local model in clustering tasks.

[0112] Optionally, in the adaptive local mask generation phase: after the server completes the personalized aggregation, each client will receive the updated model parameters. However, relying solely on a global personalized aggregation strategy has limitations, as it only considers the correlations between other client-side local models and fails to identify which specific parameters are more critical to the current client. To address this, an adaptive local masking mechanism is proposed to filter local model parameters that are beneficial for clustering.

[0113] Specifically, the first step is to calculate the gradient importance score Ψ, as shown in the following formula:

[0114]

[0115] Here, ψ represents the original gradient matrix. This step improves the model's adaptability by aligning the feature dimensions, and unifies the importance scores across different gradient dimensions through mean normalization and standard deviation scaling. The mean operation is defined as: Where L l This indicates the number of propagation rounds. Based on this, the standard deviation is calculated as follows: The final normalized importance score is obtained as follows: Where δ(·) is the Sigmoid function, which controls the influence between gradient importance and mask scaling through a negative correlation.

[0116] In the second step, to avoid drastic fluctuations in the mask value, a structure-aware update mechanism is introduced, and its update method is as follows:

[0117]

[0118] Where β is the regulation The smoothing parameter that affects the update process is M, which is an initial mask matrix that is all 1s.

[0119] In this embodiment, the advantages of the adaptive local masking mechanism are reflected in the following two aspects: 1) By introducing Sigmoid scaling, truncation constraints and normalization strategies, the stability of the optimization process is promoted, and the excessive deviation of the local model when fusing global knowledge is effectively prevented; 2) A parameter refinement method is proposed, which retains the key parameters in the local model, thereby better supporting global personalized sharing and improving the graph-level clustering effect.

[0120] To further illustrate the aforementioned beneficial effects, this application provides a verification scheme to verify the clustering effect of the trained clustering model. The details are as follows:

[0121] To evaluate the effectiveness of the proposed method, experiments were conducted using 15 graph-level benchmark datasets from five domains, including: small molecule compounds (e.g., MUTAG, BZR, COX2, DHFR, PTC_MR, AIDS, BZR_MD), bioinformatics (e.g., DD, PROTEINS), synthetic data (e.g., SYNTHETIC), social networks (e.g., COLLAB, IMDB-MULTI), and computer vision (e.g., Letter-high, Letter-low, Letter-med)

[33] . In addition, to simulate the heterogeneity of graph-level data, five non-IID settings proposed in FedGCN were used.

[0122] The model training method provided in this application embodiment is compared with two types of baseline methods: (1) three classic federated learning strategies, namely FedAvg, FedPer and FedProx; (2) six state-of-the-art federated graph learning (FGL) methods, including FedSage, GCFL, FedStar, LG-FGAD, FGAD and FedGCN.

[0123] To ensure fairness in the validation results, all methods were evaluated under the same hardware environment and configuration settings. On each client, a three-layer GIN model was used to extract graph-level representations with a local batch size of 128 and a learning rate of 1e-2. Additionally, a three-layer variational autoencoder was used as the generator, and a single-layer MLP as the discriminator, both optimized with a learning rate of 1e-3. Based on this, 10% of the samples (i.e., k = 10%) were uploaded to the server. On the server side, a three-layer GIN model was also used to extract graph-level representations, with the same batch size set for each local model. During model optimization, both the local and global models used the Adam optimizer. During federated learning, each client communicated with the server for 10 rounds, with the local model trained for 10 epochs in each round. All methods were implemented using the PyTorch framework, and experiments were conducted on a single NVIDIA GeForce RTX 4090 GPU.

[0124] Four widely used evaluation metrics were employed to assess the performance of all compared methods in graph-level clustering tasks: Accuracy (ACC), Normalized Mutual Information (NMI), Adjusted Rand Index (ARI), and F1 Score (F1). These metrics reflect different aspects of clustering performance, with higher values ​​generally indicating better performance. Performance comparisons for different federated graph-level clustering tasks are as follows:

[0125] Table 1:

[0126]

[0127] Table 1: Performance Comparison for Different Federated Graph Clustering Tasks

[0128] In Table 1, all comparison methods were evaluated in unsupervised scenarios under five non-IID settings to ensure a fair comparison. It should be noted that these supervised FGL methods have been adapted for unsupervised scenarios. Furthermore, these unsupervised FGL tasks have been transformed into federated graph clustering tasks.

[0129] Table 1 shows the performance of nine methods in federated graph-level clustering under five non-IID settings when comparing performance on the FGC task. To ensure fairness, each experiment was repeated five times, and the mean and standard deviation of the four clustering metrics were evaluated. Based on these results, the following significant observations can be summarized: 1) Compared with traditional federated learning strategies, taking the results under the SM setting as an example, the training method of the model provided in this application embodiment outperforms FedAvg, FedPer, and FedProx by 5.2%, 4.5%, and 3.5% in accuracy (ACC), respectively. These results verify that the global aggregation strategy proposed in this application embodiment can effectively promote each local model to learn a graph-level representation more suitable for clustering tasks; 2) Under all non-IID settings, PERFECT consistently outperforms FedSage, GCFL, FedStar, LG-FGAD, and FGAD in terms of ACC. This is because these FGL methods are not specifically designed for graph-level clustering tasks; 3) PERFECT surpasses the strongest baseline method, FedGCN, in clustering results. For example, under the SM, SM-BIO, SM-BIO-SY, SN, and CV settings, the ACC was improved by 0.9%, 1.2%, 1.7%, 1.0%, and 1.3%, respectively. This improvement is attributed to the training method of the model provided in the embodiments of this application, which can infer differentiated aggregation strategies for each client, thereby mitigating the adverse effects of structural heterogeneity on clustering performance.

[0130] A comparison with FGL methods with limited labels was conducted. To further verify the model training method provided in this application, it was compared with six FGL methods (i.e., FedSage, GCFL, FedStar, LG-FGAD, and FGAD) under the condition of limited label data, where each method was trained using only 2% of the labeled data. Specific verification results are as follows: Figure 3 As shown, it is easy to see that graph-level clustering tasks still demonstrate strong competitiveness even compared to federated graph learning tasks performed on limited labeled data, further highlighting the superior performance of well-trained clustering models.

[0131] To verify the effectiveness of the proposed personalized global aggregation strategy and adaptive local masking strategy, ablation experiments were designed under four non-IID settings. Specifically, "w / o P" and "w / o M" represent the two PERFECT variants with the personalized global aggregation strategy and adaptive local masking strategy removed, respectively; "BS" represents the local model of PERFECT. Figure 4 As shown, we can conclude that: 1) Compared to "BS", both "w / o P" and "w / o M" exhibit significant performance improvements in all four non-IID settings, indicating that both the personalized global aggregation strategy and the adaptive local masking strategy play a crucial role in effectively addressing federated graph-level clustering tasks; 2) Compared to "w / o P", PERFECT achieves a significant performance improvement, demonstrating that the adaptive local masking strategy can effectively preserve key model parameters, thereby enhancing the graph encoding capability of the local model; 3) PERFECT outperforms "w / o M" in all settings, further validating the effectiveness of developing differentiated aggregation strategies for each client, which helps the local model learn sample representations that are more conducive to graph-level clustering. Overall, the ablation experiment results fully validate the rationality and effectiveness of the design of each component in the model training scheme provided in this application.

[0132] In the application embodiments, three variants are derived from the personalized federated graph clustering network and comprehensively evaluated. Specifically, the three variants are defined as follows: 1) Local: This variant relies solely on local training on each client, completely abandoning knowledge sharing between clients; 2) PERFECT_V1: This variant represents learning a global consensus model parameter on the server side and sending it back to the client to guide local model training; 3) PERFECT_V2: This variant removes the personalized aggregation strategy on the server side and instead uses an average aggregation method to fuse the global model parameters uploaded by each client, and then sends the fused consensus model parameters back to the client for training. Table 2 below shows the experimental results of PERFECT and its three variants under five non-IID settings:

[0133]

[0134] Table 2: Analysis of Personalized Federated Graph Clustering Networks and Their Three Variants in Five Non-IID Settings

[0135] In Table 2, ↑ indicates the average performance improvement across the four evaluation metrics. The results in Table 2 reveal the following: First, Local shows a significant performance decline compared to PERFECT_V1, PERFECT_V2, and the Personalized Federated Graph Clustering Network. This is primarily because Local focuses solely on local training, neglecting collaboration with other local models. Second, under the setting of learning global consensus model parameters, PERFECT_V1 and PERFECT_V2 show performance improvements compared to the Local model, with average increases of 1.7%, 1.6%, 0.1%, 12.3%, and 0.7% under the SM, SM-BIO, SM-BIO-SY, SN, and CV settings, respectively. However, these methods still lag significantly behind the Personalized Federated Graph Clustering Network. This indicates that uploading de-anonymized representative samples to the server helps local models achieve better clustering, but consensus from multi-source information may slightly deflect the learning direction of multi-client models. Finally, the Personalized Federated Graph Clustering Network significantly outperforms PERFECT_V2 across all settings. This finding demonstrates the crucial importance of personalized aggregation strategies, which can effectively uncover potential relationships between clients and tailor aggregation strategies for each client, thereby enabling local models to improve their coding capabilities and achieve better clustering performance.

[0136] The Personalized Federated Graph Clustering Network introduces a hyperparameter k to select the representative proportion of samples uploaded to the server. Hyperparameter experiments were conducted under five non-IID settings, adjusting k from 1% to 20% to analyze its impact on performance. Figure 5 As shown, experimental results indicate that the model performance first increases and then decreases with the change of $k$. This suggests that during data completion, the personalized federated graph clustering network requires a suitable value of k to collect and retain the most representative information. Furthermore, under all non-IID settings, the optimal k value falls within the range of [1%, 20%], indicating that choosing a relatively small k value is both reasonable and practically beneficial for federated graph clustering tasks. Based on these observations, the value of k is set to 10%.

[0137] In this validation scheme, experiments under five non-IID settings demonstrate that the personalized federated graph clustering network can effectively uncover potential correlations between clients. Future work could further leverage the concept of personalized learning to explore missing graph-level samples and extend the proposed personalized federated graph clustering network into a more efficient version.

[0138] Combination Figure 5 As shown, this application embodiment provides a model training device, which is integrated into the server in the foregoing embodiment. The device includes:

[0139] The acquisition module 501 is used to acquire the graph-level representative samples and model parameters uploaded by each client; the graph-level representative samples are reconstructed by the client based on the graph training data; the model parameters are the parameters of the client's local clustering model.

[0140] Learning module 502 is used to learn the graph-level representative samples uploaded by each client through a global clustering model in order to obtain graph-level embedding representations.

[0141] Evaluation module 503 is used to evaluate the potential similarity between graph-level representative samples uploaded by each client based on the graph-level embedding representation.

[0142] The update module 504 is used to update the model parameters uploaded by each client based on the potential similarity between the graph-level representative samples uploaded by each client.

[0143] The distribution module 505 is used to distribute the updated model parameters from each client to each client, so that each client can train the local clustering model based on the updated model parameters and graph training data.

[0144] The determination module 506 is used to determine that the global clustering model training is complete when the parameters of the global clustering model converge; the parameters of the global clustering model are generated based on the model parameters uploaded by each client.

[0145] The training device for the model provided in this application embodiment is used to execute the actions of the server in the above embodiment. Its implementation principle and technical effect are similar, and will not be described again here.

[0146] Combination Figure 6 As shown, this application embodiment provides a model training device, which is integrated into the client in the aforementioned embodiment. The device includes:

[0147] Training module 601 is used to train the local clustering model based on graph training data to obtain the model parameters of the local clustering model.

[0148] Reconstruction module 602 is used to reconstruct graph-level representative samples based on graph training data.

[0149] The upload module 603 is used to upload the model parameters of the local clustering model and the graph-level representative samples to the server, so that the server can learn the graph-level representative samples through the global clustering model to obtain the graph-level embedding representation. Based on the graph-level embedding representation, the server evaluates the potential similarity between the graph-level representative samples uploaded by each client, updates the model parameters uploaded by each client according to the potential similarity between the graph-level representative samples uploaded by each client, and distributes the updated model parameters of each client to each client respectively.

[0150] The training module 601 is also used to train the local clustering model based on the updated model parameters and graph training data.

[0151] The confirmation module 604 is used to determine that training is complete when the model parameters of the local clustering model converge.

[0152] Optionally, when the training module 601 trains the local clustering model based on the updated model parameters and graph training data, it specifically performs the following: filtering the updated model parameters through a local masking mechanism to obtain local model parameters; and training the local clustering model based on the local model parameters and graph training data.

[0153] The training device for the model provided in this application embodiment is used to execute the actions of the client in the above embodiment. Its implementation principle and technical effect are similar, and will not be described again here.

[0154] This application provides a server, which includes a processor and a memory storing program instructions. The processor is configured to execute the training method of the model in the above embodiments when running the program instructions.

[0155] This application provides a client, which includes a processor and a memory storing program instructions. The processor is configured to execute the training method of the model in the above embodiments when running the program instructions.

[0156] This application provides a storage medium storing computer-executable instructions configured to execute the training method of the model in the above embodiments.

[0157] The aforementioned storage medium can be a transient computer-readable storage medium or a non-transitory computer-readable storage medium.

[0158] The technical solutions of this application embodiment can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes one or more instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method of this application embodiment. The aforementioned storage medium can be a non-transitory storage medium, including: USB flash drive, portable hard drive, read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk, and other media capable of storing program code; it can also be a transient storage medium.

[0159] This application provides a computer program product, including a computer program that, when executed by a processor, implements the training method of the model described in the above embodiments.

[0160] The foregoing description and accompanying drawings fully illustrate embodiments of this disclosure to enable those skilled in the art to practice them. Other embodiments may include structural, logical, electrical, procedural, and other changes. The embodiments represent only possible variations. Individual components and functions are optional unless explicitly required, and the order of operation may vary. Parts and features of some embodiments may be included in or replace parts and features of other embodiments. Moreover, the terminology used in this application is for describing embodiments only and is not intended to limit the claims. As used in the description of embodiments and claims, the singular forms “a,” “an,” and “the” are intended to equally include the plural forms unless the context clearly indicates otherwise. Similarly, the term “and / or” as used in this application means including one or more of the associated listed items and all possible combinations thereof. Additionally, when used in this application, the term "comprise" and its variations "comprises" and / or "comprising" refer to the presence of stated features, integrals, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or groups thereof. Without further limitations, an element defined by the phrase "comprises a..." does not exclude the presence of other identical elements in the process, method, or apparatus that includes said element. In this document, each embodiment may focus on the differences from other embodiments, and similar or identical parts between embodiments can be referred to mutually. For methods, products, etc., disclosed in the embodiments, if they correspond to the method section disclosed in the embodiments, the relevant parts can be referred to the description of the method section.

[0161] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the embodiments of this application. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0162] The methods and products (including but not limited to devices and equipment) disclosed in the embodiments herein can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For instance, the division of units may be merely a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to implement this embodiment according to actual needs. In addition, the functional units in the embodiments of this application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0163] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than that shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. In the descriptions corresponding to the flowcharts and block diagrams in the accompanying drawings, the operations or steps corresponding to different blocks may also occur in a different order than disclosed in the description; sometimes there is no specific order between different operations or steps. For example, two consecutive operations or steps may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. Each block in a block diagram and / or flowchart, and combinations of blocks in a block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

Claims

1. A method for training a model, characterized in that, Applied to a server, wherein the server deploys a global clustering model, the server establishes connections with multiple clients, and each client deploys a local clustering model, the method includes: Obtain the graph-level representative samples and model parameters uploaded by each client; the graph-level representative samples are reconstructed by the client based on graph training data; the model parameters are the parameters of the client's local clustering model; The global clustering model is used to learn the graph-level representative samples uploaded by each client to obtain graph-level embedding representations; Based on the graph-level embedding representation, the potential similarity between the graph-level representative samples uploaded by each client is evaluated; Update the model parameters uploaded by each client based on the potential similarity between the graph-level representative samples uploaded by each client; The updated model parameters are distributed to each client respectively, so that each client can train the local clustering model based on the updated model parameters and the graph training data; When the parameters of the global clustering model converge, the training of the global clustering model is considered complete; the parameters of the global clustering model are generated based on the model parameters uploaded by each client.

2. A method for training a model, characterized in that, The method is applied to a client-side application, wherein the client has a local clustering model deployed, the client establishes a connection with a server, and the server has a global clustering model deployed; the method includes: The local clustering model is trained based on the graph training data to obtain the model parameters of the local clustering model; Graph-level representative samples are reconstructed from the graph training data; The model parameters of the local clustering model and the graph-level representative samples are uploaded to the server so that the server can learn the graph-level representative samples through the global clustering model to obtain graph-level embedding representations. Based on the graph-level embedding representations, the potential similarity between the graph-level representative samples uploaded by each client is evaluated. Based on the potential similarity between the graph-level representative samples uploaded by each client, the model parameters uploaded by each client are updated, and the updated model parameters of each client are distributed to each client respectively. The local clustering model is trained based on the updated model parameters and the graph training data; Training is considered complete when the model parameters of the local clustering model converge.

3. The method according to claim 2, characterized in that, Based on the updated model parameters and the graph training data, the local clustering model is trained, including: The updated model parameters are filtered using a local masking mechanism to obtain local model parameters; The local clustering model is trained based on the local model parameters and the graph training data.

4. A training device for a model, characterized in that, The device is integrated into a server, which deploys a global clustering model. The server establishes connections with multiple clients, and each client deploys a local clustering model. The device includes: The acquisition module is used to acquire the graph-level representative samples and model parameters uploaded by each client; the graph-level representative samples are reconstructed by the client based on graph training data; the model parameters are the parameters of the client's local clustering model. The learning module is used to learn the graph-level representative samples uploaded by each client through the global clustering model in order to obtain graph-level embedding representations; An evaluation module is used to evaluate the potential similarity between graph-level representative samples uploaded by each client based on the graph-level embedding representation; The update module is used to update the model parameters uploaded by each client based on the potential similarity between the graph-level representative samples uploaded by each client. The distribution module is used to distribute the updated model parameters from each client to each client, so that each client can train the local clustering model based on the updated model parameters and the graph training data. The determination module is used to determine that the global clustering model training is complete when the parameters of the global clustering model converge; the parameters of the global clustering model are generated based on the model parameters uploaded by each client.

5. A training device for a model, characterized in that, The device is integrated into a client, which deploys a local clustering model. The client establishes a connection with a server, and the server deploys a global clustering model. The device includes: The training module is used to train the local clustering model based on the graph training data to obtain the model parameters of the local clustering model. The reconstruction module is used to reconstruct graph-level representative samples based on graph training data. The upload module is used to upload the model parameters of the local clustering model and the graph-level representative samples to the server, so that the server can learn the graph-level representative samples through the global clustering model to obtain graph-level embedding representations, and evaluate the potential similarity between the graph-level representative samples uploaded by each client based on the graph-level embedding representations, so as to update the model parameters uploaded by each client according to the potential similarity between the graph-level representative samples uploaded by each client, and distribute the updated model parameters of each client to each client respectively. The training module is also used to train the local clustering model based on the updated model parameters and the graph training data; The confirmation module is used to determine that training is complete when the model parameters of the local clustering model converge.

6. The apparatus according to claim 5, characterized in that, When the training module trains the local clustering model based on the updated model parameters and the graph training data, it is specifically used for: The updated model parameters are filtered using a local masking mechanism to obtain local model parameters; The local clustering model is trained based on the local model parameters and the graph training data.

7. A server, characterized in that, The server is deployed with a global clustering model, and the server establishes connections with multiple clients, each of which deploys a local clustering model; the server includes a processor and a memory storing program instructions, the processor being configured to execute the training method of the model as described in claim 1 when running the program instructions.

8. A client application, characterized in that, The client has a local clustering model deployed, the client establishes a connection with the server, and the server has a global clustering model deployed; the client includes a processor and a memory storing program instructions, the processor being configured to execute the training method of the model as described in claim 2 or 3 when running the program instructions.

9. A storage medium storing program instructions, characterized in that, When the program instructions are executed, they perform the training method of the model as described in any one of claims 1 to 3.

10. A computer program product, characterized in that, It includes a computer program, which, when executed by a processor, implements the training method for the model as described in any one of claims 1 to 3.

Images