A communication-efficient adaptive cluster-level graph-enhanced federated recommendation method

Abstract

The application discloses a communication-efficient adaptive cluster-level graph-enhanced federated recommendation method, which adopts a coordinate-level Top-k sparse upload and error feedback mechanism on the client side, reduces the communication burden, and relieves the compression bias; on the server side, adaptive clustering is carried out based on a low-dimensional signature, the cluster structure is dynamically adjusted through a dispersion criterion, and the client heterogeneity is accurately described; then, a client relationship graph is constructed only in the cluster and light-weight graph propagation and aggregation are executed, the local collaborative modeling is enhanced, and the cross-group noise is inhibited; meanwhile, a hybrid delivery mechanism with cache enhancement is designed, and global sharing and personalized expression are balanced. Experimental results show that the C²G-FedRec method has advantages in recommendation accuracy, communication efficiency and convergence stability, and can realize better recommendation performance under lower communication overhead.

Description

Technical Field

[0001] This invention relates to the field of communication technology, and in particular to a highly efficient adaptive cluster-level graph-enhanced federated recommendation method. Background Technology

[0002] With the widespread application of recommender systems in content distribution, e-commerce, and online services, achieving high-quality personalized recommendations while protecting user privacy has become a key research challenge. However, traditional centralized recommender methods rely on centralized storage and unified training of user interaction data, which presents serious challenges such as the risk of leakage of sensitive user information, the vulnerability of centralized architectures to attacks, and difficulty in adapting to current stringent privacy compliance requirements. Federated learning, a new framework and paradigm, is a collaborative training method that trains global machine learning models through distributed iterative collaboration without requiring users to share private data. Through a collaborative training mechanism where "data does not leave the source," it provides a feasible path for recommender systems to balance privacy protection and model learning capabilities. Therefore, federated recommendation is gradually becoming an important research direction at the intersection of recommender systems and federated learning.

[0003] While federated recommendation has inherent advantages in privacy protection, it still faces several challenges in practical applications. First, recommendation models typically involve large item embedding matrices, requiring repeated exchange of model parameters or updates between the client and server in multiple rounds of communication, resulting in high communication overhead and becoming a significant factor limiting system deployment. Second, significant heterogeneity in preferences and differences in data distribution are prevalent among different clients, and simply adopting a uniform global aggregation strategy can easily lead to insufficient modeling ability to characterize local user preferences. Furthermore, when introducing relationship modeling or graph aggregation mechanisms into federated recommendation to enhance the utilization of collaborative information, direct global relation propagation may cause interference between collaborative signals from dissimilar clients, introducing cross-group noise and affecting training stability and final performance. Summary of the Invention

[0004] The purpose of this invention is to provide a communication-efficient adaptive cluster-level graph-enhanced federated recommendation method, C²G-FedRec. By designing a unified framework for federated recommendation, coupling communication compression, heterogeneity awareness, and local collaborative modeling, recommendation performance and training stability are improved while reducing communication overhead. On the client side, coordinate-level Top-k sparse uploading is performed on item embedding differences, and an error feedback mechanism is used to mitigate information loss caused by compression. On the server side, the client update direction is characterized based on low-dimensional signatures, and the client cluster structure is dynamically partitioned through a discreteness-driven adaptive clustering strategy. Furthermore, a client relationship graph is constructed only within clusters, and lightweight graph propagation is performed to enhance high-order collaborative information between similar clients while effectively suppressing cross-group noise propagation. In addition, a cache-enhanced hybrid delivery mechanism is introduced to achieve a cross-round balance between global sharing and personalized expression.

[0005] To achieve the above objectives, the present invention is implemented according to the following technical solution:

[0006] This invention adopts a three-stage structural design of "compressing and uploading first, then grouping and modeling, and finally cluster-level aggregation and distribution", which integrates communication compression, heterogeneity perception and cluster-level collaborative modeling into the same training framework to simultaneously alleviate communication bottlenecks and heterogeneity mismatch.

[0007] The first stage involves communication compression and local signature construction: Considering that item embeddings often account for the majority of communication costs in federated recommendations, the client does not directly upload complete updates. Instead, it performs Top-k sparse compression on the embedding differences and combines this with an error feedback mechanism to compensate for untransmitted information across rounds, thereby reducing uplink communication overhead while maintaining the stability of the recommendation model optimization process as much as possible. Simultaneously, the client extracts a low-dimensional signature representation based on the embedding differences and performs privacy processing through operations such as random projection, norm pruning, and noise injection, thus constructing a compact representation that characterizes the client's preference update direction for subsequent heterogeneity-aware modeling.

[0008] The second stage is heterogeneity-aware modeling: To avoid forcibly mixing clients with large differences in preferences in highly heterogeneous scenarios, the server characterizes the similarity between clients based on the privacy signatures uploaded by the clients, and performs adaptive clustering to construct a client relationship graph. The core objective of this stage is to use update direction information to divide clients into groups with high homogeneity, so that subsequent aggregation operations can be performed within user groups with more similar preferences, thereby reducing ineffective collaboration and improving training stability.

[0009] The third stage is cluster-level graph aggregation and distribution: After obtaining the client's cluster structure, the server only constructs a relationship graph within the cluster and performs lightweight graph propagation to mine local high-order collaborative signals within similar user groups and suppress cross-group noise propagation. Based on this, the server combines cluster-level representation, global representation, and user history cache to generate the item embeddings for the next round of distribution, thereby achieving a dynamic balance between globally shared information and personalized preference expression.

[0010] The beneficial effects of this invention are:

[0011] This invention integrates communication compression, adaptive clustering, and graph collaborative modeling; based on a signature-driven adaptive clustering mechanism, the cluster structure is dynamically adjusted through a discreteness criterion to effectively characterize client heterogeneity; this invention proposes a graph aggregation and cache-enhanced distribution mechanism for cluster-level local collaboration, which improves recommendation performance while enhancing training stability; experiments were conducted on multiple datasets and with low participation rates to verify the comprehensive advantages of the C²G-FedRec method in terms of recommendation accuracy and communication efficiency. Attached Figure Description

[0012] Figure 1 This is an overall framework diagram of the C²G-FedRec method of the present invention. Detailed Implementation

[0013] The present invention will be further described below with reference to the accompanying drawings and specific embodiments. The illustrative embodiments and descriptions herein are used to explain the present invention, but are not intended to limit the present invention.

[0014] Federated recommendation aims to train recommendation models through a federated learning mechanism without centrally storing users' original interaction data. Existing research typically starts from traditional recommendation frameworks, transferring matrix factorization, neural collaborative filtering, and graph recommendation methods to federated scenarios to balance privacy protection and recommendation performance. FedNCF is a representative work that introduced neural collaborative filtering into federated recommendation early on, verifying the feasibility of related models under federated training and emphasizing the role of secure aggregation in updating recommendation models. PFedRec further points out that federated recommendation not only needs to learn a globally shared model but should also explicitly model the personalized differences between the user side and the item side to improve adaptability to local preferences.

[0015] Building upon this, GPFedRec introduces relationship modeling to enhance the utilization of collaborative information in federated recommendations by constructing a client-side relationship graph on the server side and performing graph-guided aggregation. This type of approach demonstrates that leveraging the latent relationship structure between clients can effectively improve recommendation performance. However, these methods primarily focus on model expressiveness and personalized modeling, neglecting the transmission costs of large-scale item embeddings in multi-turn communications. Furthermore, their relationship modeling is typically performed globally, which can easily introduce cross-group noise propagation in scenarios with high client heterogeneity, thus affecting training stability and final recommendation performance.

[0016] Efficient Federated Learning and Federated Recommendation in Communication:

[0017] Communication efficiency is one of the core issues in federated learning. Regarding model update compression, several classic strategies have been proposed, including Top-k sparsity, error feedback, quantization compression, and low-rank approximation. Among these, the Top-k method significantly reduces communication overhead by uploading only the parameters with larger amplitudes; the error feedback mechanism mitigates information loss caused by compression by accumulating residuals across rounds; and quantization methods further compress communication costs by reducing numerical precision. These techniques have been widely validated as effective in general federated optimization problems.

[0018] In recent years, communication optimization mechanisms have been introduced into the federated recommendation field. For example, CoLR-FedRec uses low-rank representation to reduce the scale of parameter transmission, and FNCF-MAB adaptively selects uploaded content through a multi-armed slot machine mechanism. These methods effectively answer the question of "how to reduce communication overhead," but they usually treat communication compression as an independent component, lacking deep integration with client heterogeneity and collaborative modeling mechanisms in recommendation tasks. Therefore, relying solely on communication compression is insufficient to improve the overall performance and stability of federated recommendations.

[0019] Heterogeneity-aware federated learning and graph collaborative modeling:

[0020] In non-independent identically distributed (Non-IID) scenarios, heterogeneity among clients is a significant factor affecting the performance of federated learning. Clustering federated learning alleviates the problem of the global model's inability to adapt to diverse data distributions by dividing clients with similar characteristics into different clusters and aggregating the model at the cluster level. Existing research indicates that the client grouping structure should be dynamically adjusted during the training process, rather than remaining fixed.

[0021] On the other hand, graph structure modeling has been proven to effectively capture high-order collaborative signals in recommender systems. Through graph messaging mechanisms, local neighborhood information between users or between users and items can be fused, thereby improving recommendation performance. GPFedRec introduced graph modeling into federated recommendations, demonstrating the important role of relational structures in collaborative learning. Meanwhile, graph learning research such as GloMP-GNN further points out that traditional layer-by-layer local messaging easily weakens the information propagation of high-order similar nodes in heterogeneous or dissimilar graphs, while more effective high-order propagation mechanisms help enhance relational modeling capabilities. However, existing methods often construct graph structures globally, ignoring the heterogeneity differences between clients, which can easily lead to information mixing and noise diffusion between dissimilar clients in highly heterogeneous scenarios.

[0022] In summary, existing research on federated recommendation has made progress in communication optimization, heterogeneity modeling, and graph collaborative learning, but most of these have developed independently in different directions, lacking a unified mechanism to effectively combine the three. Against this backdrop, this invention proposes a federated recommendation method that unifies and couples communication compression, adaptive clustering, and intra-cluster graph aggregation.

[0023] C²G-FedRec method:

[0024] This invention will detail the complete technical solution of the proposed C²G-FedRec method, including problem definition and symbol explanation, overall framework design, communication compression mechanism, heterogeneity-aware modeling method, and cluster-level collaborative modeling and distribution strategy.

[0025] Problem definition:

[0026] Consider a federated recommendation scenario: the server coordinates a set of users. corresponding One client, each client Only holds local interaction data The server cannot directly access the raw interaction data of these clients. The item collection is denoted as... Its scale is .

[0027] The model parameters consist of two parts: 1) a globally shared item embedding matrix 2) Client-side local parameters This includes user representations and prediction network parameters (such as local MLP parameters).

[0028] During training, the client performs joint optimization based on local data. and But only uploaded Update Used for server aggregation; local parameters It remains on the client side and does not participate in cross-client sharing or aggregation. After receiving updates uploaded by the client, the server... Aggregation and updates are performed, and then distributed to each client. This configuration aligns with personalized federated recommendation methods such as PFEdRec, which share item representations while preserving user-specific parameters. The goal of this design is to reduce communication and aggregation costs without exposing local data, and to improve recommendation performance and training stability in heterogeneous client scenarios.

[0029] Under the above settings, the optimization objective of the present invention is defined as follows: Let the first... The set of participating clients is ,in This indicates the embedding of items sent from the server to the client. This indicates that the item embedding has been completed on the client side after local updates. (For the client...) Its local target is defined as

[0030]

[0031] Based on this, define the first The recommended wheel-level objective by the wheel federation is

[0032]

[0033] Therefore, the first The round optimization problem can be written as

[0034]

[0035] Symbol interpretation:

[0036] To facilitate the subsequent method description, the main symbols are defined uniformly as shown in Table 1. The remaining symbols will be defined and explained in the subsequent method section.

[0037] Table 1. Main Symbols and Their Meanings

[0038] symbol meaning Client Collection Item collection Client Index Communication rounds No. Round participation client set No. The server sends the message to the client. Item embedding Embedded items after local client update Embedded update differential on the client Update after error feedback correction Compressed update package uploaded by the client Client update after decompression Client residuals Client-side low-dimensional signature representation Privacy-processed client signature No. Number of round clusters Client In the Cluster allocation of rounds Cluster-level polymer embedded Server-wide item embedding

[0039] Overall Framework of the C²G-FedRec Method

[0040] While balancing privacy protection and model learning capabilities, federated recommendations still face challenges such as high communication overhead, strong client heterogeneity, and the potential for cross-group noise to be introduced during global relationship propagation. To address these challenges, this invention proposes C²G-FedRec, which follows a three-stage structural design of "compressed upload, grouped modeling, and cluster-level aggregation and distribution." This design unifies communication compression, heterogeneity awareness, and cluster-level collaborative modeling into a single training framework, thereby simultaneously alleviating communication bottlenecks and heterogeneity mismatch.

[0041] The first stage involves communication compression and local signature construction. Considering that item embeddings often account for the majority of communication costs in federated recommendations, the client does not directly upload the complete update. Instead, it performs Top-k sparse compression on the embedding differences and incorporates an error feedback mechanism to compensate for untransmitted information across rounds, thereby reducing uplink communication overhead while maintaining the stability of the recommendation model optimization process as much as possible. Simultaneously, the client extracts a low-dimensional signature representation based on the embedding differences and performs privacy processing through random projection, norm pruning, and noise injection, thus constructing a compact representation that characterizes the client's preference update direction for subsequent heterogeneity-aware modeling.

[0042] The second stage is heterogeneity-aware modeling. To avoid forcibly mixing clients with significantly different preferences in highly heterogeneous scenarios, the server characterizes the similarity between clients based on the privacy signatures uploaded by the clients, and performs adaptive clustering to construct a client relationship graph. The core objective of this stage is to use update direction information to segment clients into groups with higher homogeneity, so that subsequent aggregation operations can be performed within user groups with more similar preferences, thereby reducing ineffective collaboration and improving training stability.

[0043] The third stage is cluster-level graph aggregation and distribution. After obtaining the client's cluster structure, the server only constructs a relationship graph within the cluster and performs lightweight graph propagation to mine local high-order collaborative signals within similar user groups and suppress cross-group noise propagation. Based on this, the server combines cluster-level representation, global representation, and user history cache to generate the item embeddings for the next round of distribution, thereby achieving a dynamic balance between globally shared information and personalized preference expression.

[0044] The three stages described above are not independent, sequential components, but rather work collaboratively towards the same federated recommendation goal: communication compression determines the way uploaded information is expressed, heterogeneity perception determines the scope of local collaboration, and cluster-level representation updates and distribution determine how the server leverages the relationship structure between similar clients to improve the quality of the recommendation representation. To further illustrate the overall process, Figure 1 The overall framework of the C²G-FedRec method is presented.

[0045] The specific process is as follows: After receiving the item embeddings from the server, the client performs local training to obtain the updated embedding differences. And construct a correction update through an error feedback mechanism. Then, Top-k sparse compression is performed to generate the upload packet. Simultaneously, low-dimensional signature representations are extracted based on differential extraction. On the server side, the server first decompresses and reconstructs the compressed update to restore the client's local embedding; then, adaptive clustering is performed based on the privacy signature, and the cluster structure is dynamically adjusted according to the intra-cluster dispersion to form client groups with high homogeneity. In the cluster-level representation update stage, the server constructs a relationship graph within each cluster and performs lightweight graph propagation to obtain cluster-level item embeddings; then, by combining global embeddings and user cached representations, the item embeddings for the next round are generated, thereby achieving a dynamic balance between global sharing and personalized expression.

[0046] The diagram corresponds one-to-one with the "three-stage structure" in the method overview: the information construction and compression process on the client side corresponds to the first stage, the clustering and structural modeling process on the server side corresponds to the second stage, and the intra-cluster propagation and representation fusion process corresponds to the third stage. In this way, the diagram concretizes the abstract process, thus more clearly demonstrating the information flow and dependencies between modules. Through this design, C²G-FedRec, without changing the basic training paradigm of federated recommendation, integrates communication constraints and heterogeneity constraints into the same optimization process, thereby providing a consistent overall perspective for the subsequent development of each sub-module.

[0047] Phase 1: Communication Compression and Local Signature Construction

[0048] In the first phase, the client performs communication compression and signature construction around local item embedding updates. Specifically, the client first calculates the local update difference based on the item embeddings sent by the server, and reduces upload overhead and mitigates compression bias through Top-k sparse compression and error feedback mechanisms. Subsequently, a low-dimensional signature representation is extracted from the high-dimensional difference and privacy processing is performed to provide a compact representation foundation for subsequent client similarity modeling. The core objective of this phase is to retain the key information required for subsequent heterogeneity-aware modeling while controlling communication costs.

[0049] Embedded update difference computation

[0050] In the In the round, the client Receive item embedding from server The updated item embeddings are obtained after local training. Therefore, the client updates the embedded difference definition.

[0051]

[0052] This difference characterizes the updated information generated by the client in the current round based on local data regarding the representation of shared items, serving as the foundation for subsequent compression and uploading. Compared to directly transmitting the complete embedded representation, the update difference typically exhibits stronger sparsity and structure, making it more suitable for subsequent communication compression and efficient transmission.

[0053] Top-k sparse compression and error feedback

[0054] In recommendation tasks, item embedding updates typically exhibit a sparse saliency distribution, allowing Top-k sparse strategies to retain the main optimization direction. However, Top-k sparse compression inevitably leads to some information loss. To address this issue, this invention employs an error feedback mechanism, incorporating untransmitted information from the previous round into the current update to obtain the corrected update amount.

[0055]

[0056] Subsequently, Perform coordinate-level Top-k compression, retaining the largest absolute value. Each coordinate is used, and the compressed result is denoted as... Wherein, the coordinate-level Top-k means first flattening the difference matrix into a one-dimensional vector, and then selecting the top k according to the magnitude. Upload coordinates individually; compared to coarse-grained compression by filtering by row, this method can retain key update information with finer granularity. This is a reconstruction update corresponding to the compression result; specifically, it is based on the compressed package. The recovered dense reconstruction update retains only the coordinates selected by the Top-k coordinates, setting the rest to zero. The client-side error feedback residual update is then...

[0057]

[0058] This mechanism effectively mitigates information loss caused by Top-k compression by accumulating and compensating for untransmitted update information across rounds, thus maintaining the stability of the optimization process while reducing communication overhead. The client ultimately uploads a compressed update package. Instead of a complete item embedding matrix, the client constructs the differential matrix using a reference baseline that is consistent with the server's decompression and reconstruction baseline, ensuring that compression, decompression, and subsequent aggregation processes always revolve around the same reference baseline.

[0059] Signature Construction and Privacy Handling

[0060] While completing the compressed update construction, the client is based on embedded differential. A low-dimensional signature representation is extracted to characterize its update direction. Specifically, mean pooling is performed on the difference matrix along the item dimension, defining the original signature as...

[0061]

[0062] in, This represents a signature construction operator that performs mean pooling on the item dimension to extract a compact directional representation from the high-dimensional embedding difference. Although this representation is a low-dimensional compressed form, its mean representation can still retain the main update trend information due to the consistency of the embedding difference in direction. This reflects the potential trend of preference changes and provides a basis for subsequent client similarity measurement and clustering.

[0063] Based on this, to reduce the risk of the signature revealing high-dimensional update details, further privacy processing is performed on the original signature to obtain a privacy signature.

[0064]

[0065] in, Indicates random projection, The privacy processing flow consists of norm pruning and Gaussian noise injection, in sequence. The client then uploads a compressed update. With privacy signature The latter is used for subsequent heterogeneity-aware modeling on the server side, and does not directly participate in parameter aggregation. This design further reduces the potential risk of privacy leakage while preserving update direction information.

[0066] Phase 2: Heterogeneity-aware modeling

[0067] In the second stage, the server characterizes the preference similarity among clients based on the privacy signatures uploaded by the clients, and constructs structured client groups through adaptive clustering and intra-cluster graph relationship modeling, thus providing a foundation for subsequent cluster-level collaborative modeling. The core of this stage lies in using update direction information to reasonably divide clients and further explicitly modeling the relationship structure between clients within a cluster, enabling collaborative modeling to be carried out within a more consistent group, thereby reducing interference from heterogeneity.

[0068] Adaptive clustering and splitting mechanism

[0069] In each round, the server determines the privacy signature based on the data uploaded by the participating clients. The cluster structure is re-estimated. During initialization, all participating clients are uniformly assigned to the same initial cluster. Subsequently, the server dynamically adjusts the cluster structure based on changes in the privacy signature. Specifically, the server calculates the cosine similarity between the client's signature and the current cluster center, and assigns the client to the cluster with the highest similarity. To improve clustering stability, the cluster centers are updated using an exponential moving average, thereby mitigating the impact of fluctuations in a single update.

[0070] After obtaining cluster allocation, let the first... Round Each cluster is Its mean signature is defined as

[0071]

[0072] Furthermore, the intra-cluster dispersion is defined as

[0073]

[0074] When the constraints of minimum cluster size and maximum cluster number are satisfied, and When a preset threshold is exceeded, the server performs a binary search operation on the corresponding cluster. This mechanism enables the cluster structure to evolve adaptively during the training process, thereby improving intra-cluster homogeneity and reducing ineffective aggregation between heterogeneous clients.

[0075] Cluster intra-cluster relation graph modeling

[0076] After obtaining the client cluster partitioning, the server further characterizes the structural relationships between clients within each cluster to provide a topological foundation for subsequent collaborative modeling. Compared to global graph modeling, intra-cluster graph construction can limit the scope of information propagation, avoid the mixing of invalid information between heterogeneous clients, and thus improve the purity of collaborative signals.

[0077] Specifically, for the first Each cluster of the wheel The server first decompresses the compressed update package uploaded by the client to obtain the reconstructed update. And restore the client's local embedded representation:

[0078]

[0079] Based on this, the recovered embedding representations of each client within the cluster are flattened into vector form, and a pairwise relation score matrix is ​​calculated accordingly. Specifically, the server uses the recovered item embedding vectors of the clients as input to calculate the pairwise relation scores between clients within the cluster; subsequently, a threshold-based neighborhood filtering strategy is used to construct the intra-cluster relation matrix. For each client, only local neighbor connections with relation scores higher than a given threshold are retained; for the retained neighbor edges, they are then uniformly normalized and weighted according to the number of neighbors, ultimately yielding the intra-cluster relation matrix used for graph propagation. .

[0080] Through the above process, the server can obtain a graph structure reflecting the client relationships within the cluster in each round, thereby further transforming the heterogeneity problem into a structured modeling problem and laying the foundation for subsequent cluster-level aggregation updates.

[0081] Phase 3: Cluster-level graph aggregation and distribution

[0082] In the third stage, the server further performs cluster-level collaborative modeling and representation updates based on the client cluster structure and intra-cluster relationship graph obtained in the second stage. Specifically, firstly, local high-order collaborative information is mined within the cluster through lightweight graph propagation to obtain a cluster-level representation; then, the cluster-level representation is fused with the global representation and user history cache to generate the item embedding for the next round of distribution, thereby achieving a dynamic balance between global sharing and personalized expression.

[0083] Intra-cluster GloMP-lite graph aggregation

[0084] After obtaining the intra-cluster relationship graph, the server in each cluster Lightweight graph propagation is performed internally to model high-order collaboration relationships between clients within a cluster. Considering that one-hop propagation is insufficient to fully capture these high-order collaboration relationships, while excessively deep propagation may introduce oversmoothing and increase computational overhead, this invention employs a two-hop lightweight graph propagation mechanism. Specifically, let... Embedding the row-stacked representation matrix into the intra-cluster clients, the two-layer graph propagation can be written as:

[0085]

[0086]

[0087] Based on the two-hop propagation results, the cluster-level intermediate representation is first defined as follows:

[0088]

[0089] in This is used to balance the information propagated within the cluster graph with the original intra-cluster representation. Furthermore, the final cluster-level embedding is redefined as...

[0090]

[0091] in This controls the fusion strength between the cluster-level representation and the globally shared representation. Thereafter, the final representation will be used consistently. This represents the cluster-level aggregation result. The above aggregation process is always restricted to within a cluster to suppress cross-group noise propagation on the global user graph; the above design maintains lightweight propagation while also taking into account the modeling of local high-order cooperative information.

[0092] Cluster-level fusion and global update

[0093] In obtaining the aggregate embeddings of each cluster Afterwards, the server updates the global item embedding. Specifically, the global embedding is obtained by weighting the embeddings of each cluster by cluster size:

[0094]

[0095] This update method ensures that the contribution of different clusters to the global model matches their size, thereby comprehensively reflecting the collaborative information of various clients in the globally shared representation.

[0096] Cache-enhanced hybrid delivery mechanism

[0097] To balance global sharing and personalized representation during cross-round training, the server maintains a user-level cache embedding for each client. For clients participating in the current round of training Its cache is updated to the aggregate embedding of the corresponding cluster:

[0098]

[0099] in Indicates client In the Cluster allocation for each round. For clients not participating in this round of training. Its cache remains unchanged:

[0100]

[0101] In the next round, the server executes a cluster-aware delivery strategy based on cached information. This applies to clients that already have cached data. Its embedded definition is:

[0102]

[0103] For users who do not yet have a cache, global embedding will be used directly. This serves as the baseline for deployment. In round 0, all participating clients have no historical cache, therefore a global baseline cold start is uniformly adopted; the first batch of user caches is established only after the aggregation in round 0 is completed. To alleviate the excessively strong personalized bias that may result from embedding historical caches in the early stages of training, this invention introduces piecewise linear mixing coefficients. Let the total number of rounds be... The update interval is ,but Only for Update when the value is an integer multiple of the given value, and satisfy the following conditions:

[0104]

[0105] Therefore, in stage, Gradually increase; when hour, This remains unchanged. The design allows for greater reliance on globally shared information in the early stages of training, gradually enhancing the role of personalized cluster-level representations in the mid-to-late stages. It's important to note that global embeddings are consistently involved in the final setup: serving as a fallback baseline for uncached users and as a global component in the blend for cached users. .

[0106] The algorithm pseudocode is shown in Table 2.

[0107] Table 2. Pseudocode for C²G-FedRec Federated Training Process

[0108] Algorithm 1: C²G-FedRec Federated Training Process Input: Total number of rounds Client engagement rate Server initial state , , Output: Final global embedding Cluster-level embedding With user cache 1. Initialize server state, user cache, and error feedback residuals. 2 for do:3 Sampling client set by participation rate Phase 1: Communication Compression and Local Signature Construction 4 for do: 5. Issued The client-side local training yielded 6. Calculate the updated difference 7. Construction error feedback correction update 8. Perform coordinate-level Top-k compression to obtain = 9. Restructuring And update the residuals 10. Construction 11 Uploads and To server 12 end for phase 2: heterogeneity-aware modeling 13 server decompression of all get 14 Refactoring the client-side local embedding 15. Based on privacy signatures If the conditions for updating cluster allocation 16 are met, then perform adaptive splitting to obtain the cluster set. 17. Based on intra-cluster client reconstruction, embedding the relationship graph construction in stage 3: cluster-level association graph aggregation and distribution 18 for do: Perform GloMP-lite aggregation on the 19 cluster intragraphs to obtain 20end for 21 Update , Cache for participating communication clients 22 Calculation 23 end for

[0109] Experimental Results and Analysis

[0110] Experimental setup

[0111] Datasets and their partitioning

[0112] Experiments were conducted on four publicly available datasets: MovieLens-100K, MovieLens-1M, LastFM-2K, and HetRec2011-ML-2K. To ensure comparability, all methods employed a consistent data preprocessing workflow: first, users with fewer than 3 interactions were filtered out; then, a Leave-One-Out split was performed, ensuring that each valid user contained samples from the training, validation, and test datasets. Statistical information for each processed dataset is shown in Table 3.

[0113] Table 3 Statistical information of the experimental dataset

[0114] Dataset Number of users Item count Total number of interactions sparsity ML-100K 943 1682 100000 93.70% ML-1M 6040 3706 1000209 95.53% LFM-2K 1883 17626 92824 99.72% HR-2K 2113 10109 855598 96.01%

[0115] Training settings

[0116] In all experiments using this algorithm, the client-side employed an embedding-based neural collaborative filtering model for local training. Specifically, users and items were mapped as follows: The embedding vectors are used to model interactions through a multilayer perceptron, ultimately outputting user preference scores for items. Unless otherwise specified, all methods use the same parameter settings to ensure the fairness of experimental results. Unless otherwise specified, experiments are conducted with a client participation rate of 0.1 to simulate real-world conditions. Other key hyperparameters remain consistent: local training epochs are set to 1, batch size to 256, embedding dimension to 32, and negative sampling number to 4, etc. C²G-FedRec enables coordinate-level Top-k upload, error feedback, adaptive discrete clustering, intra-cluster GloMP-lite aggregation, and cache-enhanced segmented hybrid delivery by default.

[0117] Evaluation indicators and results reporting methods

[0118] Evaluation metrics used were HR@10 and NDCG@10. HR@10 represents a hit if a positive test sample appears in the top 10 recommendation results; NDCG@10 is calculated based on the ranking position of the positive test sample in the top 10 results. During training, each method recorded test results round by round, and the best test metric and the round in which it was achieved were uniformly reported in the main table. In the communication analysis, the cumulative upload cost at which the corresponding optimal accuracy was achieved was further statistically analyzed. For baselines that could only be approximately aligned, their protocol differences were simultaneously annotated in the result interpretation to avoid inappropriate comparisons.

[0119] To present the algorithm's performance, the following sections will discuss it from six aspects: comparative experiments, client participation rate analysis experiments, ablation experiments, upload sparsity analysis experiments, upload communication overhead comparison experiments, and noise sensitivity analysis experiments.

[0120] Comparative experiment

[0121] Comparison Method Explanation

[0122] The comparison methods include FedNCF, PFedRec, GPFedRec, CoLR-FedRec, FNCF-MAB, and three communication compression variants based on FedNCF: Top-k FedNCF, EF-Top-k FedNCF, and QuantizedFedNCF. Top-k FedNCF performs coordinate-level Top-k sparsity on model updates in each round of communication, uploading only the k parameters with the largest amplitudes. EF-Top-k FedNCF further introduces an error feedback mechanism to mitigate compression bias by accumulating untransmitted information across rounds. Quantized FedNCF quantizes model updates to reduce communication overhead. All three variants maintain the original FedNCF model structure and local training method, modifying only the communication mechanism.

[0123] To ensure fairness in the comparison, we standardized key settings for each method during the experiment, including data partitioning, random seed, number of training rounds, client participation rate, number of local training rounds, batch size, embedding dimension, and negative sampling number.

[0124] Recommended performance comparison

[0125] Table 4 shows the best HR@10 and NDCG@10 achieved by each method on the four datasets with a client participation rate of 0.1. The results show that the proposed C²G-FedRec achieves the best HR@10 and NDCG@10 on the ML-100K dataset, improving upon the best baseline PFEdRec by approximately 0.37% and 3.26%, respectively. On the ML-1M, LFM-2K, and HR-2K datasets, although the proposed method does not achieve the absolute best performance, it still maintains strong competitiveness overall. For example, on the ML-1M dataset, compared to the best baseline PFEdRec, the proposed method's HR@10 and NDCG@10 differ by only approximately 2.48% and 4.44%, respectively, and C²G-FedRec still maintains good recommendation performance compared to other methods. Although PFEdRec or CoLR-FedRec achieve higher performance values ​​on some datasets, these methods are generally accompanied by higher communication overhead; further analysis will be conducted in subsequent communication experiments. It is noteworthy that on the LFM-2K dataset, the performance of FedNCF and its communication variants (including Top-k FedNCF, EF-Top-k FedNCF, and Quantized FedNCF) degrades significantly, with some metrics approaching degradation levels. This is because the LFM-2K dataset has higher sparsity and a lower average number of user interactions, resulting in extremely limited effective supervision signals available for training on the client-side. Under these circumstances, traditional federated recommendation methods that rely on independent modeling of local interaction data struggle to learn stable user preference representations, leading to significant performance degradation.

[0126] Table 4. Recommended performance comparison results

[0127] method ML-100K ML-1M LFM-2K HR-2K C²G-FedRec 0.73910.4432 0.71620.4263 0.76850.5806 0.68580.4131 PFedRec 0.73640.4292 0.73440.4461 0.77000.6017 0.71040.4312 GPFedRec 0.57900.3266 0.66390.3951 0.74880.5767 0.58590.2954 CoLR-FedRec 0.65540.3706 0.69250.4117 0.82900.6028 0.66350.4066 FNCF-MAB 0.66700.3876 0.67000.3951 0.76160.5854 0.67160.4127 FedNCF 0.46810.2412 0.46360.2668 0.09570.0524 0.67770.4134 TopK-FedNCF 0.30850.1632 0.30460.1856 0.09380.0531 0.44080.2521 EF-TopK-FedNCF 0.47920.2350 0.43540.2497 0.09620.0557 0.62090.3459 Quantized-FedNCF 0.46640.2408 0.45930.2644 0.09330.0517 0.65670.4132

[0128] Client engagement rate analysis experiment

[0129] Table 4 presents the client engagement rate analysis results. To analyze the impact of changes in client engagement rate on model performance, the experimental effects were examined at engagement rates of 0.1, 0.2, and 0.5 (corresponding to 800, 500, and 200 communication rounds, respectively), while keeping other settings constant. Overall, C²G-FedRec maintains relatively stable performance under different engagement rate settings, and HR@10 and NDCG@10 on each dataset did not show significant degradation, indicating that the proposed method has good robustness and adaptability to changes in client engagement rate. Specifically, the optimal engagement rate is not entirely consistent across different datasets: ML-100K performs best at an engagement rate of 0.1, while ML-1M, LFM-2K, and HR-2K achieve better results at higher engagement rates. A significant reason for this phenomenon may be related to differences in dataset sparsity. Compared to ML-100K, the other datasets have higher overall sparsity, and the local interaction information available to a single client is more limited. Therefore, with a higher participation rate, a single training round can cover more clients and introduce richer collaborative signals, which is more conducive to alleviating the representation learning difficulties caused by data sparsity and improving the final recommendation performance.

[0130] Table 5. Client Participation Rate Analysis Results

[0131] Participation rate ML-100K ML-1M LFM-2K HR-2K 0.1 0.73910.4432 0.71620.4263 0.76850.5806 0.68580.4131 0.2 0.71580.4176 0.72950.4436 0.77180.5931 0.69240.4150 0.5 0.71260.4322 0.72430.4347 0.76920.5872 0.68720.4127

[0132] ablation experiment

[0133] Table 6 reports the ablation experiment results of our algorithm on the ML-100K dataset, used to systematically analyze the impact of key designs in the method of this invention on the final performance. The experiments mainly focus on three aspects: clustering strategy, intra-cluster aggregation strategy, and communication compression strategy.

[0134] First, regarding clustering strategies, three settings were compared: "adaptive (maximum 10 clusters)," "fixed 10 clusters," and "no clustering." "Adaptive (maximum 10 clusters)" employs the discrete-driven dynamic splitting mechanism proposed in this invention, with an upper limit on the number of clusters; "fixed 10 clusters" maintains the cluster structure unchanged during training; and "no clustering" degenerates into global aggregation of single clusters. The results show that adaptive clustering achieves optimal or near-optimal performance under all aggregation strategies, significantly outperforming the fixed cluster and no clustering settings. This indicates that client heterogeneity has a significant impact on federated recommendation, and dynamically adjusting the cluster structure can effectively improve intra-cluster homogeneity, thereby reducing ineffective aggregation and improving model performance.

[0135] Secondly, regarding intra-cluster aggregation strategies, three methods—"glomp_lite," "avg," and "mp"—were compared. "glomp_lite" is the lightweight graph-enhanced aggregation strategy adopted in this invention, "avg" is simple average aggregation, and "mp" is a variant of one-hop message-passing aggregation. Experimental results show that glomp_lite outperforms avg and mp under various clustering settings. This indicates that introducing lightweight graph propagation at the cluster level can effectively mine high-order collaborative information and, compared to simple average or shallow propagation, can more fully utilize the structural relationships between similar clients, thereby improving recommendation performance.

[0136] Regarding communication compression strategies, two schemes, "coordinate-level Top-k" and "row-level Top-k," were compared. Under the condition of roughly equivalent communication budgets, coordinate-level Top-k significantly outperformed row-level Top-k. This indicates that in item embedding difference scenarios, fine-grained coordinate-level sparse selection can more accurately preserve key update information, while row-based coarse-grained selection may lose important local changes, thus affecting model convergence. In the table, the first row, "Adaptive (maximum 10 clusters); glomp_lite; coordinate-level Top-k," corresponds to the default configuration of the proposed method.

[0137] The first row in the table, “Adaptive (maximum 10 clusters); glomp_lite; coordinate-level Top-k”, corresponds to the default configuration of the method of this invention. It shows a stable advantage in all comparisons, verifying the effectiveness of the collaborative design of each module.

[0138] Table 6 Summary of Ablation Experiment Results

[0139] Clustering strategy Aggregation strategy Sparse strategy Optimal performance HR10 / NDCG10 @ round Adaptive glomp_lite Top-k coordinates 0.7391 / 0.4432 @ 684 Adaptive glomp_lite Top-k rows 0.7164 / 0.4261 @ 766 Adaptive avg Top-k coordinates 0.7259 / 0.4286 @ 739 Adaptive mp Top-k coordinates 0.6910 / 0.3931 @ 538 Fixed 10 clusters avg Top-k coordinates 0.7206 / 0.4340 @ 760 Fixed 10 clusters glomp_lite Top-k coordinates 0.7228 / 0.4260 @ 653 Fixed 10 clusters mp Top-k coordinates 0.6963 / 0.4067 @ 741 No clustering avg Top-k coordinates 0.7185 / 0.4230 @ 707 No clustering glomp_lite Top-k coordinates 0.7195 / 0.4293 @ 614 No clustering mp Top-k coordinates 0.7111 / 0.4258 @ 619

[0140] To further analyze the role of the splitting mechanism in adaptive clustering, Table 7 presents comparative experiments on different splitting metrics. Specifically, it compares two splitting criteria: one based on intra-cluster dispersion and the other based on cosine variance. The results show that the splitting strategy based on dispersion achieves superior performance. This is mainly because dispersion directly characterizes the distribution range of signatures within a cluster, more stably reflecting the structural differences between clients. In contrast, cosine variance focuses more on directional consistency and is easily affected by local perturbations in high-noise or highly heterogeneous scenarios, leading to unstable splitting decisions.

[0141] Table 7 Summary of experimental results for splitting index

[0142] Split Criteria Optimal performance HR10 / NDCG10 @ round Dispersion 0.7391 / 0.4432 @ 684 cosine variance 0.7232 / 0.4234 @ 645

[0143] Overall, the results in Tables 6 and 7 together demonstrate that communication compression, adaptive clustering, and cluster-level graph aggregation in the method of this invention are not independent components, but rather work synergistically within a unified framework: communication compression determines the information representation method, adaptive clustering defines the scope of collaborative modeling, and graph aggregation improves the representation quality within a local scope. The synergistic design of these three components is key to the performance advantage of the method of this invention.

[0144] Upload sparsity analysis experiment

[0145] Table 8 presents the experimental results of our algorithm on the ML-100K dataset for upload sparsity analysis, used to analyze the correlation between recommendation accuracy and communication budget. The results show that when the Top-k budget is increased from 256 to 2048, the recommendation accuracy is significantly improved, indicating that moderately increasing the upload sparsity budget in the current task helps to retain more sufficient effective update information; at the same time, the setting of 2048 still maintains low upload complexity, thus forming a more reasonable balance between performance and communication cost.

[0146] Table 8 Summary of Communication Trade-off Experiment Results

[0147] Communications Budget Optimal performance HR10 / NDCG10 @ round Top-k2048 0.7391 / 0.4432 @ 684 Top-k1024 0.7200 / 0.4174 @ 649 Top-k512 0.7147 / 0.4140 @ 763

[0148] Upload communication overhead comparison and analysis experiment

[0149] The uploaded content of the method of this invention consists of two parts:

[0150] 1) Coordinate-level Top-k compressed difference (index and value), approximately [size missing] ;

[0151] 2) Low-dimensional privacy signatures, approximately [size missing] .

[0152] Therefore, the single-client, single-round uplink complexity can be written as: Correspondingly, if the complete item is directly uploaded using embedding or dense difference, the single-client, single-round uplink complexity is... .when and Compared to the full upload method, the method of this invention can significantly reduce the uplink communication overhead of a single client. Table 8 compares the communication costs of each method on the ML-100K dataset from the perspectives of single-round upload complexity and cumulative upload volume when achieving optimal accuracy. Considering that the accuracy of FedNCF and its variants is significantly lower under the current settings, only the top five more representative methods are retained in the table. The results show that C²G-FedRec maintains superior recommendation performance while its communication cost is significantly lower than the comparison methods: the single-round upload overhead is only 24,640B, which is about 88.56% lower than PFEdRec; the cumulative upload volume when achieving optimal accuracy is only 16.10MiB, which is about 50.06% lower than PFEdRec; compared with the federated recommendation algorithm CoLR-FedRec, which is optimized for transmission overhead, the method of this invention reduces the total communication overhead by about 83.74%, while improving HR@10 and NDCG@10 by about 12.77% and 19.59%, respectively. This demonstrates that the advantages of the method of the present invention are reflected in both the final recommendation accuracy and the communication cost required to achieve that accuracy.

[0153] Table 9 Comparison of Upload Communication Overhead

[0154] method Single-round upload Optimal performance @ round Total uploads C²G-FedRec 24,640 B 0.7391 / 0.4432 @684 16.10 MiB PFedRec 215,296 B 0.7364 / 0.4292 @156 32.24 MiB GPFedRec 215,296 B 0.5790 / 0.3266 @500 102.87 MiB CoLR-FedRec 228,752 B 0.6554 / 0.3706 @453 99.04 MiB FNCF-MAB 738,692 B 0.6670 / 0.3876 @503 355.05 MiB

[0155] Noise sensitivity analysis experiment

[0156] Table 10 presents our experimental results on the noise sensitivity analysis of the ML-100K dataset. This invention evaluates the impact of privacy perturbations on model performance by controlling the Gaussian noise intensity during the signature privacy processing stage. Specifically, after completing the random projection and L2 pruning of the signature, the independent injection of each dimension of the signature has a mean of 0 and a standard deviation of [missing value]. Gaussian noise. By setting... This study analyzes the impact of different noise intensities on clustering quality and final recommendation performance. The results show that as noise intensity increases, the overall model performance exhibits a slight downward trend, but... The time-varying performance changes are relatively limited. This indicates that the proposed method is robust to signature perturbations to a certain extent. This is because the signature is only used to characterize the client update direction and participate in clustering, not directly in parameter aggregation; therefore, moderate noise will not significantly disrupt the model training process. Furthermore, when the noise intensity increases to... At higher frequencies, performance fluctuates more significantly, primarily because excessive noise interferes with the similarity measurement between clients, thus affecting clustering quality and further weakening the intra-cluster collaborative modeling effect. Overall, this experiment demonstrates that the method of this invention maintains relatively stable recommendation performance while ensuring a certain level of privacy perturbation, achieving a reasonable trade-off between privacy protection and model performance.

[0157] Table 10 Summary of Noise Sensitivity Experiment Results

[0158] Noise level Optimal performance HR10 / NDCG10 @ round 0 0.7391 / 0.4432 @ 684 0.01 0.7328 / 0.4264 @ 638 0.05 0.7306 / 0.4313 @ 650 0.1 0.7275 / 0.4400 @ 765

[0159] This invention addresses the challenges of high communication overhead, strong client heterogeneity, and the introduction of cross-group noise during global relationship propagation in federated recommendations. It proposes a communication-efficient adaptive cluster-level graph-enhanced federated recommendation method, C²G-FedRec. This method reduces uplink communication costs through coordinate-level sparse uploading and error feedback, mitigates the impact of heterogeneity by combining signature-driven adaptive clustering and intra-cluster GloMP-lite graph aggregation, and maintains cross-round personalized states using cache-enhanced piecewise linear hybrid delivery. Experiments on multiple datasets, along with analyses of ablation, participation rate, compression mode, splitting criteria, noise sensitivity, and communication overhead, demonstrate that this method can balance recommendation accuracy, convergence performance, and uploading efficiency even under low participation rate conditions. Future work will further expand upon this method with more stringent privacy attack evaluations and robustness analyses.

[0160] The technical solutions of the present invention are not limited to the specific embodiments described above. Any technical modifications made in accordance with the technical solutions of the present invention fall within the protection scope of the present invention.

Claims

1. A communication-efficient adaptive cluster-level graph-enhanced federated recommendation method, characterized in that, Includes the following steps: Step S1: The client performs Top-k sparse compression on the embedding difference and combines it with an error feedback mechanism to compensate for the information not transmitted in each round. At the same time, the client extracts a low-dimensional signature representation based on the embedding difference and performs privacy processing through random projection, norm pruning and noise injection operations for subsequent heterogeneity-aware modeling. Step S2: Heterogeneous load modeling: The server characterizes the similarity between clients based on the privacy signatures uploaded by the clients, and performs adaptive clustering and constructs a client relationship graph accordingly; Step S3: Cluster-level graph aggregation and distribution: After obtaining the client cluster structure, the server constructs a relationship graph within the cluster and performs lightweight graph propagation to mine local high-order collaborative signals within similar user groups and suppress cross-group noise propagation. The server combines cluster-level representation, global representation and user history cache to generate the item embedding for the next round of distribution, thereby achieving a dynamic balance between globally shared information and personalized preference expression.

2. The communication-efficient adaptive cluster-level graph-enhanced federated recommendation method according to claim 1, characterized in that: Step S1 includes the following sub-steps: Step S1.1: The client updates the embedded difference definition as follows: In the In the round, the client Receive item embedding from server The updated item embeddings are obtained after local training. ; Step S1.2: Incorporate the information not transmitted in the previous round into the current update to obtain the corrected update amount: Subsequently, Perform coordinate-level Top-k compression, retaining the largest absolute value. Each coordinate is used, and the compressed result is denoted as... ,remember The client-side error feedback residual is updated to correspond to the reconstruction update based on the compression result. ; Step S1.3: Perform mean pooling on the difference matrix along the item dimension, defining the original signature as follows. in, The signature construction operator performs mean pooling on the item dimension to extract a compact directional representation from the high-dimensional embedding difference; privacy processing is then performed on the original signature to obtain the privacy signature. in, Indicates random projection, The privacy processing flow consists of norm clipping and Gaussian noise injection in sequence.

3. The communication-efficient adaptive cluster-level graph-enhanced federated recommendation method according to claim 1, characterized in that: In step S2, the server characterizes the preference similarity among clients based on the privacy signatures uploaded by the clients, and constructs structured client groups through adaptive clustering and intra-cluster graph relationship modeling. After obtaining cluster assignments, the first... Round Each cluster is Its mean signature is defined as: Intra-cluster dispersion is defined as: When the constraints of minimum cluster size and maximum cluster number are satisfied, and When the threshold is exceeded, the server performs a binary search operation on the corresponding cluster; For the Each cluster of the wheel The server first decompresses the compressed update package uploaded by the client to obtain the reconstructed update. And restore the client's local embedded representation: The recovered embedded representations of each client within the cluster are flattened into vector form, and the pairwise relationship score matrix between clients is calculated accordingly.

4. The communication-efficient adaptive cluster-level graph-enhanced federated recommendation method according to claim 3, characterized in that: The server takes the item embedding vector restored by the client as input and calculates the pairwise relationship score between clients within the cluster. Subsequently, a threshold-based neighborhood filtering strategy was used to construct the intra-cluster relation matrix. For each client, only local neighbor connections with relation scores higher than a given threshold were retained; for the retained neighbor edges, a uniform weighting was applied based on the number of neighbors, ultimately yielding the intra-cluster relation matrix used for graph propagation. .

5. The communication-efficient adaptive cluster-level graph-enhanced federated recommendation method according to claim 1, characterized in that: In step S3, after obtaining the intra-cluster relationship graph, the server performs [further processing] in each cluster. Lightweight graph propagation is implemented internally, and records are kept. Embedding the row-stacked representation matrix into the intra-cluster clients, the two-layer graph propagation can be written as: Based on the two-hop propagation results, the cluster-level intermediate representation is first defined as follows: in To balance intra-cluster graph propagation information with the original intra-cluster representation, the final cluster-level embedding is defined as: in Control the fusion strength between cluster-level representations and globally shared representations; In obtaining the aggregate embeddings of each cluster Afterwards, the server updates the global item embedding, which is obtained by weighting the embeddings of each cluster by cluster size: The server maintains a user-level cache embedding for each client. For clients participating in the current round of training Its cache is updated to the aggregate embedding of the corresponding cluster: in Indicates client In the Cluster allocation in the round, for clients that did not participate in this round of training. Its cache remains unchanged: In the next round, the server executes a cluster-aware distribution strategy based on cached information, for clients that already have cached data. Its embedded definition is: For users who do not yet have a cache, global embedding will be used directly. As the baseline for distribution, a piecewise linear mixing coefficient is introduced. Let the total number of rounds be... The update interval is ,but Only for Update when the value is an integer multiple of the given value, and satisfy the following conditions: exist stage, Gradually increase; when hour, It remains unchanged.

Images