Federal fair recommendation method based on embedding decoupling and adaptive aggregation

Abstract

The application discloses a federated fair recommendation method based on embedded decoupling and adaptive aggregation, which comprises the following steps: inputting a user-item interaction matrix extracted from a local into a multi-layer graph convolution network GNN to output a user embedding matrix and an item embedding matrix; inputting each user embedding into two mapping functions composed of a multi-layer perceptron MLP to output a user sensitive attribute related embedding and a user interest related embedding; fusing the user sensitive attribute related embedding and the user interest related embedding to form a user representation, estimating a score of a user u to an item i, and recommending an item with the highest score to the user u. A fair representation learning model based on double projection head embedded decoupling realizes effective separation of sensitive attributes and interest features through orthogonal constraint and discriminator collaborative training, and based on the above decoupled embedding, a sensitive-interest embedding fusion mechanism is proposed to fully mine beneficial information in the sensitive attribute embedding under the premise of ensuring fairness.

Description

[0001] This invention belongs to the field of federated learning technology, and more specifically, it relates to a federated fair recommendation method based on embedding decoupling and adaptive aggregation. Background Technology

[0002] In this era of information overload, recommender systems have become a crucial bridge connecting users and information, becoming indispensable in various online platforms such as e-commerce, social media, and streaming services. By learning the potential relationships between users and items, recommender systems can effectively capture users' interests and preferences, thereby providing personalized services.

[0003] Federated recommender systems introduce the federated learning paradigm into recommendation scenarios. By training models locally on distributed clients and aggregating parameters on the server side, they achieve collaborative modeling without sharing the original user data. However, different clients often have varying amounts of data, data distributions, device capabilities, and participation frequencies. This heterogeneity can lead to weaker model performance on some clients, raising fairness issues. Since recommendations may involve the allocation of social resources, such as job recommendations and medical resource allocation, unfair recommendations not only harm user interests but also damage the credibility of the recommender system itself. Therefore, fairness issues in recommender systems have attracted increasing attention. Summary of the Invention

[0004] This invention provides a federated fair recommendation method based on embedding decoupling and adaptive aggregation, aiming to improve at least one of the above-mentioned problems.

[0005] This invention is implemented as follows: a federated fair recommendation method based on embedding decoupling and adaptive aggregation, the method being as follows: (1) Extract the user-project interaction matrix from the local machine User-project interaction matrix Input a multi-layer graph convolutional network (GNN), and the GNN outputs a user embedding matrix. and project embedding matrix ; (2) Embedding users into the matrix Each user embedded in The inputs are two mapping functions composed of a multilayer perceptron (MLP), and the outputs of the two mapping functions are user embeddings. Embedding related to corresponding user-sensitive attributes And user interest-related embeddings ; (3) Embedding user-sensitive attributes Embedding related to user interests To integrate and form a user representation Estimate user u's rating of item i The highest-rated items will be recommended to user u.

[0006] Furthermore, after the client completes the project recommendation, the following steps are performed: (4) Embedding user-sensitive attributes Input predictor, predictor predicts embeddings related to user-sensitive attributes User-sensitive attribute vectors Embedding user interests Input discriminator, output user interest-related embeddings User-sensitive attribute vectors ; (5) Calculate the total loss of the client's local model based on the attribute prediction loss of the predictor and discriminator. Based on total loss The local model parameters are updated using gradients, and the updated local model parameters are... Update the local model parameters and the attribute prediction loss of the discriminator Uploaded to the server; (6) The server predicts the loss based on the discriminator's attributes. As a measure of fairness Calculate the aggregation weight of each user participating in the aggregation. Based on aggregate weight Local model parameters uploaded by each user Aggregate the data to form the global model parameters for this round's global model. The data is then distributed to all participating clients for updating local model parameters.

[0007] Furthermore, the total loss of client u The specific calculation formula is as follows: ; in, This represents the weighting coefficient of the attribute prediction loss for client u in the current round. This indicates the set weighting coefficient. Represents the user-sensitive attribute vector of client u The attribute prediction loss; Represents the user-sensitive attribute vector of client u The attribute prediction loss; Embeddings related to user-sensitive attributes of client u Embedding related to user interests orthogonal loss between them This represents the Bayesian personalized ranking loss for client u.

[0008] Furthermore, the method also includes: (7) Calculate the direction of gradient change of the global model Calculate the gradient feedback coefficients that will participate in the next round of local model fine-tuning. And send it to the client; (8) Based on local model parameters The update direction involves K-means clustering of the user, with the user located in the k-th cluster... C k Local model parameters for all users Calculate its position in the cluster C k Inner Aggregation Weight Based on aggregate weight For local model parameters Perform aggregation to form aggregation parameters Aggregation parameters Distributed to clusters C k Internal clients; (9) The client will transfer global model parameters As initial parameters of the local model Based on gradient feedback coefficient Initial parameters for the local model Fine-tune the parameters to obtain the adjusted local model parameters. : (10) Client updates local model parameters Then, proceed with the next round of training.

[0009] Furthermore, the gradient feedback coefficient The specific calculation formula is as follows: ; in, For gradient feedback intensity coefficient, To prevent stable terms with a denominator of zero.

[0010] Furthermore, the server updates the attribute prediction loss of the discriminator. In the next round of weighting coefficients And distribute it to all clients participating in the aggregation, with weighting coefficients. The specific calculation formula is as follows: ; in, c It is expressed as the amplification constant.

[0011] Furthermore, aggregate weights The specific calculation formula is as follows: ; in, This represents the deviation weighting adjustment coefficient.

[0012] Furthermore, embedding related to user-sensitive attributes Embedding related to user interests The integration method is as follows: ; in, This indicates the fusion weight.

[0013] Furthermore, aggregate model parameters Specifically as follows: ; in, This indicates that user u is in the kth cluster. C k Aggregate weights within.

[0014] Based on the fair representation learning model with decoupled dual-projector embedding, sensitive attributes and interest features are effectively separated through orthogonal constraints and discriminator collaborative training. Based on the above decoupled embedding, a sensitive-interest embedding fusion mechanism is proposed, which fully explores the beneficial information in the sensitive attribute embedding while ensuring fairness, thereby improving the fairness and recommendation quality of the model. Attached Figure Description

[0015] Figure 1 A flowchart of a federated fair recommendation method based on embedding decoupling and adaptive aggregation provided in an embodiment of the present invention; Figure 2 The present invention provides an ablation experiment on ML-1M data based on gender attributes for each module, wherein (a) is a recommended performance index and (b) is a fairness index; Figure 3 The ablation experiment of each module based on gender attribute on ML-100K data provided in the embodiments of the present invention, wherein (a) is the recommended performance index and (b) is the fairness index; Figure 4 The following are comparative analysis charts of the convergence curves of various models provided in the embodiments of the present invention on the ML-100K dataset, wherein (a) is the HR@10 index chart and (b) is the Diff HR@10 index chart; Figure 5The following are performance graphs of different bitach sizes on the datasets ml-100k and ml-1m provided in the embodiments of the present invention, wherein (a) is the hr performance on ml-100k, (b) is the ndcg performance on ml-100k, (c) is the hr performance on ml-1m, and (d) is the ndcg performance on ml-1m. Figure 6 Parameters provided for embodiments of the present invention Impact on the ML-100k dataset, where (a) shows the impact on performance and fairness of HR@10, and (b) shows the impact on performance and fairness of NDCG@10; Figure 7 Parameters provided for embodiments of the present invention Impact on the ML-100k dataset, where (a) shows the impact on performance and fairness of HR@10, and (b) shows the impact on performance and fairness of NDCG@10; Figure 8 Parameters provided for embodiments of the present invention Impact on the ML-100k dataset, where (a) shows the impact on performance and fairness of HR@10, and (b) shows the impact on performance and fairness of NDCG@10; Figure 9 Parameters provided for embodiments of the present invention k Impact on the ML-100k dataset, where (a) shows the impact on performance and fairness of HR@10, and (b) shows the impact on performance and fairness of NDCG@10; Figure 10 The impact of the number of MLP layers in the projection head provided in the embodiments of the present invention is shown in (a) on the performance and fairness of HR@10, and (b) on the performance and fairness of NDCG@10. Detailed Implementation

[0016] The specific embodiments of the present invention will be further described in detail below with reference to the accompanying drawings, so as to help those skilled in the art to have a more complete, accurate and in-depth understanding of the inventive concept and technical solution of the present invention.

[0017] Each client utilizes local user-item interaction data to extract original user and item embeddings via a graph neural network. Subsequently, a dual-projection head structure decouples the user embeddings into two independent subspaces: sensitive attribute-related embeddings and interest-related embeddings. To force the interest embeddings to remove sensitive information, an adversarial learning mechanism is introduced: on one hand, the sensitive attribute embeddings are supervised through a predictor to strengthen their encoding ability for sensitive attributes; on the other hand, the interest embeddings are adversarially trained through a discriminator combined with a gradient reversal layer to eliminate the sensitive attribute information they contain. The decoupled embeddings are then fed into the local model to generate personalized recommendation lists for users, and the recommendation loss and discriminator loss are calculated simultaneously.

[0018] After each round of local training, the client uploads the updated model parameters and discriminator loss to the server. The server first uses the discriminator loss as a bias score to calculate the aggregation weights, assigning higher weights to clients with better fairness performance to guide the global model towards them. Subsequently, the server aggregates the parameters from each client to generate the global model. To further improve model accuracy and personalization, a dynamic clustering mechanism based on client update direction is introduced. The server analyzes the parameter update vectors of each client to identify groups of clients with similar data distribution characteristics and divides them into multiple clusters. Each cluster generates a corresponding cluster center model, providing more accurate personalized services to user groups with similar characteristics. Figure 1 The flowchart of the federated fair recommendation method based on embedding decoupling and adaptive aggregation provided in the embodiments of the present invention is as follows: (1) Extract the user-item interaction matrix R from the local machine, input the user-item interaction matrix R into the multilayer graph convolutional network GNN, and the multilayer graph convolutional network GNN outputs the user embedding matrix. and project embedding matrix ; Add the users participating in the current round of federated aggregation to the user set, denoted as . ,in Let n represent the nth user in the user set, and let n represent a set of items containing all items. , Let m be the m-th item in the item set, and let R be the user-item interaction matrix representing the user ratings for all items. ,in, Indicates user For the project The observed signal, if the user For the project If there is a rating, then If the user For the project No rating, then .

[0019] Trainable embedding vectors are initialized for all users and items; these embeddings are then propagated through a multi-layer graph convolutional network to explicitly encode high-order connectivity between nodes; finally, the propagation results from each layer are fused to obtain the final augmented representation for downstream prediction tasks. Specifically, it is assumed that there exists... n individual users and m For each item, the embedding dimension is d. A multi-layer graph convolutional network (GNN) constructs trainable embedding matrices for both users and items, denoted as the user embedding matrix and the item embedding matrix, respectively. Project Embedding Matrix During the forward propagation phase, the multi-layer graph convolutional network (GNN) first concatenates the user embedding matrix and the item embedding matrix into a global node embedding matrix. This is then used as input and encoded through multiple layers of a graph neural network. Let the normalized adjacency matrix be... Then the first The node embedding of a layer is represented as By stacking multiple propagation layers to capture higher-order neighborhood information, and averaging the outputs of each layer, the final embedding matrix is ​​obtained. From the final embedding matrix E The user and item components are extracted separately to obtain the user embedding matrix after graph network enhancement. Embedded with items The following text refers to the embedding of a single user as A single project is embedded as a record .

[0020] (2) Embedding users into the matrix Each user embedding segment The input consists of two mapping functions composed of a multilayer perceptron (MLP), and the output of the two mapping functions is the user embedding. Embedding related to corresponding user-sensitive attributes And user interest-related embeddings ; In the fair recommendation model of this invention, the core issue is how to obtain unbiased user embeddings to achieve unbiased recommendation. However, the original user embeddings extracted from the user-item interaction matrix R by a multi-layer graph convolutional network (GNN) are problematic. The embedding contains attribute-sensitive information that can interfere with the model and cause biased predictions. To remove this potential interference, the original embedding is decoupled into two parts: sensitive attribute-related embedding and interest embedding. Specifically, a dual-projection head structure is first introduced to decompose the user embedding into two subspace representations:

[0021] in, This indicates embeddings related to sensitive attributes. This indicates embeddings related to user interests. and All are mapping functions composed of multilayer perceptrons (MLPs).

[0022] (3) Embedding user-sensitive attributes Input predictor, predictor predicts embeddings related to user-sensitive attributes User-sensitive attribute vectors Embedding user interests Input discriminator, output user interest-related embeddings User-sensitive attribute vectors ; The projector serves to provide two independent mapping spaces for learning user attribute-related features and interest-related features, respectively. To ensure effective decoupling of the two subspaces, a dual optimization mechanism is designed: First, embeddings related to user-sensitive attributes are introduced into the predictor. Supervised training is conducted to enable it to accurately predict users' sensitive attributes, thereby explicitly strengthening the subspace's ability to express sensitive information; simultaneously, a discriminator is introduced to analyze user interest-related embeddings. This method infers sensitive attribute labels by imposing adversarial loss during training and using a gradient reversal layer to propagate the backward gradient to the encoder, thereby forcing the encoder to learn interest representations that are insensitive to sensitive attributes. Specifically, a predictor is introduced to attempt to extract user-sensitive attribute-related embeddings. Predicting sensitive attributes and obtaining prediction results , means as follows: ; in, These represent the weight parameters and bias parameters, respectively. Represents the normalized exponential function, This represents the sensitive attribute vector predicted by the predictor.

[0023] Relying solely on the predictor cannot guarantee the decoupling of the original user embedding space. Therefore, an adversarial mechanism is introduced, implementing adversarial training through a discriminator and a gradient inversion layer. The discriminator attempts to train the user's interest-related embeddings. Predicting sensitive attributes and obtaining prediction results , means as follows: ; in, These represent the weight parameters and bias parameters, respectively. This represents the sensitive attribute vector predicted by the discriminator.

[0024] (4) Calculate the user-sensitive attribute vector Attribute prediction loss and user-sensitive attribute vectors Confrontational losses ; In this embodiment of the invention, attribute prediction loss Loss is expressed using cross-entropy, as follows: ; Adversarial loss Similar to a predictor, it is represented as follows: .

[0025] (5) Calculate the embeddings related to user-sensitive attributes Embedding related to user interests orthogonal loss between ; During training, the gradient inversion layer inverts the gradients during backpropagation, thereby penalizing the main model with negative gradients and making the user interest-related embeddings more sensitive. Reduce the encoding of sensitive attributes to achieve unbiased representation learning. Unfortunately, user interest-related embeddings... Some information related to sensitive user attributes may still remain. This is because, in practice, discriminators often cannot perfectly infer a user's sensitive attributes, and there may be a shift between the discriminator's decision boundary and the true distribution of sensitive user attributes, resulting in incomplete elimination of sensitive information. To further evolve the user interest embedding, an orthogonal regularization strategy is introduced. Specifically, this method constrains the embeddings related to user sensitive attributes. Embedding related to user interests They are mutually orthogonal in the embedding space, which geometrically weakens the correlation between them; this is the orthogonality loss. It is expressed as follows: ; in, Representing users respectively u The embeddings are related to user-sensitive attributes and user-interest-related attributes. This constraint makes the sensitive subspace and interest subspace as orthogonal as possible in the vector space, reduces information overlap, and further weakens the interference of sensitive attributes on recommendation preferences from a structural level.

[0026] (6) Calculate the Bayesian personalized ranking loss The specific calculation formula is as follows: ; in, Represents the project embedding matrix. The weighting coefficients representing interest attributes. This represents the Bayesian personalized ranking loss.

[0027] (7) Taking into account the constraints of recommendation accuracy, fairness and consistency, calculate the total loss of the client's local model. Based on total loss The total loss of client u is calculated by updating the gradients of the local model parameters. The specific calculation formula is as follows: ; in, This represents the weighting coefficient of the attribute prediction loss in the current round. This represents the set weight coefficient. The optimization objective is continuously adjusted through an adaptive feedback mechanism, enabling the client to gradually approach a fair and stable global optimal solution while maintaining personalized performance.

[0028] (8) Client based on total loss Updated local model parameters Put local model parameters and attribute prediction loss Uploaded to the server.

[0029] (9) The server predicts the loss based on the attributes uploaded by user u. As its fairness metric Calculate the aggregation weight of each user participating in the aggregation. Based on aggregate weight Local model parameters uploaded by each user Aggregate the data to form the global model parameters for this round's global model. Distribute to all clients participating in the aggregation; In traditional federated learning recommendation frameworks, servers typically employ equal-weighted or sample-weighted aggregation strategies for model parameters uploaded by clients. However, in federated recommendation environments, client data distribution is heterogeneous, and the degree of bias information contained in the local models of different clients often varies. Simply performing mean aggregation may lead to high-biased clients having an unreasonable influence on the global model, thereby impairing the model's fairness and generalization ability. To address this, this invention proposes an adaptive weighted federated optimization mechanism based on adversarial bias. The core idea is to use the local adversarial loss of each client... As a deviation estimation index B This is used to determine the aggregation weights, thereby adaptively suppressing bias propagation at the global level while guiding the local model to optimize in a fair direction.

[0030] In each round of training, each participating client includes a discriminator used to detect potential sensitive attribute information in its user interest embedding, thus classifying the user... u The adversarial cross-entropy loss without gradient inversion layer is denoted as... and use it as a measure of fairness. Larger fairness metrics A higher bias indicates that the client's user interest embedding retains less sensitive attribute information. The server first collects the bias scores of all participating clients. Then, its adaptive aggregation weights are calculated, and the softmax function is used to assign higher weights to clients with high bias, and truncation is used to prevent extreme values: Aggregation Weights The specific calculation formula is as follows: ; in, This represents the deviation weighting adjustment coefficient. This indicates the minimum and maximum weights corresponding to the weight cutoff boundary.

[0031] In this embodiment of the invention, global model parameters The specific calculation formula is as follows: ; (10) Considering the significant differences in update directions among different clients, to avoid global model divergence, the server simultaneously generates gradient feedback signals. Let the direction of the global model gradient change in this round be: Calculate the gradient feedback coefficients involved in the local model fine-tuning. And send the gradient feedback coefficients to the client. The specific calculation formula is as follows: ; in, This is the gradient feedback strength coefficient. To prevent stable terms with a denominator of zero.

[0032] (11) Based on local model parameters The update direction is to perform K-means clustering on the user, for the user located in the k-th cluster... C k Local model parameters for all users Calculate its position in the cluster C k Inner Aggregation Weight Based on aggregate weight For local model parameters Perform aggregation to form aggregation parameters Aggregation parameters Distributed to clustersC k The client within.

[0033] To further improve the adaptability and accuracy of the recommendations, clustering is performed based on the client's update direction. The server calculates the update direction based on the model parameters uploaded by the client and performs K-means clustering based on the similarity between gradient vectors. During the training round, the local model parameters uploaded by client u are: The server saves the global model parameters from the previous round. The update direction of client u is Based on the calculated gradient vectors, the server executes the K-means clustering algorithm to divide users into K clusters. The motivation for this is that clients within the same cluster have similar update directions during training. For each cluster C, the server aggregates the local models of all users within that cluster to obtain the corresponding aggregation parameters. The specific calculation formula is as follows: ; in, This indicates that user u is in the kth cluster. C k The aggregation weights within the model are calculated using the same method as the aggregation weights used when a global model is aggregated.

[0034] (12) Calculate attribute prediction loss In the next round of weighting coefficients And distribute it to all clients participating in the aggregation; The smaller the bias, meaning the discriminator can more easily predict clients with sensitive attributes and whose embeddings still contain a lot of sensitive information, the higher the adversarial strength coefficient will be assigned to them in the next round of training to enhance the bias reduction drive. The discriminator's adversarial loss is still defined as the bias, and its adversarial coefficient is updated accordingly. The specific update formula is as follows: ; in, c Represented as an amplification constant, it controls the intensity of bias feedback. This mechanism achieves adaptive local adversarial intensity adjustment: clients with smaller biases gain stronger adversarial learning motivation, thereby gradually removing sensitive information from interest embeddings in subsequent rounds.

[0035] (13) The client receives the message sent by the server. Gradient feedback coefficient and aggregation parameters The client will set global model parameters As initial parameters of the local model Based on gradient feedback coefficient Initial parameters for the local model Fine-tune the parameters to obtain the adjusted local model parameters. ; (14) The client updates the local model parameters To proceed to the next round of training, in which... This represents the balance coefficient, used to control the fusion ratio of different model levels. This design allows the recommender system to capture global commonalities while preserving the differences between user groups and individuals, achieving hierarchical personalized predictions.

[0036] During the prediction phase, embedding user-sensitive attributes is performed. Embedding related to user interests To integrate and form a user representation Estimate user u's rating of item i The system recommends the highest-rated items to users.

[0037] Two types of embeddings were designed for each user: embeddings related to user-sensitive attributes. It mainly contains information about user-sensitive attributes and interest-related embeddings. This primarily includes user interest information. Although the goal is to eliminate bias caused by sensitive attributes during the recommendation process, in reality, user interest choices often correlate with these attributes. Therefore, completely removing sensitive attribute information may cause the model to lose some signals that contribute to recommendation performance. To address this, the two types of embeddings are fused during the prediction phase to obtain a user representation. It is expressed as follows: ; in, The above method represents the fusion weight. By maintaining fairness, the model can make full use of the understanding of sensitive attributes in the sensitive attribute embedding and the interest information after removing bias in the interest embedding, thereby achieving a fairer and more accurate recommendation effect.

[0038] The federated fair recommendation method based on embedding decoupling and adaptive aggregation provided by this invention has the following beneficial technical effects: Based on a fair representation learning model with dual-projector embedding decoupling, sensitive attributes and interest features are effectively separated through orthogonal constraints and discriminator collaborative training. Based on the above decoupled embedding, a sensitive-interest embedding fusion mechanism is proposed, which fully mines the beneficial information in the sensitive attribute embedding while ensuring fairness, thereby improving the fairness and recommendation quality of the model. A fairness adaptive aggregation and dynamic debiasing mechanism is designed. This mechanism uses the discriminator loss as a quantitative indicator of the client's debiasing degree. The server dynamically allocates aggregation weights accordingly, allowing clients with insufficient debiasing to gain higher influence in global optimization. Simultaneously, the server feeds back the direction of debiasing intensity adjustment to the clients based on the fairness indicators of each client, and synchronously feeds back the global gradient direction, guiding the clients to correct their local update process, thereby systematically improving overall fairness. To balance fairness and personalized performance, a multi-level personalized prediction model based on clustering enhancement is constructed. By fusing the user's local individual model, the group model of its cluster, and the global model, accurate recommendations are achieved through a three-level collaborative approach of "individual-group-global". Experiments on four real-world datasets demonstrate that the proposed model can improve group fairness and the model itself while maintaining recommendation accuracy.

[0039] This section describes the experimental setup in detail, including the dataset, baseline methods, and recommendation evaluation metrics. Then, we present the results of our proposed algorithm, demonstrating its ability to improve both fairness and recommendation performance.

[0040] .

[0041] In the experiments, five publicly available real-world recommendation datasets were introduced: MovieLens-100K (ML-100K), MovieLens-1M (ML-1M), Last.FM-1K, and AliEC. ML-100K and ML-1M are two widely used movie recommendation datasets. The ML-100K dataset includes 943 users, 1682 movies, and 100,000 user-item interactions, while ML-1M includes 6040 users, 3706 movies, and 1,000,209 interactions. The Last.FM.1K dataset contains nearly 1,000 users and over 19 million "listening" relationship records between them and approximately 170,000 artists. Only data from 2008-2009 was retained, and users and items with fewer than 5 interactions were filtered out, resulting in a dataset containing 863 users, 33,753 artists, and 8,033,118 interactions.

[0042] The AliEC dataset is a Taobao display ad click-through rate prediction dataset provided by Alibaba. Users interact with items in four ways: browse (ipv), add to cart (cart), like (fav), and buy (buy). I only selected interactions of cart, fav, and buy, and filtered out users with fewer than 10 interactions. The final dataset contains 147,913 users, 8,486 items, and 2,558,478 interactions.

[0043] During data preprocessing, various types of user interactions were converted into implicit feedback, and duplicate user-item interactions were removed. Furthermore, the dataset was split into training and test sets using leave-one-out method. Three sensitive attributes—gender, age, and occupation—were selected to evaluate customized fairness. Specifically, the gender group was divided into two subgroups (male and female), the age group into four subgroups (under 24, 25-34, 35-49, and over 50), and the occupation group into four subgroups based on occupational relevance and social class. For the ML-1M and ML-100K datasets, we used gender, age, and occupation as sensitive attributes; for the Last.FM.1K and AliEC datasets, we used gender and age as sensitive attributes.

[0044] In the fair federated recommendation task, to fully verify the effectiveness and universality of the proposed method, several representative fair federated learning baseline methods were selected for comparison. These methods embody typical approaches to improving model fairness from different levels. Specifically, the proposed method is compared and analyzed with four methods: CustomFair, F2PGNN, FairFed, and FPFR, covering multi-dimensional fairness improvement mechanisms from the model training layer, the optimization objective layer, to the aggregation policy layer.

[0045] CustomFair: This method achieves a balance between the global and subgroup models by dynamically adjusting the L2 regularization coefficient and adaptively adjusting the number of local training rounds based on the subgroup convergence state, thereby achieving performance balance and fairness among different groups. F2PGNN: By jointly considering utility loss and fairness loss in the global optimization objective, using the negative of the utility loss as the fairness loss, and dynamically adjusting the learning rate of different user groups using global group statistics, this method improves the model performance of disadvantaged groups, achieving group fairness optimization in federated recommendation. FairFed: This method dynamically updates the aggregation weights based on the difference between the local fairness index of each client and the global fairness index, thereby reducing the influence of biased clients. FPFR: By introducing group fairness loss into the global objective function, this method achieves user-oriented group fairness optimization.

[0046] (1) Recommendation Accuracy. The standard ranking metric HR@10 and normalized discount cumulative gain (NDCG@10) were used to evaluate the model's accuracy on the top 10 recommendations. Higher metric values ​​indicate that the recommendation list better reflects the user's true preferences. In the experiments, to avoid calculating the scores of all user-item pairs across the entire item space, a negative sampling strategy was used for evaluation: 100 non-interactive items were randomly sampled for each user as negative samples, and these negative samples were combined with positive samples from the validation or test sets to construct a candidate list. Model performance was obtained by calculating F1@10 and NDCG@10 in the candidate list.

[0047] (2) Recommendation Fairness. In fairness research on recommender systems, from the user's perspective, user fairness can usually be further divided into two categories: individual fairness and group fairness. The former focuses on the consistency of recommendation results among similar users, while the latter focuses on the balance of recommendation effects among different groups divided by sensitive attributes. This study mainly focuses on group fairness. Since user data usually contains sensitive attributes, such as gender and race, the model may exhibit systematic bias in the prediction results. Group fairness aims to compare the recommendation effects among different groups divided by sensitive attributes. S The differences in recommendation performance between different groups are defined to evaluate the fairness of the model.

[0048] In sensitive attributes S For example, gender, which divides users into two groups. and In this context, two-group fairness requires that the recommendation system maintain consistent performance between the two groups. Its fairness metric is defined as: ; in u Representation group S Each user in, H This indicates the recommended performance based on evaluation metrics, such as Normalized Discounted Cumulative Gain (NDCG). , Subgroup Subgroup The number of users. Smaller. diff The value indicates that the model is more equitable between the two groups.

[0049] In a more general case, if users are categorized based on sensitive attributes... m For a single group (e.g., different age groups), the definition of fairness across multiple groups is: ; This metric measures the performance differences among all groups. When m=2, the above formula degenerates into the definition of two-group fairness. diff The smaller the value, the more balanced the system's recommendation performance across different groups. In this experiment,H We use F1@10 and NDCG@10 to simultaneously measure recommendation performance and fairness under a unified evaluation model.

[0050] Hyperparameters were carefully selected to optimize the model for best performance. The training learning rate for all datasets was 0.01. In the experiments, a graph convolutional network was used as the graph neural network model. For all models, the embedding size for users and items was set to 64. The dot product method was used for the rating predictor. In each round of model training, a batch of 128 users was selected, and the hyperparameters were... Set to 1 / 3. Set the value to le-3. The value is set to 0.5. The user batch size is 128. The result is the average of 10 different runs.

[0051] .

[0052] .

[0053] .

[0054] The effectiveness of the algorithm in recommending accuracy and fairness was verified by comparing it with other fairness methods. Experiments were conducted on four datasets: ML-1M, ML-100K, Last.FM.1K, and AliEC, and user groups were segmented based on three different sensitive attributes (gender, age, and occupation). Experimental results are shown in Tables 2, 3, and 4. The overall performance was calculated across the entire test set. HR@10 and NDCG@10 were used to measure recommending performance; higher HR@10 and NDCG@10 values ​​indicate better algorithm performance. The performance difference (diff) between different user groups was calculated; a smaller diff indicates greater fairness in recommendations across different user groups.

[0055] Overall, the federated fair recommendation method based on embedding decoupling and adaptive aggregation (F2EA-PRec) provided in this invention demonstrates superior comprehensive performance in most experimental settings. Regarding recommendation accuracy, F2EA-PRec achieves optimal HR@10 and NDCG@10 metrics on the ML-1M, ML-100K, and AliEC datasets, regardless of whether the sensitive attribute is gender, age, or occupation. Specifically, F2EA-PRec significantly improves the overall performance of the baseline. Taking the ML-1M dataset (gender-split) as an example, F2EA-PRec increases HR@10 from the best value of 60.35% in the baseline method to 64.69%, a relative improvement of 7.2%; on the AliEC dataset (gender-split), our method further pushes HR@10 to 89.40%, fully demonstrating its powerful ability to improve recommendation quality.

[0056] In terms of fairness, F2EA-PRec also achieved significant improvements, effectively narrowing the performance gap between different user groups. Our method achieved minimal inter-group performance differences in several key scenarios. For example, on the Last.FM.1K dataset (gender splitting), F2EA-PRec's HR@10 diff value was as low as 0.13%, and the NDCG@10 diff value was as low as 1.08%. Compared with F2PGNN's results on this dataset, our method reduced inter-group differences by approximately 13% while maintaining a higher HR@10. On the ML-100K dataset (gender splitting), our method also reduced the HR@10 diff to 0.23%, demonstrating its stable effectiveness in promoting user fairness. However, we note that the fairness metrics of the ML-100K dataset fluctuate significantly with different grouping criteria: when using gender as the grouping criterion, the inter-group performance differences are small; while when using age or occupation for grouping, the inter-group differences increase significantly. This phenomenon may stem from the following reasons: age grouping typically involves more categories, and the distribution of user numbers and behavioral patterns often differ significantly across age groups. In moderately sparse datasets like ML-100K, some age groups may experience insufficient local training due to relatively limited interaction data, thus affecting the fairness calibration effect during the global aggregation phase. In contrast, gender grouping is generally binary classification, with a more balanced sample size between groups, which is conducive to the fairness constraint mechanism playing a more stable role.

[0057] Specific data comparisons further confirm our advantages: On the ML-1M dataset (occupational classification), compared to FPFR, F2EA-PRec improved HR@10 from 59.88% to 63.68%, while reducing the diff value from 1.83% to 1.32%, achieving simultaneous optimization of accuracy and fairness. Experimental results show that, under specific experimental settings, F2EA-PRec's fairness metrics are not optimal in all cases. Specifically, in scenarios where age is a sensitive attribute, such as the ML-100K, Last.FM.1K, and AliEC datasets, F2EA-PRec's HR@10 or NDCG@10 inter-group differences are slightly higher than the CustomFair method on some metrics; on the ML-100K dataset where occupation is a sensitive attribute, its diff value is also slightly higher than the FairFed method. This phenomenon may stem from the fundamental differences in the core strategies and optimization objectives adopted by different methods in addressing the "accuracy-fairness tradeoff." Customfair may have sacrificed recommendation accuracy excessively for fairness, while F2EA-PRec achieves a better overall trade-off, validating its effectiveness and practicality as a fair recommendation scheme.

[0058] In summary, the experimental results demonstrate that the proposed F2EA-PRec algorithm not only consistently delivers the highest level of recommendation quality but also effectively promotes fairness among different user groups. Its robustness across multiple datasets and sensitive attributes validates its effectiveness as an efficient and fair recommendation solution.

[0059] This invention proposes four variants, each without a specific key component, to allow for a deeper exploration of the design of our F2EA-PRec. Details are as follows: w / o Embedding Decoupling (w / o ED): Removed the embedding decoupling module for the original embedding. w / o Hybrid Embedding (w / o HE): Removed the hybrid embedding generated using user interest embeddings and sensitive attribute embeddings; only interest embeddings are used. w / o Adaptive Aggregation and Dynamic Debiasing (w / o AD): Removed the server-side adaptive aggregation model and the server-side feedback mechanism to the client. w / o Personalization Model Module (w / o PM): Removed the personalization module.

[0060] We conducted comparative experiments by sequentially removing each core component of the model to evaluate their independent contribution to improving overall performance and fairness. Due to the large dataset size, we performed ablation experiments on two representative datasets, ML-1M and ML-100K, and selected gender as a sensitive attribute to divide users into male and female groups, thereby calculating the fairness index of the groups. Figure 2 andFigure 3 Experimental results for two datasets are presented. The results clearly show that F2EA-PRec performs best in both recommendation performance metrics and group fairness metrics, balancing recommendation accuracy and group fairness, and verifying the overall effectiveness and synergistic effect of the proposed model. Specifically: (1) The removal of the embedding deentanglement module (w / o ED) leads to a significant decrease in fairness, indicating that it plays a key role in isolating the potential association between sensitive attributes and interest representations. This module effectively reduces the model's dependence on attributes such as gender by separating sensitive and non-sensitive information in the representation space, thereby improving the fairness of the recommendation results. (2) The removal of the hybrid embedding module (w / o HE) significantly reduces the overall performance of the model, indicating that considering both interest embedding and sensitive attribute embedding can more comprehensively capture the distribution of user preferences. Without this module, the model is unable to fully express individual differences among users, which leads to a decrease in recommendation accuracy and stability. (3) The elimination of the adaptive aggregation and dynamic debiasing modules (w / o AD) not only causes performance degradation but also significantly weakens fairness. This result reveals the importance of dynamic interaction between the server and the client: the server adjusts the influence weights of different clients through adaptive aggregation and effectively suppresses the propagation of bias caused by data heterogeneity by combining dynamic debiasing strategy. (4) The removal of the personalized model module (w / o PM) leads to the most obvious decrease in fairness, which shows that the local personalization mechanism plays a key role in alleviating the performance imbalance between different groups. Personalized modeling enables each client to adaptively optimize the local model according to its user characteristics, thereby achieving a fairer individual recommendation. Overall, the various components in the model cooperate with each other at different levels and form a synergistic effect: embedding deentanglement is responsible for debiasing the representation space, hybrid embedding enhances the feature expression ability, adaptive aggregation and dynamic debiasing provide global coordination across clients, while the personalization module ensures fairness among user groups at the local level.

[0061] These results collectively demonstrate that the proposed federal fair recommendation model has significant advantages in balancing recommendation accuracy and fairness, with the personalization mechanism and adaptive bias removal strategy playing a particularly crucial role in balancing model performance and social equity goals.

[0062] from Figure 4 As can be seen, all models tend to stabilize after approximately 1300 training rounds. In terms of convergence speed, FPFR converges the fastest, while CustomFair converges the slowest. Overall, F2EA-PRec not only performs well in convergence efficiency but also achieves optimal results in both recommendation performance and fairness.

[0063] When the batch size was set to [16, 32, 64, 128, 256, 512], the experimental results are as follows:Figure 5 As shown, the experimental results indicate that as the batch size increases, the recommendation performance metrics fluctuate across both datasets without exhibiting a clear monotonic trend; while the fairness metrics show a trend of first improving and then decreasing, achieving optimal fairness performance at a batch size of 128. Considering both performance and fairness, a batch size of 128 was ultimately determined as the baseline setting for model training.

[0064] To investigate the impact of the interest embedding ratio on the experimental results, we will... The values ​​are set to [0.1, 0.3, 0.5, 0.7, 0.9] respectively. Figure 6 (a) It can be observed that, with As the value of HR@10 gradually increases, the HR@10 metric continues to rise, while DiffHR@10 gradually decreases. This indicates that appropriately increasing the proportion of interest embedding helps the model better capture user interest features, thereby improving recommendation accuracy while mitigating recommendation bias between different groups. When increased to 0.5, HR@10 reached its peak; when As the value continues to increase, HR@10 begins to decrease, while Diff HR@10 also rebounds and fluctuates. This indicates that a high proportion of interest embeddings may cause the model to over-rely on interest signals, thereby weakening its generalization ability and adaptability to diverse needs, leading to a simultaneous degradation in performance and fairness. This suggests that excessively high... This will lead to a simultaneous degradation in both performance and fairness. Therefore, to balance recommendation effectiveness and fairness, we will... Set it to 0.5.

[0065] We will parameters The values ​​were set to [0.1, 0.5, 1.0, 2.0, 5.0] respectively to explore their impact on model performance and fairness. The results are as follows: Figure 7 As shown. From Figures 5-6 (a) It can be observed that Diff HR@10 shows a trend of first increasing and then decreasing, and in The optimal value is reached when the value is 0.1; meanwhile, although the HR@10 indicator fluctuates slightly, it remains within the range of 0.1. Optimal performance is achieved when the value is 0.1. Figure 7 In (b), Diff NDCG@10 exhibits a large fluctuation trend, and in The optimal value is reached when NDCG@10 is equal to 2; while NDCG@10 increases with increasing NDCG@10. The value exhibits a non-linear change of "first decreasing – then increasing – then decreasing again". The highest value is achieved when the value is 0.1. Combining the results of both indicators, the final result will be... The value is set to 0.1. This trend can be observed from the predictor constraint. We will explain its mechanism of action. The weights of predictor constraints in the total loss were controlled, when When smaller, The constraints on the optimization process are relatively weak; the model mainly focuses on the reconstruction loss. Therefore, it can more fully fit user preferences, thus achieving optimal accuracy. With As the value increases, the predictor constraints gradually strengthen, forcing the model to reduce the imbalance caused by sensitive attributes while optimizing recommendation performance. However, when... When the value is too large, the predictor loss accounts for too high a proportion of the total optimization objective. The model is forced to undergo excessive biasing during training, leading to weakened interest-related features, decreased model representation ability, and consequently, a decline in recommendation performance. Simultaneously, fairness may fluctuate due to information loss. When the value is 0.1, a relative balance is achieved between recommendation accuracy and fairness, therefore... Set it to 0.1.

[0066] To evaluate the orthogonal loss coefficient Sensitivity, will The values ​​were set to [0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 1.1] respectively. The results are as follows... Figure 8 As shown, Figure 8 (a) with Increasing the orthogonality constraint from 0.4 to 1.0 resulted in an upward trend in the recommendation performance HR@10, indicating that moderate orthogonality constraints reduced feature redundancy, thereby enhancing the model's discriminative ability. However, when... When the orthogonality penalty increased from 1.0 to 1.1, HR@10 began to decrease, indicating that excessively strong orthogonality penalties limit the flexibility of model representation, leading to impaired expressive power. Regarding fairness, Diff HR@10 exhibited significant fluctuations across the entire parameter range, generally showing a trend of first increasing and then decreasing, reflecting a sensitive range in the moderating effect of orthogonality constraints on group bias; its... By achieving optimal values ​​in the vicinity, the model can effectively balance recommendation results among different user groups while maintaining high accuracy. Figure 8 A similar trend is observed in (b), where the combined performance and fairness metrics ultimately... The value is set to 1.0.

[0067] To test the sensitivity of the number of cluster centers K, the parameter range was set to [5, 10, 15, 20]. Figure 9As shown in (a), as the number of cluster centers increases, the HR@10 value increases, while the Diff HR@10 value decreases. When K=15, HR@10 reaches its highest value, and Diff HR@10 reaches its lowest value, indicating that the model achieves the optimal balance between recommendation accuracy and personalization at this point. Figure 9 Other relevant indicators in (b) also show a similar trend, further validating the rationality of this parameter setting. This indicates that an appropriate number of clusters can more finely segment user interest groups, making user interests within clusters more consistent, thereby improving the quality of the model after federated aggregation. We ultimately set the K value to 15.

[0068] To determine the optimal number of MLP layers in the projector, we conducted experiments with the number of layers set to [1, 2, 3, 4, 5]. The experimental results are shown in Figure 10. It can be observed that when the number of MLP layers is 4, HR@10 and NDCG@10 reach their highest values, while Diff HR@10 and Diff NDCG@10 also exhibit superior performance. Considering overall performance, we set the number of MLP layers to 2 in the experiment.

[0069] This invention addresses the core challenges of recommender systems in federated learning environments, such as group bias, client heterogeneity, and global-local performance imbalance caused by sensitive attribute interference. It proposes a personalized federated fair recommender model, F2EA-PRec, based on embedding decoupling and fairness aggregation. This method employs an embedding decoupling mechanism that integrates dual projection heads, adversarial learning, and orthogonal regularization. By effectively separating sensitive attributes from interest subspaces, sensitive information in user embeddings is removed at the representation level, improving recommendation expressiveness while ensuring fairness. Secondly, a fairness-oriented adaptive aggregation and dynamic feedback mechanism is designed. Discriminator loss characterizes the degree of client bias, and scalar and gradient feedback guide local optimization, mitigating the impact of data heterogeneity on global model performance. Finally, a three-level personalized prediction framework—individual, group, and global—is constructed, achieving a balance between common knowledge sharing and individual difference modeling through federated clustering and weighted fusion. Experimental results on four real-world datasets demonstrate that F2EA-PRec significantly improves recommendation accuracy while effectively narrowing the performance gap between different genders, ages, and occupations of users, validating its effectiveness and robustness in the synergistic optimization of fairness and recommendation performance. This study still has certain limitations, such as the model's adaptability in scenarios with multiple types of sensitive attributes coexisting has not been fully validated. Future research will further expand this framework to support fair modeling and joint optimization under conditions of multiple sensitive attributes.

[0070] The present invention has been described by way of example. Obviously, the specific implementation of the present invention is not limited to the above-described manner. Any non-substantial improvements made using the inventive concept and technical solution of the present invention, or the direct application of the inventive concept and technical solution of the present invention to other occasions without modification, are all within the protection scope of the present invention.

Claims

1. A federated fair recommendation method based on embedding decoupling and adaptive aggregation, characterized in that, The method is as follows: (1) Extract the user-project interaction matrix from the local machine User-project interaction matrix Input a multi-layer graph convolutional network (GNN), and the GNN outputs a user embedding matrix. and project embedding matrix ; (2) Embedding users into the matrix Each user embedded in The inputs are two mapping functions composed of a multilayer perceptron (MLP), and the outputs of the two mapping functions are user embeddings. Embedding related to corresponding user-sensitive attributes And user interest-related embeddings ; (3) Embedding user-sensitive attributes Embedding related to user interests To integrate and form a user representation Estimate user u's rating of item i The highest-rated items will be recommended to user u.

2. The federated fair recommendation method based on embedding decoupling and adaptive aggregation as described in claim 1, characterized in that, After the client completes the project recommendation, perform the following steps: (4) Embedding user-sensitive attributes Input predictor, predictor predicts embeddings related to user-sensitive attributes User-sensitive attribute vectors Embedding user interests Input discriminator, output user interest-related embeddings User-sensitive attribute vectors ; (5) Calculate the total loss of the client's local model based on the attribute prediction loss of the predictor and discriminator. Based on total loss The local model parameters are updated using gradients, and the updated local model parameters are... Update the local model parameters and the attribute prediction loss of the discriminator Uploaded to the server; (6) The server predicts the loss based on the discriminator's attributes. As a measure of fairness Calculate the aggregation weight of each user participating in the aggregation. Based on aggregate weight Local model parameters uploaded by each user Aggregate the data to form the global model parameters for this round's global model. The data is then distributed to all participating clients for updating local model parameters.

3. The federated fair recommendation method based on embedding decoupling and adaptive aggregation as described in claim 2, characterized in that, Total loss of client u The specific calculation formula is as follows: ; in, This represents the weighting coefficient of the attribute prediction loss for client u in the current round. This indicates the set weighting coefficient. Represents the user-sensitive attribute vector of client u The attribute prediction loss; Represents the user-sensitive attribute vector of client u The attribute prediction loss; Embeddings related to user-sensitive attributes of client u Embedding related to user interests orthogonal loss between them This represents the Bayesian personalized ranking loss for client u.

4. The federated fair recommendation method based on embedding decoupling and adaptive aggregation as described in claim 2, characterized in that, The method further includes: (7) Calculate the direction of gradient change of the global model Calculate the gradient feedback coefficients that will participate in the next round of local model fine-tuning. And send it to the client; (8) Based on local model parameters The update direction involves K-means clustering of the user, with the user located in the k-th cluster... C k Local model parameters for all users Calculate its position in the cluster C k Inner Aggregation Weight Based on aggregate weight For local model parameters Perform aggregation to form aggregation parameters Aggregation parameters Distributed to clusters C k Internal clients; (9) The client will transfer global model parameters As initial parameters of the local model Based on gradient feedback coefficient Initial parameters for the local model Fine-tune the parameters to obtain the adjusted local model parameters. ; (10) Client updates local model parameters Then, proceed with the next round of training.

5. The federated fair recommendation method based on embedding decoupling and adaptive aggregation as described in claim 4, characterized in that, Gradient feedback coefficient The specific calculation formula is as follows: ； in, For gradient feedback intensity coefficient, To prevent stable terms with a denominator of zero.

6. The federated fair recommendation method based on embedding decoupling and adaptive aggregation as described in claim 5, characterized in that, Server updates discriminator's attribute prediction loss In the next round of weighting coefficients And distribute it to all clients participating in the aggregation, with weighting coefficients. The specific calculation formula is as follows: ； in, c It is expressed as the amplification constant.

7. The federated fair recommendation method based on embedding decoupling and adaptive aggregation as described in claim 2, characterized in that, Aggregate weight The specific calculation formula is as follows: ； in, This represents the deviation weighting adjustment coefficient.

8. The federated fair recommendation method based on embedding decoupling and adaptive aggregation as described in claim 1, characterized in that, Embedded user-sensitive attributes Embedding related to user interests The integration method is as follows: ； in, This indicates the fusion weight.

9. The federated fair recommendation method based on embedding decoupling and adaptive aggregation as described in claim 4, characterized in that, Aggregate model parameters Specifically as follows: ； in, This indicates that user u is in the kth cluster. C k Aggregate weights within.

Images