Enhanced federated recommendation method based on adaptive client aggregation and sampling

Abstract

The application discloses an enhanced federated recommendation method based on adaptive client aggregation and sampling, the core of which is to improve the recommendation performance and protect the user privacy by using an improved negative sampling strategy, interactive clustering weighted aggregation and adaptive differential privacy technology. The method first carries out clustering-driven enhanced negative sample sampling (CD-ENS), determines the non-interaction data item combination by obtaining the user interaction item list, carries out clustering by using a K-means algorithm, considers multiple factors as features, determines the optimal clustering number by calculating the contour coefficient, so as to realize more accurate negative sample sampling and improve the recommendation performance; then carries out interactive clustering weighted aggregation (IC-WA), calculates the client weight according to the proportion of the user and the project category interaction, multiplies the encrypted gradient uploaded by the client by the client weight and accumulates the same as the server gradient, so that the aggregation more accurately reflects the contribution of each client; and then introduces adaptive dynamic local differential privacy (LDP_MAD), adjusts the Laplace noise intensity based on the dynamic tensor median, so as to balance the gradient transmission and the recommendation system performance and protect the user privacy. In the recommendation process, the improved technology is used instead of the traditional method, the problems existing in the existing federated recommendation system are solved, the accuracy and personalization of the recommendation are improved, and the user privacy is effectively protected.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of federated learning and recommendation systems, and specifically points out an enhanced federated recommendation method based on adaptive client aggregation and sampling. BACKGROUND

[0002] In today's information age, recommendation systems play an important role in helping users access information of interest. However, traditional recommendation systems have deficiencies in data collection and privacy protection, and federated recommendation systems have emerged. However, existing federated recommendation algorithms still face challenges in negative sample sampling, client data diversity processing, and privacy protection.

[0003] The present application introduces federated learning and adaptive sampling technology, combines clustering analysis and differential privacy, and proposes a personalized federated recommendation framework to improve the performance and privacy protection level of the recommendation system. SUMMARY

[0004] To solve the problems of unreasonable negative sample sampling, improper handling of client data diversity, and insufficient privacy protection in existing federated recommendation systems, the present application combines federated learning and adaptive sampling technology to provide an enhanced federated recommendation method based on adaptive client aggregation and sampling. The method performs the following steps in turn: enhanced negative sample sampling based on clustering-driven, interactive clustering weighted aggregation, introduction of adaptive dynamic local differential privacy, model training and evaluation.

[0005] An enhanced federated recommendation method based on adaptive client aggregation and sampling, characterized by:

[0006] (1) Perform enhanced negative sample sampling based on clustering-driven;

[0007] (2) Perform interactive clustering weighted aggregation;

[0008] (3) Introduce adaptive dynamic local differential privacy;

[0009] (4) Perform model training and evaluation.

[0010] The method of claim 1, wherein in the negative sample sampling step, the client first obtains a list of interaction items for each user to determine a group of non-interaction data items when selecting a negative sample label, and then performs clustering using a K-means algorithm, wherein multiple factors such as average score, maximum score, minimum score, interaction timestamp, and co-occurrence between items are considered as features in the clustering process. The hyperparameters of the K-means algorithm are crucial to the quality of clustering, as they directly affect the clustering results. The optimal hyperparameters are selected by a specific calculation method that takes into account the average distance of the sample from other samples in the same cluster (cohesion) and the distance from the nearest sample in other clusters. The relevant calculation results of the entire data set can be used as a standard to evaluate the clustering effect. Specifically, multiple hyperparameters and corresponding results are calculated, and the hyperparameter with the optimal result is selected as the optimal hyperparameter.

[0011] The method of claim 1, wherein in the interaction clustering and weighted aggregation step, the federated recommendation system introduces a new method to calculate the weight of each client in the aggregation process, which is determined by the proportion of user interaction with each item category. Specifically, for each item category that the user has interacted with, the proportion of the number of user-interacted items in the total number of items in that category is calculated, and the weighted sum of all these proportions is obtained by adding them up, and then multiplied by the total number of user-interacted items to obtain the clustering weight of the user. The server updates the global recommendation model by aggregating in a specific way. In addition, the encrypted gradient uploaded by each client is multiplied by the client weight and accumulated, and the result is used as the server's gradient.

[0012] The method of claim 1, wherein in the adaptive differential privacy step, an adaptive differential privacy technology based on dynamic tensor median is introduced. In the relevant data set, the definition of the median is related to the parity of the number of data. For each element in the data set, the absolute deviation of the element from the median is calculated, and finally the median of all absolute deviations is taken as the median deviation. Based on the median deviation, the strength of the Laplace noise is adjusted, and the noise is added through a specific formula to achieve the purpose of privacy protection.

[0013] The method according to claim 1 is characterized in that: in the model training and evaluation steps, three datasets, MovieLens-100K, MovieLens-1M, and Lastfm-2K, are used for evaluation, and two indicators, click-through rate (HR) and normalized discounted cumulative gain (NDCG), are used as evaluation criteria. Among them, HR@10 is used to measure whether the relevant items appear in the top 10 recommendations. NDCG@10 is evaluated by calculating the discounted cumulative gain (DCG@10) and the ideal discounted cumulative gain (IDCG@10), and is finally calculated by dividing DCG@10 by IDCG@10.

[0014] For example, on the MovieLens-100K dataset, the HR@10 value reached 0.754 and the NDCG@10 value reached 0.421, which are significantly improved compared to other methods. Similar good results were achieved on the MovieLens-1M dataset and the Lastfm-2K dataset. This shows that the method of the present invention can provide more accurate and personalized recommendations, improving the user experience. At the same time, by introducing adaptive differential privacy technology, the user's privacy is effectively protected and the security of the system is enhanced. During the model implementation process, the embedding vector size was set to 32 and the batch size was 512. The model was implemented using the PyTorch framework, and the average results were obtained through five-fold cross-validation.

[0015] The beneficial effects of the above technical solution of the present invention are as follows:

[0016] In this approach, the performance and privacy protection of federated recommendation systems are enhanced through the synergistic effects of cluster-driven enhanced negative sampling, interactive clustering weighted aggregation, and adaptive dynamic local differential privacy. Specifically, the improved negative sampling strategy and enhanced client-side aggregation algorithm improve the model's recommendation performance; the introduction of adaptive local differential privacy technology better balances gradient transmission and recommendation system performance, effectively protecting user privacy. This invention provides a novel framework for distributed recommendation systems, enabling more accurate and personalized recommendations while protecting user privacy. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 This is the model architecture diagram of AdapFedRec

[0018] Figure 2 This is the effect diagram of the model ablation experiment DETAILED DESCRIPTION

[0019] 1. Dataset and evaluation criteria:

[0020] MovieLens-100K, MovieLens-1M and Lastfm-2K are used for evaluation, and the evaluation indicators are hit rate (HR) and normalized discounted cumulative gain (NDCG). Among them, HR@10 is used to measure whether the relevant items appear in the top 10 recommendations, and its calculation formula is HR@10 = hit times / total number of test instances, and the hit times refers to the number of times the relevant items appear in the top 10 recommendations. The normalized discounted cumulative gain (NDCG@10) evaluates the ranking quality by calculating the discounted cumulative gain (DCG@10) and the ideal discounted cumulative gain (IDCG@10).

[0021] 2. Performance of real data sets:

[0022] AdapFedRec achieved a value of 0.754 for HR@10 and a value of 0.421 for NDCG@10 on the MovieLens-100K dataset, showing a clear advantage over other comparison methods. On the MovieLens-1M dataset, AdapFedRec achieved a HR@10 of 0.7669 and a NDCG@10 of 0.4458, also showing significant advantages. In the Lastfm-2K dataset, AdapFedRec also showed good performance in HR@10 and NDCG@10.

[0023] 3. Experimental setup:

[0024] The size of the embedding vector is set to 32, the batch size is set to 512, the model is implemented using the PyTorch framework, and the average result is obtained through five-fold cross-validation. The silhouette coefficient of each clustering scenario is calculated to determine the number of item clusters. For example, when using the ML-100k dataset, the average silhouette coefficient corresponding to different cluster numbers will be different, and the k value with the highest silhouette coefficient is selected as the cluster number.

[0025] 4. Result analysis:

[0026] AdapFedRec has a significant improvement on MovieLens-100K and MovieLens-1M datasets, and can more accurately select irrelevant items to the target user in the negative sampling process, enhancing the ability of personalized recommendation. AdapFedRec is superior to other federated recommendation baselines, and the weighted aggregation of different client updates in its framework can reduce the risk of overfitting of the model on specific client data, improving the generalization ability of the model. While some existing federated learning frameworks use the average method to aggregate client gradient information, they cannot effectively handle the heterogeneity of client data, affecting the accuracy and robustness of the model. The importance of the enhanced negative sample sampling based on clustering driven (CD-ENS) and the interaction clustering weighted aggregation (IC-WA) modules is verified through ablation experiments. AdapFedRec-Variation1 (remove CD-ENS module) makes the recommendation performance of the model decrease significantly, and AdapFedRec-Variation2 (remove IC-WA module) shows that this module is extremely critical to improve the recommendation accuracy. The relevant figures of removing the experimental modules can be seen in the accompanying drawings. After introducing the adaptive dynamic local differential privacy (LDP_MAD) technology, with the increase of the noise parameter λ, the performance will decrease slightly, but it is still within an acceptable range. Therefore, the introduction of moderate noise can effectively balance the recommendation accuracy and privacy protection.

[0027] The above is the preferred embodiment of the present application. It should be noted that for those skilled in the art, without departing from the principles of the present application, several improvements and refinements can be made, which should also be considered within the scope of protection of the present application.

Claims

1. An enhanced federated recommendation method based on adaptive client aggregation and sampling, characterized by: (1) Using the cluster-driven enhanced negative sampling (CD-ENS) module, which selects negative samples through cluster analysis; (2) Using the Interaction Clustering Weighted Aggregation (IC-WA) module, this module calculates the weight of the client based on the proportion of user interactions with item categories; (3) Introducing the adaptive dynamic local differential privacy (LDP_MAD) module, which adjusts the strength of Laplace noise based on the dynamic tensor median; (4) Training and evaluation of the model; the enhanced negative sample sampling (CD-ENS) module is as follows: when the client selects the negative sample label, it first obtains the list of interactive items of each user and then determines the combination of non-interactive data items; then, it uses the K-means algorithm to perform clustering, and considers the average score, maximum score, minimum score, interaction timestamp and co-occurrence between items in the clustering process; then, it determines the number of clusters k; The Interaction Clustering Weighted Aggregation (IC-WA) module calculates the weight of each client during the aggregation process. This weight is determined by the proportion of the user's interactions with each item category. The proportion of the number of items in each category that the user interacted with is calculated as a percentage of the total number of items in that category. All these proportions are added together to form a weighted sum, which is then multiplied by the total number of items the user interacted with to obtain the clustering weight. During aggregation, the encrypted gradient uploaded by each client is multiplied by the client weight and accumulated, and the result is used as the server gradient. The adaptive dynamic local differential privacy (LDP_MAD) module is as follows: in a dataset, when the number of elements is odd, the median is the value of the middle element; when the number of elements is even, the median is the average of the values ​​of the two middle elements. For each element, its absolute deviation from the median is calculated, that is, the absolute value of the difference between the element and the median. Finally, the median of all absolute deviations is taken as the median deviation.

Images