Enhanced federal recommendation method based on adaptive client aggregation and sampling

Abstract

The invention discloses an enhanced federal recommendation method based on adaptive client aggregation and sampling. The core of the method is to improve recommendation performance and protect user privacy by using an improved negative sampling strategy, interactive clustering weighted aggregation and an adaptive differential privacy technology. According to the method, firstly, enhanced negative sample sampling (CD-ENS) based on clustering driving is carried out, non-interactive data item combinations are determined by obtaining a user interaction item list, clustering is carried out by using a K-means algorithm, multiple factors are considered as features, and an optimal clustering number is determined by calculating a contour coefficient, so that more accurate negative sample sampling is realized, and the accuracy of the negative sample sampling is improved. The recommendation performance is improved; then interactive clustering weighted aggregation (IC-WA) is carried out, the client weight is calculated according to the interaction proportion of the user and the project category, the encryption gradient uploaded by the client and the client weight are multiplied and accumulated to serve as the server gradient, and the aggregation can reflect the contribution of each client more accurately; and then introducing adaptive dynamic local differential privacy (LDPMAD), and adjusting Laplacian noise intensity based on a dynamic tensor median so as to balance gradient transmission and recommendation system performance and protect user privacy. In the recommendation process, the improved technology is used for replacing a traditional method, the problems existing in an existing federal recommendation system are solved, the recommendation accuracy and personalization are improved, and meanwhile user privacy is effectively protected.

Description

Technical Field

[0001] The present invention relates to the field of federated learning and recommendation systems, and particularly points out an enhanced federated recommendation method based on adaptive client aggregation and sampling. Background Art

[0002] In today's information age, recommendation systems play an important role in helping users obtain information of interest. However, traditional recommendation systems have shortcomings in data collection and privacy protection, and federated recommendation systems have emerged. However, existing federated recommendation algorithms still face challenges in negative sample sampling, client data diversity processing, and privacy protection.

[0003] This paper introduces federated learning and adaptive sampling technology, combines cluster analysis and differential privacy, and proposes a personalized federated recommendation framework, aiming to improve the performance and privacy protection level of the recommendation system. Summary of the invention

[0004] In order to solve the problems of unreasonable negative sample sampling, improper handling of client data diversity, and insufficient privacy protection in the existing federated recommendation system, the present invention combines federated learning and adaptive sampling technology to provide an enhanced federated recommendation method based on adaptive client aggregation and sampling. The method sequentially performs the following steps: cluster-driven enhanced negative sample sampling, interactive clustering weighted aggregation, introduction of adaptive dynamic local differential privacy, and model training and evaluation.

[0005] An enhanced federated recommendation method based on adaptive client aggregation and sampling, characterized by:

[0006] (1) Perform cluster-driven enhanced negative sample sampling;

[0007] (2) Perform interactive clustering weighted aggregation;

[0008] (3) Introducing adaptive dynamic local differential privacy;

[0009] (4) Train and evaluate the model.

[0010] The method according to claim 1 is characterized in that: in the negative sample sampling step, when the client selects the negative sample label, it first obtains the list of interactive items of each user, thereby determining the combination of non-interactive data items. Then, the K-means algorithm is used for clustering, and multiple factors such as average score, maximum score, minimum score, interaction timestamp, and co-occurrence between items are considered as features during the clustering process. Determining the hyperparameters of the K-means algorithm is crucial to the quality of clustering because it directly affects the effect of clustering. The best hyperparameters are selected by a specific calculation method, which comprehensively considers the average distance (cohesion) between the sample and other samples in the same cluster and the distance to the nearest sample in other clusters. The relevant calculation results of the entire data set can be used as a standard for evaluating the clustering effect. Specifically, multiple hyperparameters and corresponding results are calculated, and then the hyperparameter with the best result is selected as the best hyperparameter.

[0011] The method according to claim 1 is characterized in that: in the interactive clustering weighted aggregation step, the federated recommendation system introduces a new method to calculate the weight of each client in the aggregation process, and the weight is determined by the proportion of the user's interaction with each project category. In the specific calculation, for each project category that the user has interacted with, the proportion of the number of projects in the category that the user has interacted with in the total number of projects will be calculated, and all these proportions will be added together to obtain a weighted sum, which is then multiplied by the total number of projects that the user has interacted with to obtain the clustering weight of the user. When the server updates the global recommendation model, it will perform aggregation in a specific way. In addition, the encrypted gradient uploaded by each client is multiplied and accumulated by the client weight, and the result is used as the gradient of the server.

[0012] The method according to claim 1 is characterized in that: in the step of introducing adaptive differential privacy, an adaptive differential privacy technology based on the median of the dynamic tensor is introduced. In the relevant data set, the definition of the median is related to the parity of the number of data. For each element in the data set, its absolute deviation from the median is calculated, and finally the median of all absolute deviations is taken as the median deviation. The intensity of Laplace noise is adjusted based on the median deviation, and noise is added through a specific formula to achieve the purpose of privacy protection.

[0013] The method according to claim 1 is characterized in that: in the model training and evaluation steps, three data sets, MovieLens-100K, MovieLens-1M and Lastfm-2K, are used for evaluation, and two indicators, click-through rate (HR) and normalized discounted cumulative gain (NDCG), are used as evaluation criteria. Among them, HR@10 is used to measure whether the relevant items appear in the top 10 recommendations. NDCG@10 evaluates the ranking quality by calculating the discounted cumulative gain (DCG@10) and the ideal discounted cumulative gain (IDCG@10), and finally it is calculated by dividing DCG@10 and IDCG@10.

[0014] For example, on the MovieLens-100K dataset, the HR@10 value reached 0.754, and the NDCG@10 value reached 0.421, which is a significant improvement compared to other methods. Similar good results were achieved on the MovieLens-1M dataset and the Lastfm-2K dataset. This shows that the method of the present invention can provide more accurate and personalized recommendations and improve the user experience. At the same time, by introducing adaptive differential privacy technology, the user's privacy is effectively protected and the security of the system is enhanced. In the model implementation process, the embedding vector size is set to 32, the batch size is 512, the PyTorch framework is used for model implementation, and the average results are obtained through five-fold cross validation.

[0015] The beneficial effects of the above technical solution of the present invention are as follows:

[0016] In the above scheme, the method improves the performance and privacy protection level of the federated recommendation system through the synergy of modules such as cluster-driven enhanced negative sample sampling, interactive clustering weighted aggregation, and adaptive dynamic local differential privacy. Specifically, the recommendation performance of the model is improved through the improved negative sampling strategy and enhanced client aggregation algorithm; the introduction of adaptive local differential privacy technology better balances the performance of gradient transmission and recommendation system, and effectively protects user privacy. The present invention provides a new framework for distributed recommendation systems, which can achieve more accurate and personalized recommendations while protecting user privacy. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 Model architecture diagram of AdapFedRec

[0018] Figure 2 This is the effect diagram of the model ablation experiment DETAILED DESCRIPTION

[0019] 1. Dataset and evaluation criteria:

[0020] The three datasets MovieLens-100K, MovieLens-1M and Lastfm-2K were used for evaluation, and the evaluation indicators used were click-through rate (HR) and normalized discounted cumulative gain (NDCG). Among them, HR@10 is used to measure whether the relevant item appears in the top 10 recommendations. The calculation formula is HR@10 = number of hits / total number of test instances, and the number of hits refers to the number of times the relevant item appears in the top 10 recommendations. Normalized discounted cumulative gain (NDCG@10) evaluates the ranking quality by calculating the discounted cumulative gain (DCG@10) and the ideal discounted cumulative gain (IDCG@10).

[0021] 2. Performance of real data sets:

[0022] On the MovieLens-100K dataset, AdapFedRec achieved a HR@10 value of 0.754 and a NDCG@10 value of 0.421, which are significantly superior to other comparison methods. On the MovieLens-1M dataset, AdapFedRec's HR@10 reached 0.7669 and NDCG@10 reached 0.4458, also showing significant advantages. In the Lastfm-2K dataset, AdapFedRec's HR@10 and NDCG@10 also showed good performance.

[0023] 3. Experimental setup:

[0024] The embedding vector size is set to 32, the batch size is set to 512, the model is implemented using the PyTorch framework, and the average results are obtained through five-fold cross validation. The silhouette coefficient is calculated for each clustering scenario to determine the number of project clusters. For example, when using the ML-100k dataset, the average silhouette coefficient corresponding to different numbers of clusters will be different. The k value with the highest silhouette coefficient is selected as the number of clusters.

[0025] 4. Result analysis:

[0026] AdapFedRec has significantly improved performance on the MovieLens-100K and MovieLens-1M datasets. It can more accurately select items that are not related to the target user during negative sampling, enhancing the ability of personalized recommendations. AdapFedRec is superior to other federated recommendation baselines. The weighted aggregation of updates from different clients in its framework can reduce the risk of overfitting the model on specific client data and improve the generalization ability of the model. However, some existing federated learning frameworks use an average method to aggregate client gradient information, which cannot effectively handle the heterogeneity of client data, affecting the accuracy and robustness of the model. The importance of cluster-driven enhanced negative sample sampling (CD-ENS) and interactive clustering weighted aggregation (IC-WA) modules was verified through ablation experiments. AdapFedRec-Variation1 (removing the CD-ENS module) significantly reduces the recommendation performance of the model, and AdapFedRec-Variation2 (removing the IC-WA module) shows that this module is extremely critical to improving the accuracy of recommendations. The relevant diagrams of the removed experimental modules can be viewed in the attached figure. After introducing the adaptive dynamic local differential privacy (LDP_MAD) technology, as the noise parameter λ increases, the performance will decrease slightly, but it is still within an acceptable range. Therefore, introducing moderate noise can effectively balance the recommendation accuracy and privacy protection.

[0027] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. An enhanced federated recommendation method based on adaptive client aggregation and sampling, characterized by: (1) A cluster-driven enhanced negative sampling (CD-ENS) module is used. This module selects negative samples through cluster analysis, which can ensure the balance of feature distribution and thus improve the performance of the recommendation system. (2) Using the Interaction Clustering Weighted Aggregation (IC-WA) module, this module calculates the weight of the client according to the proportion of user-item category interactions, so that the aggregation process can more accurately reflect the contribution of each client. (3) An adaptive dynamic local differential privacy (LDP_MAD) module is introduced, which adjusts the strength of Laplace noise based on the dynamic tensor median to balance the performance of gradient transmission and recommendation system and effectively protect user privacy. (4) The performance of the model is improved by adopting an improved negative sampling strategy and an enhanced client aggregation algorithm. (5) It provides a new framework for distributed recommendation systems that can effectively implement recommendation functions and protect user privacy.

2. According to the enhanced federated recommendation method based on adaptive client aggregation and sampling described in claim 1, the main feature of its (CD-ENS) module is that: when the client selects a negative sample label, the interactive project list of each user is first obtained, and then the non-interactive data item combination is determined; then, the K-means algorithm is used for clustering, and multiple factors are considered as features in the clustering process, such as average score, maximum score, minimum score, interaction timestamp, and co-occurrence between items; then, the optimal number of clusters k is determined by a specific calculation method, which considers the average distance between the sample and other samples in the same cluster and the distance to the nearest sample in other clusters to ensure the best clustering effect.

3. The feature of the (IC-WA) module is that when calculating the weight of each client in the aggregation process, the weight is determined by the proportion of the user's interaction with each project category. Specifically, the proportion of the number of items that the user interacts with in each category to the total number of items in the category is calculated, and all these proportions are added to obtain a weighted sum, which is then multiplied by the total number of items that the user interacts with to obtain the clustering weight. During aggregation, the encrypted gradient uploaded by each client is multiplied by the client weight and accumulated, and the result is used as the gradient of the server. In addition, the adaptive dynamic local differential privacy technology introduced is specifically: in the data set, when the number of elements is an odd number, the median is the value of the middle element; when the number of elements is an even number, the median is the average of the values ​​of the two middle elements. For each element, its absolute deviation from the median is calculated, that is, the absolute value of the difference between the element and the median, and finally the median of all absolute deviations is taken as the median deviation. Based on this median deviation, the intensity of the Laplace noise is adjusted to achieve better privacy protection.

4. In response to the problems existing in the existing federated recommendation algorithm, such as unreasonable negative sample sampling, improper handling of client data diversity and data volume, and insufficient privacy protection, this paper proposes innovative modules such as cluster-driven enhanced negative sample sampling, interactive clustering weighted aggregation, and adaptive dynamic local differential privacy, which can effectively solve the above problems. The recommendation performance of the model is improved through the improved negative sampling strategy and enhanced client aggregation algorithm; at the same time, the introduction of adaptive local differential privacy technology better balances the performance of gradient transmission and recommendation system, and effectively protects the privacy of users.

5. The present invention provides a new framework for distributed recommendation systems, which can achieve more accurate and personalized recommendations while protecting user privacy, and has important practical application value.

Images