A news recommendation method, system, device and medium based on federated learning

Abstract

The present invention discloses a news recommendation method, system, device and medium based on federated learning, which relates to the field of information recommendation. The method includes: extracting news features from the local historical clicked news of each client through a news model; extracting behavioral features from the auxiliary data of each client through a behavioral feature model; clustering the clients by using a clustering algorithm to obtain k clusters; in each cluster, training a user model by using a news feature data set; aggregating the trained user models corresponding to each cluster, and distributing the aggregated user model to each cluster; inputting the news features extracted from candidate news into the aggregated user models in each cluster to obtain k to-be-predicted user representations; and determining whether to distribute the candidate news to the client according to the to-be-predicted user representations in each cluster and the news features extracted from the candidate news. The present invention improves the matching degree between the recommended news data and the user.

Description

Technical Field

[0001] The present invention relates to the technical field of information recommendation, and particularly to a news recommendation method, system, device and medium based on federated learning. Background Art

[0002] In recent years, with the rapid development of big data and the Internet of Things, the amount of data has shown an explosive growth. Among the massive data, the problem of user privacy protection has become particularly prominent. In the training process of ordinary distributed machine learning, the local data of several clients will be directly sent to the server for unified training, which will undoubtedly increase the risk of user privacy leakage. Therefore, it is an urgent problem to be solved at present to enable the data of multiple clients to participate in secure modeling and learning, so as to improve the matching degree between the recommended news data and users. Summary of the Invention

[0003] The purpose of the present invention is to provide a news recommendation method, system, device and medium based on federated learning, which improves the matching degree between the recommended news data and users.

[0004] To achieve the above object, the present invention provides the following solutions:

[0005] A news recommendation method based on federated learning includes:

[0006] For clients within a set range, news features are extracted from the local historical clicked news of each client through a news model;

[0007] For clients within a set range, behavior features are extracted from the auxiliary data of each client through a behavior feature model;

[0008] Based on the news features and behavior features of each client received by the server, a clustering algorithm is used to cluster the clients to obtain k clusters; each cluster includes multiple clients;

[0009] In each cluster, a user model is trained using a news feature data set to obtain a trained user model corresponding to each cluster; each cluster corresponds to a news feature data set, and each sample data in the news feature data set includes news features and user representations corresponding to the news features. The trained user model is used to output a user representation according to the input news features;

[0010] The trained user models corresponding to each cluster are aggregated, and the aggregated user model is sent to each cluster;

[0011] News features are extracted from candidate news through a news model;

[0012] Input the news features extracted from the candidate news into the user model aggregated in each cluster to obtain k user representations to be predicted;

[0013] Input the user representations to be predicted in each cluster and the news features extracted from the candidate news into a click predictor to obtain k click scores;

[0014] Send the candidate news to the clients in the cluster corresponding to the click scores exceeding the set value.

[0015] Optionally, the news features include the click probabilities of various types of news.

[0016] Optionally, the auxiliary data includes shopping records, website browsing records, and video viewing records.

[0017] Optionally, the clustering algorithm is the K-Means algorithm.

[0018] Optionally, based on the news features and behavior features of each client received by the server, use a clustering algorithm to cluster the clients to obtain k clusters, specifically including:

[0019] Distinguish each client into a first client and a second client. The first client is a client with news features, and the second client is a client without news features;

[0020] Based on the news features of each first client, use a clustering algorithm to cluster each first client to obtain k clusters;

[0021] For the i-th second client, calculate the similarity between the behavior features of the i-th second client and the behavior features of each first client, and divide the i-th second client into the cluster where the first client with the highest similarity value is located.

[0022] The present invention also discloses a news recommendation system based on federated learning, including:

[0023] A historical news feature extraction module, configured to extract news features from the local historical clicked news of each client through a news model for clients within a set range;

[0024] A historical behavior feature extraction module, configured to extract behavior features from the auxiliary data of each client through a behavior feature model for clients within a set range;

[0025] A client clustering module, configured to cluster clients using a clustering algorithm based on the news features and behavior features of each client received by the server to obtain k clusters; each cluster includes multiple clients;

[0026] A user model training module, which is used to train a user model in each cluster by using a news feature dataset, and obtain a trained user model corresponding to each cluster; each cluster corresponds to a news feature dataset, and each sample data in the news feature dataset includes news features and user representations corresponding to the news features. The trained user model is used to output user representations according to the input news features;

[0027] A user model aggregation module, which is used to aggregate the trained user models corresponding to each cluster, and distribute the aggregated user model to each cluster;

[0028] A module for extracting news features from candidate news, which is used to extract news features from candidate news through a news model;

[0029] A module for obtaining user representations to be predicted, which is used to input the news features extracted from candidate news into the aggregated user model in each cluster to obtain k user representations to be predicted;

[0030] A click prediction module, which is used to input the user representations to be predicted in each cluster and the news features extracted from candidate news into a click predictor to obtain k click scores;

[0031] A candidate news distribution module, which is used to distribute candidate news to the clients in the clusters where the click scores exceed a set value.

[0032] The present invention also discloses a device, including a memory and a processor. The memory is used to store a computer program, and the processor runs the computer program so that the device executes the news recommendation method based on federated learning.

[0033] The present invention also discloses a medium, which stores a computer program. When the computer program is executed by a processor, the news recommendation method based on federated learning is implemented.

[0034] According to the specific embodiments provided by the present invention, the following technical effects are disclosed by the present invention:

[0035] Based on the news features and behavior features of each client received by the server, the present invention uses a clustering algorithm to cluster the clients, and aggregates the client information with similar preferences through clustering, reducing the problem of slow model convergence that may be caused by the sparsity of some user data, improving the accuracy of the user model to output user representations, and thus improving the matching degree between the recommended news data and the users. Description of the Drawings

[0036] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the accompanying drawings required for use in the embodiments. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0037] Figure 1 Schematic flow diagram of a news recommendation method based on federated learning of the present invention;

[0038] Figure 2 Schematic diagram of local model training of the client of the present invention;

[0039] Figure 3 Schematic diagram of the clustering principle of the client of the present invention;

[0040] Figure 4 Schematic diagram of the structure of a news recommendation system based on federated learning of the present invention. Specific embodiments

[0041] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, rather than all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present invention.

[0042] The purpose of the present invention is to provide a news recommendation method, system, device and medium based on federated learning, which improves the matching degree between the recommended news data and users.

[0043] To make the above objects, features and advantages of the present invention more obvious and understandable, the present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0044] Embodiment 1

[0045] As Figure 1 shown, a news recommendation method based on federated learning of the present invention includes:

[0046] Step 101: For clients within a set range, extract news features from the local historical clicked news of each client through a news model.

[0047] The news features include the click probabilities of various types of news.

[0048] The types of news include but are not limited to entertainment news, livelihood news, sports news and military news.

[0049] Step 102: For clients within a set range, extract behavioral features from the auxiliary data of each client through a behavioral feature model.

[0050] The auxiliary data includes, but is not limited to, shopping records, website browsing records, and video viewing records, and also includes user age, gender, and consumption level.

[0051] The behavioral feature is the user preference obtained from the auxiliary data, that is, a user preference abstracted from the shopping records, website browsing records, and video viewing records obtained from the client (for example, if a person has just bought a house recently, they tend to prefer home furnishings and decoration; if they have had a baby, they tend to prefer baby products; if they are in school, they tend to prefer learning videos).

[0052] User preferences include, but are not limited to, home decoration, baby products, and learning videos.

[0053] Both Step 101 and Step 102 are feature extractions performed locally on the client.

[0054] Both the news model and the behavioral feature model are determined through training based on the local data of the client. The inputs and outputs of the news model and the behavioral feature model are as Figure 2 shown.

[0055] Send the news features and behavioral features of each client to the server, and the server is connected to each client.

[0056] During the upload process of news features and behavioral features, a certain security protocol (such as multi-party secure computing, homomorphic encryption, differential privacy, etc.) can be used to protect user privacy.

[0057] Step 103: Based on the news features and behavioral features of each client received by the server, use a clustering algorithm to cluster the clients and obtain k clusters; each cluster includes multiple clients.

[0058] The clustering algorithm is the K-Means algorithm, which makes clients with similar preferences in the same cluster. The value of k is usually considered in relation to the number of categories of candidate news.

[0059] Among them, Step 103 specifically includes:

[0060] Distinguish each client into a first client and a second client. The first client is a client with news features, and the second client is a client without news features.

[0061] Based on the news features of each of the first clients, use a clustering algorithm to cluster each of the first clients and obtain k clusters.

[0062] For the i-th second client, calculate the similarity between the behavioral characteristics of the i-th second client and those of each first client, and assign the i-th second client to the cluster where the first client corresponding to the highest similarity value is located.

[0063] The similarity of the characteristics is specifically the cosine similarity. For client i and client j, the cosine similarity between the two is expressed as follows:

[0064] sim(x i , x j ) = ||x i , x j ||.

[0065] Among them, xi represents the behavioral characteristics of client i, and x j represents the behavioral characteristics of client j. sim{x i , x j} represents the cosine similarity between client i and client j.

[0066] Assume that a total of N clients participate and are divided into k clusters. After the K-Means algorithm, client clusters c1, c2,..., c k can be obtained, and N = {c1 ∪ c2 ∪... ∪ c k}. The principle of client clustering is as Figure 3 shown.

[0067] Step 104: In each cluster, train a user model using the news feature dataset to obtain a trained user model corresponding to each cluster; each cluster corresponds to a news feature dataset, and each sample data in the news feature dataset includes news features and user representations corresponding to the news features. The trained user model is used to output a user representation based on the input news features.

[0068] The user representation is an abstract representation of information such as the gender composition, age range, and news type preferences of a class of similar users.

[0069] The user representation is a vector, and the gender composition, age range, and news preferences are represented by the values in the vector. For example, for gender, if 0 represents female and 1 represents male, then the value closer to 1 indicates that there are more males in the user group, and vice versa, the number closer to 0 indicates that there are more females among the users.

[0070] Among them, step 104 specifically includes: training the user model with the news features in each cluster respectively, and finally obtaining k user models.

[0071] Step 105: Aggregate the trained user models corresponding to each cluster, and distribute the aggregated user model to each cluster.

[0072] Among them, step 105 specifically includes: uploading the parameters of each trained user model to the server side for aggregation, and sending the aggregated user model to the sub-servers in each cluster.

[0073] Step 106: Extract news features from the candidate news through the news model.

[0074] Step 107: Input the news features extracted from the candidate news into the aggregated user model in each cluster to obtain k user representations to be predicted.

[0075] Step 108: Input the user representations to be predicted in each cluster and the news features extracted from the candidate news into the click predictor to obtain k click scores.

[0076] The click predictor is used to predict the probability (click score) that the client user clicks on the candidate news according to the user representation to be predicted and the news features. The click predictor is specifically a logistic regression (LR) model.

[0077] Step 109: Send the candidate news to the clients in the cluster corresponding to the click score exceeding the set value.

[0078] The present invention discloses a news recommendation method based on federated learning. This method divides the recommendation model into a news model and a user model, aggregates clients with a certain similarity in the same cluster to jointly construct a user model, reducing the computing overhead and communication cost of local clients; at the same time, when a cold start user joins, the preference of the user can be quickly determined according to the feature matrix obtained from the auxiliary data set to determine which cluster the user should join to obtain appropriate recommended news. The present invention aggregates the client information with similar preferences through clustering, reducing the problem of slow model convergence that may be caused by the sparsity of some user data, while protecting the data security of users.

[0079] Embodiment 2

[0080] Figure 4 This is a schematic structural diagram of a news recommendation system based on federated learning according to the present invention. As Figure 4 shown, a news recommendation system based on federated learning includes:

[0081] A historical news feature extraction module 201, which is used to extract news features from the local historical clicked news of each client through the news model for the clients within a set range.

[0082] A historical behavior feature extraction module 202, which is used to extract behavior features from the auxiliary data of each client through the behavior feature model for the clients within a set range.

[0083] The client clustering module 203 is used to cluster clients based on the news features and behavior features of each client received by the server by using a clustering algorithm to obtain k clusters; each cluster includes multiple clients.

[0084] The user model training module 204 is used to train a user model in each cluster by using a news feature data set to obtain a trained user model corresponding to each cluster; each cluster corresponds to a news feature data set, and each sample data in the news feature data set includes news features and user representations corresponding to the news features. The trained user model is used to output a user representation according to the input news features.

[0085] The user model aggregation module 205 is used to aggregate the trained user models corresponding to each cluster and distribute the aggregated user model to each cluster.

[0086] The news feature extraction module 206 for extracting news features from candidate news is used to extract news features from candidate news through a news model.

[0087] The to-be-predicted user representation obtaining module 207 is used to input the news features extracted from candidate news into the aggregated user model in each cluster to obtain k to-be-predicted user representations.

[0088] The click prediction module 208 is used to input the to-be-predicted user representations in each cluster and the news features extracted from candidate news into a click predictor to obtain k click scores.

[0089] The candidate news distribution module 209 is used to distribute candidate news to the clients in the cluster corresponding to the click score exceeding the set value.

[0090] Embodiment 3

[0091] This embodiment provides a device, including a memory and a processor. The memory is used to store a computer program, and the processor runs the computer program to enable the device to execute the news recommendation method based on federated learning described in Embodiment 1.

[0092] Optionally, the above device may be a server.

[0093] In addition, this embodiment further provides a medium, which stores a computer program, and when the computer program is executed by a processor, it implements the news recommendation method based on federated learning described in Embodiment 1.

[0094] The various embodiments in this specification are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. For the same or similar parts among the various embodiments, reference can be made to each other. For the system disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple. For related parts, reference can be made to the description in the method section.

[0095] Specific examples are used in this article to elaborate on the principles and implementation manners of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those of ordinary skill in the art, based on the idea of the present invention, there will be changes in the specific implementation manners and application scopes. In summary, the content of this specification should not be construed as a limitation on the present invention.

Claims

1. A news recommendation method based on federated learning, characterized in that, Including: For clients within a set range, extract news features from the local historical clicked news of each client through a news model; For clients within a set range, extract behavioral features from the auxiliary data of each client through a behavioral feature model; Based on the news features and behavioral features of each client received by the server, use a clustering algorithm to cluster the clients to obtain k clusters; each cluster includes multiple clients; In each cluster, train a user model using a news feature dataset to obtain a trained user model corresponding to each cluster; each cluster corresponds to a news feature dataset, and each sample data in the news feature dataset includes news features and user representations corresponding to the news features. The trained user model is used to output user representations based on the input news features; Aggregate the trained user models corresponding to each cluster and distribute the aggregated user model to each cluster; Extract news features from candidate news through a news model; Input the news features extracted from the candidate news into the aggregated user models in each cluster to obtain k predicted user representations; Input the predicted user representations in each cluster and the news features extracted from the candidate news into a click predictor to obtain k click scores; Distribute the candidate news to the clients in the cluster corresponding to the click score exceeding the set value.

2. The news recommendation method based on federated learning according to claim 1, wherein The news features include the click probabilities of various types of news.

3. The news recommendation method based on federated learning according to claim 1, wherein The auxiliary data includes shopping records, website browsing records, and video viewing records.

4. The news recommendation method based on federated learning according to claim 1, characterized in that, The clustering algorithm is the K-Means algorithm.

5. The news recommendation method based on federated learning according to claim 1, characterized in that The step of using a clustering algorithm to cluster the clients based on the news features and behavioral features of each client received by the server to obtain k clusters specifically includes: Distinguish each client into a first client and a second client. The first client is a client with news features, and the second client is a client without news features; Based on the news features of each first client, use a clustering algorithm to cluster each first client to obtain k clusters; For the i-th second client, calculate the similarity between the behavioral features of the i-th second client and the behavioral features of each first client, and divide the i-th second client into the cluster where the first client with the highest similarity value is located.

6. A news recommendation system based on federated learning, characterized in that, Including: A historical news feature extraction module for extracting news features from the local historical clicked news of each client through a news model for clients within a set range; A historical behavioral feature extraction module for extracting behavioral features from the auxiliary data of each client through a behavioral feature model for clients within a set range; A client clustering module for clustering clients using a clustering algorithm based on the news features and behavioral features of each client received by the server to obtain k clusters; each cluster includes multiple clients; A user model training module, which is used to train a user model in each cluster by using a news feature dataset to obtain a trained user model corresponding to each cluster; each cluster corresponds to a news feature dataset, and each sample data in the news feature dataset includes a news feature and a user representation corresponding to the news feature. The trained user model is used to output a user representation according to the input news feature; A user model aggregation module, which is used to aggregate the trained user models corresponding to each cluster and send the aggregated user model to each cluster; A module for extracting news features from candidate news, which is used to extract news features from candidate news through a news model; A module for obtaining user representations to be predicted, which is used to input the news features extracted from candidate news into the aggregated user models in each cluster to obtain k user representations to be predicted; A click prediction module, which is used to input the user representations to be predicted in each cluster and the news features extracted from candidate news into a click predictor to obtain k click scores; A candidate news distribution module, which is used to distribute candidate news to the clients in the clusters where the click scores exceed a set value.

7. A device, characterized in that, It includes a memory and a processor. The memory is used to store a computer program, and the processor runs the computer program to enable the device to execute the news recommendation method based on federated learning according to any one of claims 1 to 5.

8. A medium, characterized in that, It stores a computer program, and when the computer program is executed by the processor, it implements the news recommendation method based on federated learning according to any one of claims 1 to 5.

Images