Grouping federal recommendation method based on bilateral additive article embedding

Abstract

The invention provides a grouping federal recommendation method based on bilateral additive article embedding. A central server constructs a double-layer article embedding characterization structure under a federated learning framework: a client locally maintains user personalized embedding and an article local personalized embedding matrix, and a server generates global group shared article embedding through a dynamic clustering grouping mechanism; superposing global sharing embedding and local personalized embedding by adopting an additive fusion strategy to generate user side personalized article characterization; a progressive course learning scheme is designed, and smooth transition from complete personalization to additive representation is realized by dynamically adjusting a regularization weight coefficient; and a grouping and clustering process is optimized in combination with a knowledge migration strategy, and collaborative knowledge sharing across user groups is promoted. In a client local training stage, a personalized recommendation loss function based on binary cross entropy is constructed, global shared embedding and local embedding parameters are synchronously updated, and a server side updates a global model through grouping federation aggregation. On the premise of protecting user privacy, the problems that in traditional federated recommendation, article characterization is simplified, and personalized perception is insufficient are effectively solved, the accuracy of a recommendation system is remarkably improved, communication overhead is reduced, and the method is suitable for personalized recommendation services of privacy sensitive scenes such as e-commerce and content platforms.

Description

Technical Field

[0001] The present invention relates to the technical field of federated recommendation methods, and particularly relates to a grouped federated recommendation method based on bilateral additive item embedding. Background Art

[0002] Promoting knowledge exchange between heterogeneous clients without violating user privacy has been a key research direction in academia in recent years, which has given rise to the emerging field of federated recommendation systems. Existing federated recommendation models usually share a unified item embedding among clients while keeping user embeddings private locally. However, a single global item embedding can neither capture the personalized perception differences of users towards the same item nor reflect the common preference characteristics of different user groups, thus limiting the recommendation effect.

[0003] Federated recommendation systems mainly process client data from individual users to construct user portraits. In this framework, user portraits and rating data are stored locally on the client side, and the server only stores candidate item information. To balance communication cost and model accuracy while complying with the constraints of federated learning and protecting user privacy, federated recommendation systems need to strike a balance between communication cost and model accuracy to produce optimal recommendation results. Recently, many studies have been dedicated to addressing these challenges. Most methods adopt a partial model sharing design, that is, item embeddings are globally shared and trained through existing federated learning processes, while user functions / embeddings are trained locally and kept private. However, these methods ignore the heterogeneity of users' perception of the same item, that is, different users may have different preferences for the same item and may focus on different attributes of the item. Although PFedRec considers personalized item embeddings, it ignores the importance of knowledge sharing and collaborative filtering among users through global item embeddings. In addition, federated learning of item embeddings requires transmitting dense matrices between clients and servers, which brings a huge communication overhead, especially for users interested in multiple items. Summary of the Invention

[0004] Aiming at the above deficiencies, the present invention proposes a grouped federated recommendation method based on bilateral additive item embedding. This method follows the horizontal federated learning assumption, that is, user embeddings and datasets are different but items are shared. Specifically, the main innovations of the present invention include: (1) Designing a two-layer additive personalized architecture: In addition to user private embeddings, item embeddings include a user-specific local embedding matrix D (i) , and a global group embedding matrix C. By means of a dynamic grouping mechanism, the differential perception of different user groups towards items is captured, so as to provide a more fine-grained item representation at the global level. (2) Introducing a regularization mechanism: Strengthening C and D (i)The complementarity between them ensures that representations at different levels can capture features from different aspects. (3) Design a progressive learning strategy: Considering that additive personalization in the early stage of training may affect performance due to the time-varying nature and overlap of C and D (i) 's time-varying nature and overlap, a curriculum learning scheme is adopted to gradually increase the regularization weight, achieving a smooth transition from full personalization to additive personalization.

[0005] The process of training the grouped federated recommendation method based on bilateral additive item embeddings includes:

[0006] S1: The central server initializes the global item embedding matrix, and the client locally initializes the user embedding vector and the local item personalized embedding matrix;

[0007] S2: Based on the dynamic clustering mechanism, the server gradually divides the clients into multiple user group sets during the training process. Each set maintains a global group-shared item embedding, which is additively fused with the local item embeddings of the clients to generate personalized item representations;

[0008] S3: During the local training process of the client, a joint optimization objective including a binary cross-entropy loss function and a difference regularization term is constructed. At the same time, a progressive curriculum learning scheme is designed to dynamically adjust the coefficient to control the fusion ratio of the global shared embedding and the local embedding, and synchronously update the user embedding, local item embedding, and global shared embedding parameters;

[0009] S4: Generate user-item rating predictions based on the additively fused item representations and output personalized recommendation results.

[0010] Preferably, in the S1 initialization stage, the central server randomly initializes the global item embedding matrix, where the global item embedding matrix C includes the benchmark embedding C base and the grouped embedding The benchmark embedding C base is used for training clients that have not been grouped yet, and the grouped embedding C s is used for training clients that have already been grouped. The initialization parameters follow a normal distribution with a mean of 0 and a standard deviation of 0.01; each client locally initializes the user embedding vector u i and the local item personalized embedding matrix D (i) , where u i and D (i) 's initialization range is limited to a uniform distribution in [-0.05, 0.05], and the storage and update process of the user embedding u i and the local item embedding D (i) are completely completed locally on the client device without transmitting to the server;

[0011] Preferably, from the perspective of the S2 server, the dynamic clustering mechanism requires 1 to t rounds of training, and each round of steps includes:

[0012] S21: The server selects random clients from the clients to participate in the current training and sends the global item embedding matrix C to the clients, which depends on whether the current client has been grouped: If it is the first time to participate in training, the baseline embedding C is sent. base ; Otherwise, the embedding C of the belonging group is sent. s . The selected clients use the global item embedding matrix C and the private item personalized embedding matrix D (i) to perform additive fusion and train on the local data. After training, the new grouped item embedding is uploaded to the server;

[0013] S22: The server calculates the client embedding update difference calculates the cosine similarity between entities, and updates the similarity matrix α (t) , which is used to merge similar entities;

[0014] S23: Update each grouped item embedding further realizes knowledge sharing between different groups by introducing knowledge transfer Prox(C new ) between group embeddings;

[0015] S24: The server updates the baseline embedding C base according to the existing group embeddings. When the client performs local updates in the next round, it can use the updated global item embedding matrix;

[0016] S25: Repeat steps S21 to S25 until no new client groups are generated for N consecutive rounds.

[0017] Further, the formula for calculating the client model update difference in S22 is:

[0018]

[0019] where represents the grouped item embedding uploaded by client i to the server in the t-th round, and C (t-1) represents the global item embedding matrix sent by the server to the client in the (t-1)-th round.

[0020] The formula for calculating the cosine similarity between clients is:

[0021]

[0022] where The similarity metric between client i and client j in the r-th round, <·,·> represents the inner product of two vectors, and ∥·∥ represents the Euclidean norm (L2 norm) of the vector.

[0023] Construct the similarity matrix α (r) , α (r) is a symmetric matrix, where the element represents the similarity between client i and client j in the r-th round.

[0024] Furthermore, in S23, the grouped embedding update formula is:

[0025]

[0026] where M s is the set of clients participating in the current round of training in the current group s, and n i is the number of local data samples of client i.

[0027] The inter-group knowledge transfer formula is:

[0028]

[0029] where C old and C new are all group model parameters before and after proximal optimization respectively; ∥·∥ F represents the Frobenius norm, which is used to measure the difference between two matrices and calculate the magnitude of the model parameter update; λ is the regularization parameter, and g(·) is the regularization function, usually choosing the L2 norm.

[0030] Furthermore, in S24, the update of the benchmark embedding C base . The global model update formula is:

[0031]

[0032] where represents the total number of data samples of all clients in group s, represents the total number of data samples of all clients in all groups.

[0033] Preferably, during the local training process of the client in S3, it is necessary to minimize the reconstruction error between the actual score r i and the predicted score This is the main optimization goal of the model. The binary cross-entropy loss function is used to measure the difference between the predicted score and the true score. This loss function is particularly suitable for processing binary scoring data and can effectively capture user preferences. The loss function is:

[0034]

[0035] To ensure that the item information learned by D (i) and C on the i-th client is different, enforce the difference between them:

[0036]

[0037] In the early stage of learning, the item information learned by and C may not be sufficient for recommendation, resulting in the additive personalization may degrade the performance. Therefore, design a progressive curriculum learning scheme, dynamically adjust the regularization weight coefficient, control the fusion ratio of the globally shared embedding and the local embedding, and achieve a smooth transition from fully personalized to additive representation, and finally the complete optimization objective can be expressed as:

[0038]

[0039] where, is the dynamically adjusted weight coefficient, a is the current training round, a0 is the preset weight growth period, and v is the weight upper limit. This progressive weight adjustment strategy can achieve a smooth transition from pure personalization to additive personalization and improve the training stability of the model.

[0040] Preferably, in S4, the scoring prediction function is defined as:

[0041]

[0042] where, u i is the user embedding, σ(x) = 1 / (1 + e -x ) is used as the sigmoid activation function to map the predicted score of the i-th client to the interval [0, 1], and (D (i) + C) j represents the combined embedding of the j-th item, and the inner product <·> reflects the similarity or matching degree between the user and the item. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 is the system architecture diagram of the present invention from the perspective of client i;

[0044] Figure 2 is the training flow chart of the present invention from the perspective of the server. DETAILED DESCRIPTION OF THE INVENTION

[0045] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0046] A group federated recommendation method based on bilateral additive item embedding, as shown in the embodiments Figure 1 below, includes the following steps:

[0047] Step S1, the central server initializes the global item embedding matrix, and the client locally initializes the user embedding vector and the local item personalized embedding matrix;

[0048] Specifically, it includes two parts: server-side initialization and client-side initialization:

[0049] (1) The central server randomly initializes the global item embedding matrix, where the global item embedding matrix C includes the baseline embedding C base and the grouped embedding The baseline embedding C base is used for training clients that have not been grouped, and the grouped embedding C s is used for training clients that have completed grouping. The initialization parameters follow a normal distribution with a mean of 0 and a standard deviation of 0.01.

[0050] (2) Each client initializes the user embedding vector u i and the local item personalized embedding matrix D (i) on the local device, where the initialization ranges of u i and D (i) are limited to a uniform distribution in [-0.05, 0.05], and the storage and update processes of the user embedding u i and the local item embedding D (i) are completely completed locally on the client and are not transmitted to the server.

[0051] Step S2, as shown in the embodiments Figure 2 below. Based on the dynamic clustering mechanism, the server gradually divides the clients into multiple user group groups during the training process. Each group maintains a global group-shared item embedding, and generates a personalized item representation through additive fusion with the local item embedding of the client.

[0052] Specifically, it includes the following steps that are carried out in sequence:

[0053] S21: The server selects a random client from the clients to participate in the current training and sends the global item embedding matrix C to the client, which depends on whether the current client has been grouped: If it is the first time to participate in the training, the baseline embedding C is sent base ; Otherwise, the embedding C of the group to which it belongs is sent s . The selected client uses the global item embedding matrix C and the private item personalized embedding matrix D (i) to perform additive fusion and train on the local data. After training, the new grouped item embedding is uploaded to the server;

[0054] S22: The server calculates the client embedding update difference calculates the cosine similarity between entities, and updates the similarity matrix α (t) , which is used to merge similar entities;

[0055] Specifically, calculate the client model update difference The formula is:

[0056]

[0057] where represents the grouped item embedding uploaded by client i to the server in the t-th round, and C (t-1) represents the global item embedding matrix sent by the server to the client in the (t - 1)-th round.

[0058] Calculate the cosine similarity between clients The formula is:

[0059]

[0060] where is the similarity measure between client i and client j in the r-th round, <·,·> represents the inner product of two vectors, and ∥·∥: represents the Euclidean norm (L2 norm) of the vector.

[0061] Construct the similarity matrix α (r) , α (r) is a symmetric matrix, where the element represents the similarity between client i and client j in the r-th round.

[0062] S23: Update each grouped item embedding Furthermore, by introducing knowledge transfer Prox(C new ) between the group embeddings, knowledge sharing between different groups is achieved;

[0063] Specifically, the grouped embedding update formula is:

[0064]

[0065] Among them, M s is the set of clients participating in the current round of training in the current group s, and n i is the number of local data samples of client i.

[0066] The formula for knowledge transfer between groups is:

[0067]

[0068] Among them, C old and C new are all group model parameters before and after proximal optimization respectively; ∥·∥ F represents the Frobenius norm, which is used to measure the difference between two matrices and calculate the magnitude of the update of the model parameters; λ is the regularization parameter, and g(·) is the regularization function, usually choosing the L2 norm.

[0069] S24: The server updates the benchmark embedding C base according to the existing group embeddings currently. The client can use the updated global item embedding matrix during the next round of local update;

[0070] Specifically, the update of the benchmark embedding C base . The global model update formula is:

[0071]

[0072] Among them, represents the total number of data samples of all clients in group s, represents the total number of data samples of all clients in all groups.

[0073] S25: Repeat steps S21 to S25 until no new client groups are generated for N consecutive rounds.

[0074] In step S3, during the local training process of the client, construct a joint optimization objective that includes a binary cross-entropy loss function and a difference regularization term, and at the same time design a progressive curriculum learning scheme to dynamically adjust the coefficient to control the fusion ratio of the global shared embedding and the local embedding, and synchronously update the user embedding, local item embedding, and global shared embedding parameters.

[0075] Specifically, during the local training process of the client, it is necessary to minimize the reconstruction error between the actual score r i and the predicted score based on the scored items. This is the main optimization objective of the model. The binary cross-entropy loss function is used to measure the difference between the predicted score and the true score. This loss function is particularly suitable for processing binary score data and can effectively capture user preferences. The loss function is:

[0076]

[0077] To ensure that the item information learned by D (i) and C on the i-th client is different, force the difference between them:

[0078]

[0079] In the early stage of learning, the item information learned by and C may not be sufficient for recommendation, resulting in the possibility that additive personalization may reduce performance. Therefore, a progressive curriculum learning scheme is designed to dynamically adjust the regularization weight coefficient, control the fusion ratio of the globally shared embedding and the local embedding, and achieve a smooth transition from full personalization to additive representation, and finally the complete optimization objective can be expressed as:

[0080]

[0081] where, is the dynamically adjusted weight coefficient, a is the current training round, a0 is the preset weight growth period, and v is the weight upper limit. This progressive weight adjustment strategy can achieve a smooth transition from pure personalization to additive personalization and improve the training stability of the model.

[0082] Step S4, generate a user-item rating prediction based on the additively fused item representation, and output a personalized recommendation result.

[0083] Specifically, the rating prediction function is defined as:

[0084]

[0085] where, u i is the user embedding, σ(x) = 1 / (1 + e -x ) is used as the sigmoid activation function to map the predicted rating of the i-th client to the interval [0, 1], (D (i) + C) j represents the combined embedding of the j-th item, and the inner product <·> reflects the similarity or matching degree between the user and the item.

[0086] Although the embodiments of the present invention have been shown and described, for those of ordinary skill in the art, it can be understood that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principles and spirit of the present invention, and the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A grouped federated recommendation method based on bilateral additive item embedding, characterized in that It includes the following steps: S1: The central server initializes the global item embedding matrix, and the client locally initializes the user embedding vector and the local item personalized embedding matrix; S2: Based on the dynamic clustering mechanism, the server gradually divides the clients into multiple user group sets during the training process. Each set maintains a global group-shared item embedding, and generates a personalized item representation through additive fusion with the local item embedding of the client; S3: During the local training process of the client, a joint optimization objective including a binary cross-entropy loss function and a difference regularization term is constructed. At the same time, a progressive curriculum learning scheme is designed to dynamically adjust the coefficient, control the fusion ratio of the global shared embedding and the local embedding, and synchronously update the user embedding, the local item embedding, and the global shared embedding parameters; S4: Generate user-item rating predictions based on the item representation after additive fusion, and output personalized recommendation results.

2. The group federated recommendation method based on bilateral additive item embedding according to claim 1, wherein In the S1 initialization stage, the central server randomly initializes the global item embedding matrix, where the global item embedding matrix C contains the baseline embedding C base and the grouped embedding The baseline embedding C base is used for the training of clients that have not been grouped, and the grouped embedding C s is used for the training of clients that have completed grouping. The initialized parameters follow a normal distribution with a mean of 0 and a standard deviation of 0.01; each client initializes the user embedding vector u i and the local item personalized embedding matrix D (i) locally on its own device, where u i and D (i) are initialized within the range of [-0.05, 0.05] with a uniform distribution, and the storage and update processes of the user embedding u i and the local item embedding D (i) are completely completed locally on the client side without being transmitted to the server.

3. A grouped federated recommendation method based on bilateral additive item embedding according to claim 1, characterized in that In S2, the dynamic clustering mechanism requires 1... t rounds of training, and each round of steps includes: S21: The server selects a random client from the clients to participate in the current training and sends the global item embedding matrix C to the client, which depends on whether the current client has been grouped: If it is the first time to participate in the training, the baseline embedding C is sent base ; Otherwise, the embedding C of the group to which it belongs is sent s . The selected client uses the global item embedding matrix C and the private item personalized embedding matrix D (i) to perform additive fusion and train on the local data. After training, the new grouped item embedding is uploaded to the server; S22: The server calculates the embedding update difference of the client Calculate the cosine similarity between entities and update the similarity matrix α (t) , which is used to merge similar entities; S23: Update each grouped item embedding Further introduce knowledge transfer Prox(C new ) between group embeddings to achieve knowledge sharing between different groups; S24: The server updates the reference embedding C based on the existing group embeddings. base The client can use the updated global item embedding matrix during the next round of local updates. S25: Repeat steps S21 to S25 until no new client groups are generated for N consecutive rounds.

4. The group federated recommendation method based on bilateral additive item embedding according to claim 3, wherein, Calculate the difference in the client model update in S22 The formula is: Among them, represents the grouped item embedding uploaded by client i to the server in the t-th round, C (t-1) represents the global item embedding matrix sent by the server to the client in the (t - 1)-th round. Calculate the cosine similarity between clients The formula is: Among them, The similarity metric between client i and client j in the r-th round, <·,·> represents the inner product of two vectors, and ∥·∥ represents the Euclidean norm (L2 norm) of the vector. Construct the similarity matrix α (r) , α (r) is a symmetric matrix, where the element represents the similarity between client i and client j in the r-th round.

5. A grouped federated recommendation method based on bilateral additive item embedding according to claim 3, characterized in that In S23, the formula for updating the grouped embedding is: Among them, M s is the set of clients participating in this round of training in the current group s, and n i is the number of local data samples of client i. The formula for knowledge transfer between groups is: Among them, C old and C new are all sets of model parameters before and after proximal optimization, respectively; ∥·∥ F denotes the Frobenius norm, which is used to measure the difference between two matrices and calculate the magnitude of the model parameter update; λ is the regularization parameter, and g(·) is the regularization function, usually the L2 norm is chosen.

6. The group federated recommendation method based on bilateral additive item embedding according to claim 3, characterized in that, In S24, the update of the reference embedding C base is as follows. The global model update formula is: Among them, represents the total number of data samples of all clients in group s, represents the total number of data samples of all clients in all groups.

7. A grouped federated recommendation method based on bilateral additive item embedding according to claim 1, wherein During the local training process of the S3 client, it is necessary to minimize the actual score r based on the scored items i and the predicted score The reconstruction error between them is the main optimization goal of the model. The binary cross-entropy loss function is used to measure the difference between the predicted score and the true score. This loss function is particularly suitable for processing binary score data and can effectively capture user preferences. The loss function is as follows: To ensure that the item information learned by D (i) and C on the i-th client is different, enforce the differences between them: In the early stage of learning, the item information learned by and C may not be sufficient for recommendation, resulting in the possibility that additive personalization may degrade performance. Therefore, a progressive curriculum learning scheme is designed to dynamically adjust the regularization weight coefficient, control the fusion ratio of the globally shared embedding and the local embedding, achieve a smooth transition from full personalization to additive representation, and finally the complete optimization objective can be expressed as: Among them, is a dynamically adjusted weight coefficient, a is the current training round, a0 is a preset weight growth period, and v is the weight upper limit. This progressive weight adjustment strategy can achieve a smooth transition from pure personalization to additive personalization and improve the training stability of the model.

8. A grouped federated recommendation method based on bilateral additive item embedding according to claim 1, characterized in that In S4, the rating prediction function is defined as: where, u i is the user embedding, σ(x) = 1 / (1 + e -x ) serves as the sigmoid activation function to map the predicted score of the i-th client to the interval [0, 1], (D (i) + C) j represents the combined embedding of the j-th item, and the inner product <·> reflects the similarity or matching degree between the user and the item.

Images