An efficient lightweight federated recommendation method

Abstract

The application discloses a kind of high-efficiency lightweight federal recommendation methods, comprising: server initialization article continuous value embedding matrix and article bias vector, by binary processing to obtain binary article matrix and distribute to client;Client is based on local data, by difficult negative sample mining to construct training triple, adopts the hybrid loss function combined with sorting loss, reconstruction loss and regularization loss to carry out local training, updates user parameter and calculates article gradient;Server aggregates the gradient uploaded by client to update global parameter.The application enhances model expression ability by introducing bias term, optimizes sorting performance using hybrid loss function, improves training efficiency by combining difficult negative sample mining, significantly improves recommendation accuracy while retaining the privacy protection advantages of federal learning, while significantly reducing client storage, communication and computing overhead, especially suitable for resource-constrained mobile terminal deployment.

Description

Technical Field

[0001] This invention belongs to the field of intelligent recommendation technology, and in particular relates to an efficient and lightweight federated recommendation method. Background Technology

[0002] With the implementation of regulations such as the General Data Protection Regulation (GDPR) and increased user awareness of privacy, traditional centralized data processing models face significant challenges in the recommender system field. Federated learning offers an effective privacy-preserving solution, allowing a global recommender model to be collaboratively trained by aggregating model updates from various clients (users) while user data remains on local devices (such as mobile phones and smart home devices). While protecting user privacy, this model also places stringent demands on the resources of the devices deployed on the clients, particularly in terms of storage space, network bandwidth, and computing power.

[0003] To address the challenges of resource constraints, the design of Federated Recommender Systems (FRS) has gradually shifted towards lightweight design. Existing technologies can be broadly categorized into two main types:

[0004] The first category is federated recommendation methods based on traditional continuous-valued latent factor models. These methods typically maintain a global, high-dimensional continuous-valued (i.e., floating-point) latent feature matrix for items on the server side. In each training round, the server distributes this complete item feature matrix to selected clients. The clients locally use this matrix and their own user data to update their private user latent feature vectors and upload the updated gradients or model parameters to the server for aggregation. However, this type of method has a significant drawback: enormous communication overhead. In real-world applications, such as e-commerce platforms with millions of products or streaming services with massive amounts of video, the item feature matrix can reach gigabyte levels. Requiring users' mobile devices to download such massive amounts of data in each training round not only consumes significant network bandwidth and user data but also severely slows down the training process due to transmission latency, which is unacceptable for mobile applications.

[0005] The second category comprises lightweight federated recommendation methods based on discretization or binarization techniques, proposed to address the aforementioned communication bottlenecks. Examples include Learning to Hash (L2H) techniques, exemplified by LightFR. These methods convert high-dimensional continuous-value user and item feature vectors into extremely compact binary codes (e.g., 64-bit or 128-bit binary sequences) using specific algorithms. This conversion offers significant advantages: firstly, communication and storage overhead are drastically reduced because transmitting and storing binary codes is far more efficient than using floating-point matrices; secondly, during recommendation prediction on the client side, complex floating-point inner product operations can be replaced by efficient bitwise operations (such as Hamming distance calculation), greatly improving inference speed and reducing power consumption.

[0006] While the second type of lightweight method has achieved a significant breakthrough in efficiency, the pursuit of ultimate efficiency has also introduced new technical shortcomings, resulting in recommendation accuracy that cannot meet the needs of practical applications. Specifically, its shortcomings are mainly reflected in the following three aspects:

[0007] (1) The prediction model is oversimplified and ignores key bias information: Existing lightweight models typically rely solely on the interaction between user and item binary codes (such as Hamming distance) to predict user preferences. This model ignores the inherent bias information that is prevalent in recommendation scenarios. For example, some users tend to give high scores (user bias), while some items are generally popular due to their inherent popularity (item bias). The lack of modeling for this bias information can lead to a bias in the model's understanding of user preferences, thus limiting the upper limit of recommendation accuracy.

[0008] (2) Mismatch between optimization objective and actual task: Most current lightweight methods use "point-wise reconstruction loss" as the optimization objective, that is, to make the model's predicted rating as close as possible to the user's actual rating (e.g., 4 or 5 points). However, in the practical application of modern recommender systems, the core task is usually "Top-N" ranking recommendation, that is, to provide the user with a list of items with the best ranking, and the absolute rating value of the list is not so important. Optimizing only the accuracy of the rating does not guarantee that the ranking of the recommended list is optimal, resulting in poor performance of the model on key ranking evaluation metrics (such as NDCG).

[0009] (3) Low training efficiency and slow model convergence: To train the ranking model, it is necessary to match uninteracted "negative samples" with each user's interacted "positive samples". Most existing methods use a simple uniform random sampling strategy to select negative samples. In the large pool of uninteracted items, most items are obvious "simple negative samples" to users, and the model can easily rank them correctly. Training on these samples will hardly generate effective learning signals (gradients), thus wasting a lot of computational resources, resulting in low model learning efficiency, slow convergence speed, and difficulty in learning highly discriminative feature representations. Summary of the Invention

[0010] To address the aforementioned technical problems, this invention proposes a highly efficient and lightweight federated recommendation method to resolve the issues present in the prior art.

[0011] To achieve the above objectives, this invention provides an efficient and lightweight federated recommendation method, comprising:

[0012] The server initializes the item continuous value matrix and the item bias vector; based on the item continuous value matrix, it obtains a binarized item matrix through binarization; the server distributes the binarized item matrix to several clients;

[0013] Each client initializes its local user binary vector, user bias vector, and item bias vector based on the binarized item matrix. Combining local data, it constructs training triples using the hard negative sample mining module. Based on the training triples, it trains the local model using a hybrid loss function to update the local user binary vector, user bias vector, and item bias vector, and calculates the item gradient matrix. Each client then uploads the calculated item gradient matrix to the server.

[0014] The server aggregates and updates the global item continuous value matrix and item bias vector based on the item gradient matrix uploaded by each client, completing one round of client updates. Each client then recommends items based on the updated model.

[0015] Optionally, the steps of the server initializing the item continuous value matrix and the item bias vector include: randomly initializing the item continuous value matrix; and initializing the item bias vector as a zero vector.

[0016] Optionally, a sign function can be applied to the continuous value matrix of items to generate a binarized item matrix.

[0017] Optionally, the steps for constructing training triples using the hard negative sample mining module include:

[0018] For each user's positive sample items, candidate negative samples are randomly sampled from the negative sample set; based on the current model's predicted score, the item with the highest predicted score is selected as the difficult negative sample from the candidate negative samples; training triples are constructed based on the user, positive sample items, and difficult negative samples.

[0019] Optionally, obtain the local user's binary vector and the binary item matrix and calculate the inner product. Summate the inner product with the user's bias vector and the item's bias vector to obtain the user's predicted rating for the candidate items.

[0020] Optionally, the hybrid loss function is composed of a weighted combination of ranking loss, reconstruction loss, and regularization loss;

[0021] The formula for the hybrid loss function is as follows:

[0022] ;

[0023] In the formula, For ranking loss, To reconstruct the loss, This is the regularization loss; These are the weighting coefficients for the mixed loss.

[0024] Optionally, the ranking loss is a pairwise ranking loss. For each training triplet, the difference between the user's predicted rating for positive sample items and the predicted rating for difficult negative samples is calculated. The difference in predicted ratings is then transformed nonlinearly using the Sigmoid function. The natural logarithm of the transformed result is taken to obtain the final ranking loss. The average of the results for all training triplets is then taken as the negative value to obtain the final ranking loss.

[0025] The sorting loss is as follows: ;

[0026] The basic score prediction formula is as follows: In hard negative sample mining, when constructing ranked triples, the predicted score of candidate negative samples is calculated as follows: The difference in scores between positive and negative samples is calculated using the following formula: ;

[0027] Among them, specific users Specific items , For user collection, Collection of items For users The real-valued embedding vector; Positive sample items The real-valued embedding vector; For the user bias vector, Positive sample Item bias vector; Let p be the real-valued embedding vector of the negative sample item p; Let p be the item bias vector for the negative sample.

[0028] Optionally, the reconstruction loss is a point-by-point mean square error loss, which is obtained by calculating the mean square of the difference between the observed score and the predicted score;

[0029] The reconstruction loss is as follows: ;

[0030] in, For the observed user-item rating pairs, For the rating matrix The elements in represent users For items The score.

[0031] Optionally, calculate the sum of squares of the L2 norm of all user latent factor vectors, the sum of squares of the L2 norm of all item latent factor vectors, the sum of squares of all user bias terms, and the sum of squares of all item bias terms; then add the four sums of squares obtained and multiply by the regularization strength parameter to obtain the final regularization loss value.

[0032] The regularization loss is as follows: ;

[0033] Among them, specific users Specific items , For user collection, Collection of items For regularization parameters; For users The real-valued embedding vector; Positive sample items The real-valued embedding vector; For the user bias vector, Positive sample The item bias vector.

[0034] Compared with the prior art, the present invention has the following advantages and technical effects:

[0035] This invention, by introducing learnable user and item biases into the prediction model, can more accurately capture users' inherent rating tendencies and the global popularity of items. This technical feature separates these baseline effects from complex user-item interactions, allowing core latent factors to focus on modeling more subtle and personalized preferences. Through this approach, this application enhances the model's expressive power, thereby improving the overall accuracy of recommendation predictions.

[0036] By designing a hybrid loss function that combines pairwise ranking loss (aimed at optimizing item ranking) and pointwise reconstruction loss (aimed at restoring ratings), the model training objective is directly aligned with Top-N recommendation tasks in real-world applications. Pairwise ranking loss ensures that items liked by users are predicted to rank higher than those disliked; pointwise reconstruction loss preserves the perception of user preference levels (e.g., the difference between 5-star "very much liked" and 4-star "somewhat liked"). This application, through this hybrid optimization objective, not only focuses on the accuracy of rating predictions but also directly optimizes the ranking quality of the recommendation list, achieving significant improvements, particularly in ranking-sensitive key evaluation metrics such as NDCG (Normalized Discount Cumulative Gain).

[0037] By introducing a challenging negative sample mining strategy, this approach intelligently selects the most challenging and easily confused negative samples from a massive pool of non-interactive items for each positive sample during model training. This avoids inefficient computation on a large number of "simple negative samples," concentrating the model's learning resources on the most informative samples. Through this strategy, the application forces the model to learn more discriminative feature representations, significantly accelerating the model's convergence speed and further enhancing its final recommendation performance.

[0038] This invention can overcome the shortcomings of existing lightweight models, such as model simplification, target mismatch, and inefficient training, while maintaining the advantages of low storage, low communication, and high inference efficiency. It can significantly improve the accuracy of recommendations and the training efficiency of models. Attached Figure Description

[0039] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:

[0040] Figure 1 This is a diagram illustrating the algorithm framework of an embodiment of the present invention;

[0041] Figure 2 This is a graph showing the model training convergence curve in an embodiment of the present invention.

[0042] Figure 3 This is a schematic diagram of the model lightweighting index in an embodiment of the present invention. Detailed Implementation

[0043] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0044] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0045] Example 1

[0046] like Figure 1 As shown, this embodiment provides an efficient and lightweight federated recommendation method, including:

[0047] I. Improved lightweight federated recommender system framework;

[0048] First, the continuous value matrix and item bias vector of each item are initialized. These are then updated using an enhanced latent factor model. The updated continuous value matrix is ​​binarized to obtain a binarized item matrix. The server distributes the binarized item matrix to distributed clients for local updates. Subsequently, each local client receives the latest binarized item matrix and initializes its own private binary vector in the first round. and bias vector and item bias Each client participates in the hybrid loss training strategy by forming a triplet through the hard negative sample mining module. The hybrid loss training strategy uses a binary item matrix and a binary vector. and bias vector as well as The resulting vectors are updated using the client's local discrete optimization module. and bias vector as well as And calculate the gradient matrix of the items to be uploaded. After receiving the gradient matrices uploaded by all selected clients, the server updates them using the global discrete aggregation module. The latest item binary matrix is ​​then distributed to each client for the next step of local optimization.

[0049] Unlike fully centralized recommendation systems, federated recommendation systems struggle to build complete user-item rating matrices on servers. To protect user privacy, it no longer retains the complete observation dataset. We can assume that there exists an independent set of users. and the collection of items stored on the central server. Following the principles of federated learning, each user... Have a local private dataset The dataset consists of some feedback tuples Composition, in which Representative client The observed items. The goal of a federated recommendation system is to predict user behavior. For each unseen or uninteracted item The system then rates the users and recommends the highest-ranked items to the target users. During the federated training process, the gradients for users and items are calculated locally using the following formula.

[0050] ;

[0051] In summary, user embedding vectors Local updates are performed independently and do not depend on any other user's private data, hence the following formula:

[0052] ;

[0053] in, It is the learning rate. Conversely, it is the item... Real-valued embedding vectors It is done on the server by aggregating data from each client. Uploaded local item gradient The formula for global updates is as follows:

[0054] ;

[0055] Within this item gradient It is a local client This is used for calculation. Simultaneously, the updated item matrix... The results are sent to all clients, and the federated process is repeated until the results converge. During the federated testing phase, the clients... Download the latest item embedding matrix from the federated recommendation system server. The score is estimated using the inner product method on the local device. .

[0056] The original system used only a simple binary randomization of the initial matrix and simple dot product prediction, which resulted in a lack of ability to model baseline rating tendencies for users and items, and an inability to handle user rating habits and the general popularity of items. This embodiment introduces user and item bias terms to enhance the training process and provide a richer parameter space. Baseline correction reduces prediction errors caused by different rating habits, and the combined effect of the mixed loss framework allows for a more accurate modeling of the rating composition. The basic rating prediction formula is: In hard negative sample mining, when constructing ranked triples, the predicted score of candidate negative samples is calculated as follows: The difference in scores between positive and negative samples is calculated using the predicted scores described above. Therefore, the ranking loss formula is as follows:

[0057] ;

[0058] The MSE ranking loss formula is obtained by calculating the difference between the score and the predicted score as follows:

[0059] ;

[0060] The bias term also participates in the L2 regularization calculation; the regularization formula for the bias term is:

[0061] .

[0062] User and item preferences should also be preserved through the characteristics between their respective binary codes. Simultaneously, the user-item rating matrix needs to be reconstructed based on this characteristic. Therefore, based on LightFR and combined with FedAvg, the objective function of the LightFR model is proposed as follows:

[0063] ;

[0064] in, It is the total number of instances on all clients. Indicates client The number of local samples. It's important to note that LightFR uses binary constraints, and the matrix factorization formula uses the traditional regularization term form. This becomes a constant and is therefore removed from the objective function formula of the LightFR model. To obtain an information-rich binary representation, a balance constraint is introduced to maximize the information entropy of each bit; this is the constraint term in the objective function formula of the LightFR model. Trade-off parameters. Control the ratio between minimizing the squared loss and the balance constraint.

[0065] To improve model generalization and recommendation quality, this embodiment employs a hybrid loss function. Therefore, the formula for the hybrid loss strategy is as follows:

[0066] ;

[0067] The principles behind each loss function will be explained below.

[0068] The mathematical formula for ranking loss is: ,in It is a set of triples constructed through hard negative sample mining. It is the user. It is a positive sample item. These are difficult negative sample items. It is a biased prediction score. It is the sigmoid function.

[0069] The mathematical formula for MSE reconstruction loss is: ,in It is the set of all observed user-item rating pairs. User For items The actual score.

[0070] The mathematical formula for regularization loss is: This includes latent factors for users, latent factors for items, and L2 regularization for two bias terms: user and item. It is the regularization strength parameter.

[0071] The complete objective loss function obtained by combining the above parts is as follows:

[0072] ;

[0073] in It is used to control the weight balance between ranking loss and MSE loss. Compared with the original, this objective loss function takes into account ranking performance and scoring accuracy, while preventing overfitting through regularization.

[0074] For each user Positive sample items he interacted with Difficult negative samples The choice is:

[0075]

[0076] in, It is a candidate set randomly sampled from all negative samples. The user predicted by the current model For items The rating.

[0077] The implementation first starts from all negative samples Random sampling Candidate items as Then calculate the user Prediction score for all candidate negative samples:

[0078] ;

[0079] Then, select the negative sample with the highest predicted score. This is how triples are constructed. The triplet is ultimately used in the calculation of the mixed loss.

[0080] Difficult negative sample mining allows for greater focus on the decision boundary, targeting samples where the model is most prone to misclassification, and avoids wasting computation time on "easy" negative samples. Challenging samples can provide more informative gradients, accelerating training convergence.

[0081] The pseudocode for the above process is shown in Table 1:

[0082] Table 1

[0083] Algorithm: Enhanced Federated Discrete Optimization Algorithm with Mixed Loss Input: Total number of clients N_A; Total number of items N_O Code length k; Regularization parameter β Global training epochs T; Local training epochs E_local Mixed loss coefficient ω; Number of candidate negative samples M Learning rate α; Batch size S_batch Output: A global item binary matrix E on the server; a local user binary vector c_a on each client a. Server execution: Initialization: Continuous item matrix Y_cont∈R^(k×N_O), random initialization. The item bias vector δ_o∈R^(N_O) is initialized to 0. Optimizer = Adam(parameters = [Y_cont,δ_o], lr = α) Scheduler = ReduceLROnPlateau(optimizer, factor = 0.5, patience = 10) Load the checkpoint if it exists. For each round t=1,..., T do At <- Randomly select a portion of clients for each client a in At in parallel do nabla_E^t, nabla_c^a <- ClientUpdate(Ycont; ca; t) end Aggregate gradients and update parameters using the optimizer. Update the scheduler based on verification loss. Save checkpoints end E <- sign(Ycont) / / Binarize the continuous item matrix Client execution: Function ClientUpdate(Ycont; ca; t): Download the latest Ycont and ca from the central server. Initialize or load user vector xa and bias gamma_a for each epoch e=1,..., E_local do / / Ranking loss with hard negative sample mining Spos <- {o | (a, o) in D_a} / / Positive sample items for user a Sneg <- {p | (a, p) not in D_a} / / Negative sample items for user a for each o in Spos do / / Sampling M candidate negative samples C_a <- Sample M items from Sneg / / Select the hardest negative sample using the current model p <- argmax_{p' in C_a}(xa^T y_p' + gamma_a + delta_p') / / Calculate the sorting gradient of the triplet (a,o,p) diff < - xa^T i + gamma_a + delta_o - (xa^T y_p + gamma_a + delta_p) grad_factor <- sigmoid(-diff) Update xa and gamma_a using sorted gradients. Calculate yo, y_p, delta_o, delta_p end / / Observe the mean squared error loss of the scores for batch(o, sao) in D_a do pred <- xa^T yo + gamma_a + delta_o error <- sao - pred Update xa and gamma_a using reconstructed gradients. Calculate the values ​​of yo and delta_o. end Apply regularization to xa, yo, gamma_a, delta_o end return nabla_Y_cont, nabla_delta_o end

[0084] Within this framework, the list of symbols and their explanations are shown in Table 2.

[0085] Table 2

[0086] symbol explain User (client) collection; Item collection Total number of users; Total number of items The rating matrix, where elements Indicates user For items Rating score Specific users ; Specific items Client Local private datasets in; with items Global feedback set of interactions Client A collection of locally observed objects; with objects Global user set for interaction user Real-valued embedding vectors; positive sample items Real-valued embedding vectors From client item embedding vector gradient User real value embedding matrix; Item real value embedding matrix user A binary vector; an item binary vector User binary vector The Bit encoding; remove User binary vector The remaining codes From client For item binary vectors The gradient of bits User binary embedding matrix; Item binary embedding matrix Client In the The binary matrix of items uploaded in each iteration gradient matrix A subset of clients randomly selected by the coordination server Learning rate; regularization parameter Number of training rounds between server and client; number of local training rounds Proportion of clients participating in the current training round User continuous value embedding matrix (before binarization); Item continuous value embedding matrix (before binarization) User bias vector; positive samples Item bias vector Mixed loss weighting coefficient Ranking loss function; mean squared error loss function Hybrid loss function user For items and Sorted triples Patience parameter of the learning rate scheduler Learning rate scheduler decay factor Number of candidate negative samples user A set of positive sample items (interacted items); user The set of negative sample items (items that have not been interacted with). Index of latent factor dimension Triple set Observed user-item rating combination Latent factor space dimension The real-valued embedding vector of the negative sample item p; the item bias vector of the negative sample p.

[0087] Figure 1 In this context, Global Server refers to the global server; Item Set refers to the item set; Upload Gradients refers to the uploaded gradients; Download Global Model refers to the downloaded global model; Discrete Opt. refers to discrete optimization; Hybrid Loss Function refers to the hybrid loss function; and Hard Negative Sample refers to hard negative sample sampling.

[0088] In this invention, "item" refers to the basic unit or object that a recommendation system recommends to users. It is a broad concept, not limited to physical goods, but encompassing all information entities that can be consumed, interacted with, or used to express user preferences.

[0089] Specifically, "items" can include, but are not limited to, the following categories:

[0090] Media content: such as movies (e.g., movies in the Filmtrust dataset), TV series, music, videos, news articles, books, etc.

[0091] Commercial products: such as electronic products, clothing, household goods (e.g., the products in the Ciao dataset), etc.

[0092] Services or locations: such as restaurants, hotels, tourist attractions, online courses, etc.

[0093] Social content: such as posts, users, and hashtags on social media.

[0094] In the technical solution of this invention, each "item" in the system is not merely a name or ID, but is transformed into a specific set of mathematical representations so that the model can perform calculations and learn. These representations specifically include:

[0095] Item Latent Feature Vector: This is the core representation used during model training. The system learns a high-dimensional vector of continuous values ​​(floating-point numbers) for each item. This vector is not based on explicit attributes of the item (such as movie genre or director), but is automatically learned from massive amounts of user-item interaction data (such as ratings). It captures the item's deep, latent characteristics. For example, two movies with similar styles, even with completely different directors and actors, will have latent feature vectors that are very close in vector space.

[0096] Item Bias Term: This is a learnable scalar value used to represent an item's global popularity or base appeal, independent of any particular user. For example, a critically acclaimed blockbuster film would typically have a high positive item bias term because it is inherently more likely to receive high ratings.

[0097] Item Binary Code: This is a key feature of the lightweight design of this invention. After model training, the continuous value latent feature vector of each item is converted into an extremely compact binary sequence (e.g., 01101001...). This binary code is the final representation used for efficient storage, network transmission, and fast recommendation calculation on client devices. By calculating the Hamming distance between the user's binary code and the item's binary code, the system can quickly perform similarity matching and generate a recommendation list.

[0098] In this invention, the "item" is both the ultimate goal of the recommendation service and the core data entity that is digitally modeled in the entire calculation process through multiple methods such as latent feature vectors, bias terms, and binary codes.

[0099] Taking product recommendations in a mobile e-commerce app as an example, this invention illustrates how it can be applied to a mobile e-commerce app (such as the "XXX Mall" app) to provide efficient and accurate personalized product recommendations while protecting user shopping privacy. In this scenario, "items" refer to the "products" on the platform.

[0100] Step 1: Data Acquisition and Distribution;

[0101] Client-side data (user local data): Each smartphone with the "XXX Mall" app installed constitutes a client. User's personal shopping behavior data, including user ID, viewed product IDs, purchased product IDs, product ratings, and cart addition records, is stored completely locally on the user's phone. This data is not uploaded to a central server. For example, user "Li Si's" phone stores {(Li Si, 'Product A', purchased), (Li Si, 'Product B', 4-star rating),...}.

[0102] Server-side data: The "XXX Mall" server stores global product information, including product ID, product name, price, category, etc. The server does not hold any user's personal shopping history.

[0103] Step 2: Data processing and model initialization;

[0104] Server-side initialization:

[0105] The server initializes the item continuous value matrix and item bias vector: Specifically, the server creates a global item continuous value matrix based on all items in its item library and randomly initializes it; at the same time, it creates a global item bias vector and initializes it to a zero vector.

[0106] Client initialization:

[0107] When user "Li Si's" mobile app participates in federated training for the first time or receives a new round of global model training, it initializes a local user binary vector and user bias vector for "Li Si". During training, these parameters exist as continuous values ​​and are updated locally.

[0108] Step 3: Implement the federally recommended training process;

[0109] The recommended method of this invention is executed iteratively in multiple rounds, and the update steps in each round are as follows:

[0110] Server-based distribution model: The server randomly selects a subset of all clients and distributes the current version of the binarized item matrix (generated by applying a sign function to the continuous item value matrix on the server side) to these selected clients. Continuous value matrices can also be distributed during the initial training phase or specific update phases.

[0111] Local training on the client: Taking user "Li Si's" client as an example, after receiving the model from the server, the following operations are performed in conjunction with its local data:

[0112] (a) Constructing training triples using the hard negative sample mining module:

[0113] The app iterates through the positive sample items of "Li Si" (such as "Item A" that has been purchased or has a high rating).

[0114] Randomly sample candidate negative samples from the negative sample set: that is, randomly select a small portion (e.g., 10) from a large number of products that "Li Si" has never interacted with as candidates.

[0115] Based on the current model's predicted ratings, the item with the highest predicted rating is selected as the difficult negative sample from the candidate negative samples: The app calculates the predicted ratings for "Li Si" on these 10 candidate items. The predicted rating is calculated as follows: obtain the local user's binary vector and the corresponding item's vector in the binary item matrix, calculate the inner product, and sum the inner product with the user bias vector and the item bias vector. The item with the highest predicted rating (e.g., "Item X") is selected as the difficult negative sample.

[0116] Based on the user, positive sample item, and difficult negative sample, construct the training triplet, namely (user "Li Si", positive sample "item A", difficult negative sample "item X").

[0117] (b) Using a hybrid loss function for local model training:

[0118] The app uses the training triples constructed in the previous step to calculate the value of the hybrid loss function. This function is a weighted combination of the ranking loss, reconstruction loss, and regularization loss.

[0119] The ranking loss is calculated by taking the difference between the predicted scores of "Li Si" for "product A" and "product X", and then transforming it with the Sigmoid function and performing subsequent calculations. The purpose is to optimize the ranking of product recommendations.

[0120] The reconstruction loss is calculated by subtracting the difference between the observed rating (such as the actual rating or the implicit rating corresponding to the purchase behavior) of "Li Si" for "goods A" and the predicted rating, with the aim of ensuring the accuracy of the prediction.

[0121] The regularization loss is obtained by calculating the L2 norm of all relevant latent factor vectors and bias terms to prevent model overfitting.

[0122] (c) Update local parameters and calculate the item gradient matrix:

[0123] Based on the calculated hybrid loss, the app updates the local user binary vector (which is in continuous value form during training) and user bias vector using gradient descent.

[0124] Simultaneously, the gradients generated by this training on the global item model parameters (i.e., the continuous value vectors and bias terms of "item A" and "item X") are calculated and accumulated into the local item gradient matrix.

[0125] Client-side gradient upload: After local training is completed, "Li Si's" client uploads the calculated item gradient matrix to the server.

[0126] Server aggregates and updates global parameters:

[0127] The server aggregates and updates the global item continuous value matrix and item bias vector based on the item gradient matrices uploaded by each client. Typically, a weighted average is used to aggregate the gradients uploaded by all clients participating in this training round.

[0128] This completes one round of client updates.

[0129] Step 4: Obtain product recommendation results;

[0130] Final model deployment: After multiple rounds of training, the server obtains the final converged continuous value matrix of items. The server then binarizes this matrix to obtain the final binarized item matrix and distributes it to all clients.

[0131] Each client recommends items based on the updated model:

[0132] When user "Li Si" opens the homepage of "XXX Mall" APP, the APP uses its latest user binary vector (which has also been binarized) and the global binarized item matrix to perform recommendation calculations on its local device.

[0133] The matching degree between "Li Si" and all products is quickly calculated and sorted using efficient bit operations (such as Hamming distance).

[0134] The app will display the top N ranked products, such as "new headphones from a certain brand" or "sports shoes", to "Li Si" in the "Recommended for You" section on the homepage.

[0135] Through the above steps, this invention implements the entire process from data processing and model training to generating recommendation results for the specific object of "product" in the context of mobile e-commerce APP.

[0136] II. Experimental Design and Analysis;

[0137] (1) Introduction to the dataset;

[0138] This embodiment uses two commonly used public datasets of different sizes for experimental analysis: Filmtrust and Ciao datasets.

[0139] The Filmtrust dataset was scraped from the online rating website FilmTrust. The original data contains approximately 35,000 ratings from 1,508 users on 2,071 films, with a data density of approximately 1.14%, averaging 24 ratings per user. The Filmtrust dataset has the lowest interaction volume among commonly used public datasets.

[0140] The Ciao dataset was scraped from the popular product review website Ciao in May 2011. It contains approximately 282,000 ratings from 7,375 users on 105,096 movies. It is one of the sparsest datasets in the public domain, with a density ratio of only 0.04%, and the average number of ratings per user is only 38. Detailed statistical information about the dataset is summarized in Table 3.

[0141] Table 3

[0142]

[0143] Here, Average represents the average number of user ratings. Average reflects the data density of clients in the dataset. Experiments on Filmtrust and Ciao datasets of different sizes and sparsity can comprehensively reflect the model's performance. In the federated setting, each user is treated as a local client, and user data is stored on the local device.

[0144] (2) Evaluation indicators and loss curves;

[0145] (a) Evaluation indicators;

[0146] This embodiment uses the same metrics as the original LightFR model performance evaluation, employing two commonly used metrics: Hit Ratio (HR) and Normalized Discounted Cumulative Gain (NDCG). These two metrics are widely used in item ranking tasks. They are typically truncated at a specific ranking level to emphasize the importance of the first items retrieved.

[0147] HR@k is used to calculate the number of times the test item appears in the pre-tested ranked items: NDCG@k truncates the ranked list at k, measures the quality of the ranking, and assigns a higher score to hits at the top of the ranking because positive items at the bottom of the ranking list contribute less to the final result.

[0148] Intuitively speaking, the HR metric measures whether the tested item appears in the top k ranking list, while the NDCG metric measures the quality of the ranking. Using these two metrics takes into account both the position of the score and the accuracy of the ranking.

[0149] (b) Loss curve;

[0150] In this embodiment, there are two loss types: Current Loss and Delta Loss. Current Loss represents the sum of the cumulative losses of all clients in the current training round. It is calculated by first calculating the local losses of each client and then adding these local losses to the total loss. The value of Current Loss reflects the performance of the current binarized model on the entire dataset. The lower the loss value, the closer the model's prediction is to the actual score. Delta loss represents the difference between the loss of the current round and the loss of the previous round. It is calculated by subtracting the cumulative loss of the previous round from the current cumulative loss. This value is mainly used to monitor the convergence of the model during training and as a condition for early stopping. Delta Loss has two early stopping strategies: Delta Loss is less than a set threshold and Delta Loss is equal to the Delta Loss value of the previous round. The first early stopping strategy means that if the loss hardly decreases anymore, it means that the model has reached a stable state, and the model does not need to continue training to save computational resources. The second strategy means that if the loss changes are roughly the same in two consecutive rounds, the model may be in a state of oscillation. In this situation, the model may jump back and forth between two states, thus failing to achieve further development. Both strategies aim to avoid ineffective training by conserving computational resources by stopping the model training process in a timely manner.

[0151] according to Figure 2 It can be seen that the LGFR proposed in this embodiment has a smaller Current Loss value and a faster convergence speed. Compared with the original lightweight federated recommendation system, it has higher prediction accuracy and recommendation quality. The Delta Loss of LGFR is smoother and converges faster than that of LightFR, indicating that the optimized model has stronger stability.

[0152] (3) Comparative experiment;

[0153] This embodiment is compared with a classic matrix factorization-based recommendation model, a recent federated matrix factorization recommendation model, and the LightFR model optimized in this invention.

[0154] Classic matrix factorization-based recommendation models are built on centralized storage setups, which fail to protect user privacy. Most existing federated matrix factorization recommendation models perform similarity searches through inner products in Euclidean space, which cannot effectively handle rating data in a privacy-enhancing manner.

[0155] PMF is a classic probabilistic latent factor model that decomposes users and items into a common subspace where the similarity between users and items can be measured by the inner product in Euclidean space. FedMF is a privacy-enhancing matrix factorization recommendation model based on secure homomorphic encryption in a federated setting. LightFR is introduced in Chapter 2.

[0156] In this embodiment, the proportion p of selected clients is set to 0.6. For the centralized MF-based model, this embodiment sets the number of training rounds E to 50 and employs an early stopping technique, i.e., if the performance does not improve after five consecutive rounds, the model will stop running. The batch size B is set to 512, and the public key length l used in FedMF is set to 1024. The comparison results are shown in Table 4. In this embodiment, LGFR outperforms the unoptimized LightFR on the Filmtrust and Ciao datasets.

[0157] Table 4

[0158]

[0159] This study compares LGFR with LightFR and FedMF on several lightweight metrics, namely local client storage cost, communication cost, and inference time cost. The Ciao dataset is used as an example to test these three non-standard metrics. First, default parameters are defined and set: the Ciao dataset contains 105,096 items, and the average number of items interacted with by users in the Ciao dataset is 38. Figure 3 The inference cost, communication cost, and storage cost of the three models are presented. The time inference cost was measured on a local client with 16GB of RAM and a 4.90 GHz 14-core processor.

[0160] from Figure 3 It can be concluded that LGFR outperforms the FedMF model in terms of memory, communication, and inference efficiency, while its storage and communication costs are comparable to LightFR. However, its training time is approximately three times that of LightFR because LGFR introduces a hybrid loss training strategy, which requires additional computation time. Furthermore, hard negative sample mining requires additional prediction calculations for each positive sample. Although these factors increase training time, LGFR is more efficient than LightFR in terms of inference.

[0161] The lightweight gradient optimization federated recommendation method and system proposed in this invention, due to its combination of privacy protection, high efficiency, and high accuracy, is particularly suitable for deployment on resource-constrained terminal devices, such as smartphones, tablets, smart TVs, and in-vehicle infotainment systems. Its core application scenarios lie in various online services requiring large-scale personalized content distribution, especially in the field of entertainment media content recommendation.

[0162] This invention has crucial application value in personalized recommendation services for streaming media and content aggregation platforms such as video, music, news, and podcasts. The Filmtrust movie rating dataset used in the experiment directly reflects this scenario.

[0163] 1. Video-on-demand and short video platforms;

[0164] Scenario Description: These platforms boast hundreds of millions, even billions, of users, and a library of millions of movies, TV series, or short videos. Users watch on their respective devices (phones, TVs, computers). The platform's core business objective is to increase user engagement, viewing time, and paid conversion rates through precise personalized recommendations. Simultaneously, users' viewing history is an extremely sensitive matter of personal privacy.

[0165] Applications and Value of this Invention: Personalized Homepage with Privacy Protection: When a user opens the application, this invention allows the recommendation model to be trained on the user's local device. The user's complete viewing history (e.g., watched movies, viewing duration of each video, likes / dislikes, etc.) does not need to be uploaded to the company's central server. The server only aggregates encrypted model gradients that do not contain privacy information. This addresses users' privacy concerns about "the platform monitoring me," complies with increasingly stringent global data protection regulations, and greatly enhances user trust.

[0166] Resource-friendly client deployment: Traditional federated recommendations require distributing massive item feature matrices (potentially gigabytes in size) to users' mobile phones or TVs, consuming significant bandwidth and valuable storage space. This invention reduces communication and storage overhead by orders of magnitude by representing item features as extremely compact binary codes. User devices only need to store their own binary codes and the global item binary code matrix, which is particularly crucial for smart TVs or older smartphones with limited storage space.

[0167] Instant responsiveness and a smooth user experience: Users expect recommended content to open instantly. This invention utilizes efficient bitwise operations (Hamming distance) for recommendation sorting, which is far faster than traditional floating-point inner product operations. This means that even if the user's device has limited performance, a high-quality recommendation list can be generated the instant the application is opened, avoiding loading delays, significantly improving the user experience, and reducing user churn.

[0168] High-quality ranking recommendation: As demonstrated by experiments on the Filmtrust dataset, this invention directly optimizes the ranking metric (NDCG) through a hybrid loss function, ensuring the ranking quality of the recommendation list. It not only recommends movies a user might like, but also places the movies most likely to be liked at the top of the list. Simultaneously, by modeling item bias, the system can effectively distinguish whether a movie's popularity stems from its global blockbuster status (high bias) or from its precise alignment with the user's unique taste (high interaction score), thereby recommending more high-quality niche or long-tail content and enhancing the surprise factor of content discovery.

[0169] 2. Music streaming and podcast platforms:

[0170] Scenario Description: Similar to video platforms, music platforms also possess massive music libraries and users. Users' listening preferences and created playlists are core pieces of private data. The task of a recommendation system is to generate personalized daily playlists and recommend similar songs or podcasts.

[0171] Applications and Value of the Invention: The technology of this invention is also applicable here. Users' listening history data is stored locally, protecting their personal music taste privacy. The lightweight model ensures that background model updates and recommendation calculations do not consume excessive battery power. Fast inference capabilities support instant recommendation functions such as "play the next similar song".

[0172] 3. News and text content aggregation platform:

[0173] Scenario Description: Users browse news and articles on their mobile phones. Users' reading habits and interests are also privacy-sensitive and need to be protected. The goal of a recommendation system is to build a diverse and engaging information stream for each user.

[0174] Applications and Value of this Invention: This invention can build a privacy-preserving news recommendation engine. By analyzing users' reading behavior locally, the system can recommend relevant news to users without sending their specific reading history to the cloud. Its efficient sorting capabilities ensure that the articles at the top of the information stream are those that users are most interested in, thereby increasing user dwell time.

[0175] This invention, through its unique technical solution, perfectly resolves the three core contradictions in modern entertainment media content recommendation: privacy protection, device resource limitations, and recommendation accuracy. It provides a solid and feasible technical foundation for building an efficient, secure, and high-quality next-generation personalized recommendation system on various terminal devices.

[0176] The above are merely preferred embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A highly efficient and lightweight federated recommendation method, characterized in that, Includes the following steps: The server initializes the item continuous value matrix and item bias vector; applies a sign function to the item continuous value matrix to generate a binarized item matrix; the server distributes the binarized item matrix to several clients; Each client initializes its local user binary vector, user bias vector, and item bias vector based on the binarized item matrix, and constructs training triples by combining local data through the hard negative sample mining module; Based on the training triples, a hybrid loss function is used to train the local model to update the local user binary vector, user bias vector, and item bias vector, and to calculate the item gradient matrix. Each client uploads the calculated item gradient matrix to the server; The steps for constructing training triples using the hard negative sample mining module include: For each user's positive sample items, candidate negative samples are randomly sampled from the negative sample set; based on the current model's predicted score, the item with the highest predicted score is selected as the difficult negative sample from the candidate negative samples; training triples are constructed based on the user, positive sample items, and difficult negative samples. Obtain the local user's binary vector and the binary item matrix, and calculate the inner product. Summate the inner product with the user's bias vector and the item's bias vector to obtain the user's predicted rating for the candidate items. The hybrid loss function is composed of a weighted combination of sorting loss, reconstruction loss, and regularization loss. The formula for the hybrid loss function is as follows: ； In the formula, For sorting loss, To reconstruct the loss, This is the regularization loss; These are the weighting coefficients for the mixed loss. The ranking loss is a pairwise ranking loss. For each training triplet, the difference between the user's predicted rating for positive sample items and the predicted rating for difficult negative samples is calculated. The difference in predicted ratings is then transformed nonlinearly using the Sigmoid function. The natural logarithm of the transformed result is taken to obtain the final ranking loss. The average of the results for all training triplets is then taken as the negative value to obtain the final ranking loss. The sorting loss is as follows: ; The basic score prediction formula is as follows: In hard negative sample mining, when constructing ranked triples, the predicted score of candidate negative samples is calculated as follows: The difference in scores between positive and negative samples is calculated using the following formula: ; Among them, users Positive sample items negative sample items , For user collection, A collection of items; For users The real-valued embedding vector, For users The transpose of the real-valued embedding vector; Positive sample items The real-valued embedding vector; For the user bias vector, Positive sample items Item bias vector; negative sample items The real-valued embedding vector; negative sample items Item bias vector; The reconstruction loss is the point-by-point mean square error loss, which is obtained by calculating the mean square of the difference between the observed score and the predicted score. The reconstruction loss is as follows: ; in, For the observed user-item rating pairs, For the rating matrix The elements in represent users For items The score; The server aggregates and updates the global item continuous value matrix and item bias vector based on the item gradient matrix uploaded by each client, completing one round of client updates. Each client then recommends items based on the updated model.

2. The efficient and lightweight federated recommendation method according to claim 1, characterized in that, The steps for the server to initialize the item continuous value matrix and the item bias vector include: randomly initializing the item continuous value matrix; and initializing the item bias vector to a zero vector.

3. The efficient and lightweight federated recommendation method according to claim 1, characterized in that, Calculate the sum of squares of the L2 norm of all user latent factor vectors, the sum of squares of the L2 norm of all item latent factor vectors, the sum of squares of all user bias terms, and the sum of squares of all item bias terms; then add the four sums together and multiply by the regularization strength parameter to obtain the final regularization loss value. The regularization loss is as follows: ; Among them, users Positive sample items , For user collection, For a collection of items, For regularization parameters; For users The real-valued embedding vector; Positive sample items The real-valued embedding vector; For the user bias vector, Positive sample items The item bias vector.

Images