Commodity recommendation model training method based on transverse federation and multivariate representation

Abstract

The invention relates to a transverse federation and multivariate representation-based commodity recommendation model training method, which comprises the following steps that: a central server runs a key generation algorithm to generate a public key and a private key, and sends the public key to a client participating in federation learning in the current round through a secure channel; the central server issues a global commodity recommendation model to each client; each client uses local user-commodity interaction data to train the global commodity recommendation model to obtain a local commodity recommendation model; the local commodity recommendation model and the performance index are encrypted through a public key of the central server and then uploaded to the central server; the central server decrypts the local commodity recommendation models encrypted by the clients and the performance indexes by using the private keys, and aggregates the local commodity recommendation models uploaded by the clients according to the performance indexes to obtain a global commodity recommendation model of the next round of iterative training; the above steps are repeatedly executed until the global commodity recommendation model converges or reaches a preset number of iterations, and the trained commodity recommendation model is obtained.The real-time performance and accuracy in recommendation can be improved, and meanwhile the generalization ability of the model is improved.

Description

Technical Field

[0001] The present invention belongs to the field of federated learning and product recommendation, and in particular relates to a product recommendation model training method based on horizontal federation and multi-dimensional representation. Background Art

[0002] In today's digital age, data has become the core resource driving the development of various industries. In the field of e-commerce, massive amounts of user-product interaction data contain rich information, such as user purchase preferences, browsing habits, product trends, etc. Accurate product recommendations can not only enhance the user shopping experience and increase user stickiness, but also significantly improve merchants' sales and operational efficiency.

[0003] Traditional product recommendation model training mostly adopts a centralized architecture, that is, collecting user-product interaction data from various data sources and aggregating them to a central server or data center for unified processing and model training. Although this method can build an effective recommendation model to a certain extent, it faces many severe challenges in practical applications.

[0004] On the one hand, data privacy protection has become a prominent issue. As people pay more and more attention to personal privacy and relevant laws and regulations (such as the EU's GDPR) are introduced, data holders (such as different business departments of e-commerce platforms, third-party applications with user data, etc.) have become extremely cautious about sharing their own user data. Because once the data is leaked, it may lead to the abuse of user information, trigger a crisis of user trust, and bring huge reputation losses and legal risks to the company.

[0005] On the other hand, data silos are common. Due to business barriers, competitive relationships and other factors, data between different companies or departments is often stored independently, making it difficult to effectively integrate and circulate. This makes it difficult for centralized model training to obtain comprehensive and rich data, limiting the accuracy and generalization of the model. For example, the mobile application and web application of an e-commerce platform may be managed by different teams, and their data storage and management methods are different, making it difficult to achieve seamless docking. Summary of the invention

[0006] In view of the problems existing in the background technology, the present invention proposes a product recommendation model training method based on horizontal federation and multi-representation, which specifically includes the following steps:

[0007] S1: The central server runs the key generation algorithm to generate its public key and private key, and sends the public key to the clients participating in this round of federated learning through a secure channel;

[0008] S2: The central server sends the global product recommendation model to each client;

[0009] S3: Each client uses the local user-product interaction data to train the global product recommendation model to obtain a local product recommendation model; and encrypts the local product recommendation model and performance indicators with the public key of the central server and uploads them to the central server;

[0010] S4: The central server uses the private key to decrypt the local product recommendation model and performance indicators encrypted by the client, and aggregates the local product recommendation models uploaded by each client according to the performance indicators to obtain the global product recommendation model for the next round of iterative training;

[0011] S5: Repeat steps S2 to S5 until the global product recommendation model converges or reaches a preset number of iterations, thereby obtaining a trained product recommendation model.

[0012] The present invention has at least the following beneficial effects

[0013] Through encrypted transmission and federated learning architecture, the client's original user-product interaction data does not need to leave the local area, avoiding the risk of privacy leakage during data transmission and centralized storage. The central server can only obtain encrypted local models and performance indicators, and cannot directly access sensitive data, ensuring data privacy. Compared with the model trained by a single data source, the model can capture a wider range of user behavior patterns and product features by using the rich local data of each client for training, thereby improving the accuracy and generalization ability of the product recommendation model and providing users with more accurate product recommendation services. The client conducts local model training in parallel, and the central server is responsible for model aggregation. This distributed architecture makes full use of the computing resources of each client and improves training efficiency. The central server sends the public key through a secure channel, the client uses the public key to encrypt the data, and the central server uses the private key to decrypt it. This encryption mechanism ensures the security of the data during transmission and prevents the data from being stolen or tampered with. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 It is a schematic diagram of the method flow of the present invention. DETAILED DESCRIPTION

[0015] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0016] See also Figure 1 The present invention provides a product recommendation model training method based on horizontal federation and multi-representation, comprising:

[0017] S1: The central server runs the key generation algorithm to generate its public key and private key, and sends the public key to the clients participating in this round of federated learning through a secure channel;

[0018] Preferably, the central server runs a key generation algorithm to generate its public key and private key, including:

[0019] S11: Randomly generate two large prime numbers p and q;

[0020] S12: Calculate the common modulus n and Euler function

[0021] S13: Select a value less than And with The mutually prime integers are used as the public key P k ; Calculate the public key P k In the model The inverse element d under Take d as the private key S k .

[0022] In this embodiment, the client uses the public key to encrypt the local product recommendation model and performance indicators, and only the central server holding the private key can decrypt it, thereby ensuring the security of the data during transmission and preventing the data from being stolen or tampered with.

[0023] S2: The central server sends the global product recommendation model to each client;

[0024] S3: Each client uses the local user-product interaction data to train the global product recommendation model to obtain a local product recommendation model; and encrypts the local product recommendation model and performance indicators with the public key of the central server and uploads them to the central server;

[0025] Preferably, the user-product interaction data is represented as is a continuous time graph, Indicates that all node sets include user node v u and product node v p , Represents the interaction relationship between user nodes and product nodes at different time points, represents the feature set of all nodes, Represents the interaction label set between user nodes and product nodes. Represents user v u With product v p At time t, there is an interaction. Represents user v u With product v p There is no interaction at time t.

[0026] In this embodiment, the interaction between the user node and the commodity node at different time points includes: purchase, browsing and liking, etc. When the user node purchases the commodity node at time t1, there is an edge from the user node to the commodity node, which is represented by (v u , v p , t1) represents the interaction relationship between the user node and the product node at t1; for the user node v u , whose characteristics may include age, gender, region, etc., which can be expressed as Represents user node v u Features; for commodity node v p The features of may include product category, price, and brand, etc., which can be expressed as Represents the product node v p Features.

[0027] Preferably, the product recommendation model adopts a DA-TGN model; and the training of the global product recommendation model using local user-product interaction data includes:

[0028] S31: Generate initial state vectors for each user node and product node respectively according to feature sets of user nodes and product nodes in user-product interaction data;

[0029] In this embodiment, when generating the initial state vector, these features may be converted into vector form according to the preset mapping rules, and a suitable encoding method may be used: standardization or normalization of numerical features, one-hot encoding of label features, and word embedding of text features. Then, the label features are converted into vectors through the embedding layer and concatenated with other numerical features to form the initial state vectors of users and products.

[0030] S32: Calculate the message received by the user node and the commodity node when the interaction event occurs according to the interaction event between the user node and the commodity node;

[0031] Preferably, the message received by the user node and the commodity node when an interaction event occurs includes:

[0032] m i (t) = s j (t - )||s i (t - )||Δt||e ij (t))

[0033] m j (t) = s i (t -)||s j (t - )||Δt||e ij (t))

[0034] Among them, m i (t) represents the message received by user node i at time t; m j (t) represents the message received by commodity node j at time t; s j (t - ) represents the state vector of commodity node j before time t; s i (t - ) represents the state vector of the user node before time t; Δt represents the time interval from time t to the last interaction event between user node i and commodity node j; e ij (t) represents the relevant information of the edge between user node i and product node j at time t; t represents the time when this interaction event occurs.

[0035] In this embodiment, various information of user nodes and commodity nodes in interaction events are integrated together to form a message vector. These messages contain the previous state vector, time interval and edge information of the node, which provides a basis for subsequent message aggregation and status update. By recording the time interval of interaction between user nodes and commodity nodes, the model can better understand the changes of user behavior patterns over time; the relevant information of the edge can take into account the specific circumstances of the interaction between users and commodities, such as different types of interaction (purchase, browsing, collection, etc.), and detailed information of the interaction (amount, duration, etc.). This information is integrated in the message, which helps the model to perform different updates and learning according to different interaction details, improve the model's ability to distinguish and understand different interaction behaviors, and provide more detailed information basis for personalized recommendations.

[0036] S33: Calculate the aggregate message of the user node or commodity node according to all messages received by each user node or commodity node;

[0037] Preferably, the aggregated message of the user node or commodity node includes:

[0038]

[0039] in, Represents the aggregated message of user node i at time t; represents the aggregated message of commodity node j at time t; m i (t b ) indicates that user node i is in t b Messages received at the moment; m j (t b ) indicates that commodity node j is in tb The message received at time t; Δt represents the time interval from time t to the last interaction event between user node i and product node j; t1 represents the time when the first interaction event between user node i and product node j occurs; agg represents the averaging function.

[0040] In this embodiment, by aggregating messages, the messages received by the user node or the commodity node at different times are integrated, avoiding the limitation of considering only the messages at a single time. By taking the average function, the information received by the node at different time points can be smoothed, the noise impact of a single message can be reduced, and the information update of the node can be more stable and comprehensive. Considering all messages from the first interaction to the current moment helps the model process the time series information of users and commodities. This aggregation method can help the model capture the changing trend of the interactive information of users or commodities at different time points, rather than focusing only on the interaction at a certain moment, so that the model can better understand the dynamic behavior pattern of the user or commodity node in the time dimension. As time goes by and new interactions occur, the aggregated messages of the node will be continuously updated, so that the model can continue to learn new behaviors and features of users and commodities. This dynamic update mechanism enables the model to adjust its internal state according to the latest behavior of users and commodities, improve the adaptability to user preferences and commodity characteristics, and provide more accurate and dynamic information support for the recommendation system, thereby achieving better recommendation effects.

[0041] S34: updating the state vectors of the user node and the commodity node respectively according to the aggregated messages of the user node and the commodity node;

[0042] Preferably, the updating of the state vectors of the user node and the commodity node includes:

[0043]

[0044] Among them, s i (t) represents the updated state vector of user node i at time t; s j (t) represents the updated state vector of commodity node j at time t; s j (t - ) represents the state vector of commodity node j before time t; s i (t - ) represents the state vector of user node i before time t; GRU represents gated recurrent unit.

[0045] In this embodiment, when a user interacts with a product at different time points, the GRU can adjust the node state according to the new aggregated message to reflect the latest information of the user or product, rather than simply replacing the old information, so that the model can better learn the long-term dynamic behavior of users and products. By continuously updating the state vector, the DA-TGN model can better handle the time series interaction behavior of users and products. For user and product interaction data with complex time series characteristics, this update mechanism allows the model to learn information such as user behavior change trends and preference evolution, thereby improving the model's ability to predict user and product interactions at different time points, and ultimately improving the accuracy and adaptability of product recommendations.

[0046] S35: Obtaining the final representation vector of the user node through the attention mechanism according to the updated state vector of the user node and the state vectors of the neighboring nodes of the user node;

[0047] Preferably, the calculating the final representation vector of the user node comprises:

[0048]

[0049] Among them, z i (t) = emb(i, t) represents the final representation vector of user node i at time t, represents the set of k-hop neighbor nodes of user node i in the time interval [0, t]; || represents the splicing operation; e ij (t) represents the edge-related information between user node i and product node j at time t; attn(·) represents the attention weight.

[0050] In this embodiment, by combining the updated state vector of the user node with the state vector and side information of the neighbor node, and using the attention mechanism, the final representation vector can fuse the information of the user node itself and the neighbor node. This method allows the model to consider the environmental information of the user node more comprehensively, rather than relying solely on the information of the user node itself. For example, the neighbor nodes of the user node may be commodities that the user has purchased or browsed before. Taking the information of these commodity nodes into consideration can more accurately reflect the user's preferences and behavior patterns. As time goes by and user behavior changes, the state vector and side information of the neighbor node will change, and the attention weight will also be adjusted accordingly, so the final representation vector is dynamically updated. This helps the model capture the dynamic changes of user behavior and preferences, better adapt to the different preferences and behavior patterns of users at different time points, thereby improving the accuracy and adaptability of the recommendation system. For example, as users purchase or browse different commodities, their neighbor node information is updated, and the final representation vector is also updated, making the recommendation more in line with the user's latest behavior.

[0051] S36: Predict the probability score of the user node and the product node interacting by inner product based on the final representation vector of the user node and the final representation vector of the product node:

[0052] Preferably, the probability score of the interaction between the user node and the commodity node includes:

[0053]

[0054] Among them, z u (t) represents the final representation vector of user node u, z p (t) represents the final representation vector of the product node p, It represents the probability score of the interaction between user node u and product node p, and e represents the natural base.

[0055] In this embodiment, this information is converted into a concise probability through the inner product and sigmoid function, and various characteristics and relationships between users and commodities are comprehensively considered, so that the recommendation system can recommend the most likely interactive commodities to users based on this probability. The inner product result is converted into a probability using the sigmoid function, so that the result is between 0 and 1, with a clear probability interpretation. At the same time, the sigmoid function introduces nonlinearity, so that the model can handle more complex relationships, avoids the linear limitations that may be caused by directly using the inner product result, and improves the model's ability to distinguish different user and commodity combinations, thereby improving the accuracy and flexibility of the recommendation system. For example, even if the inner product score difference between two user nodes and commodity nodes is small, through the nonlinear conversion of the sigmoid function, the final probability score may have a more obvious distinction, which is more conducive to recommendation sorting.

[0056] S37: Construct a cross entropy loss function based on the predicted probability score of the user node and the product node interacting, and the interaction label between the user node and the product node:

[0057] Preferably, the cross entropy loss function includes:

[0058]

[0059] Among them, y u，p represents the actual interaction label between user node u and product node p. represents the smoothed interaction label, ∈ is a small smoothing parameter with a value between [0, 1]. Indicates the predicted probability score of the interaction between the user node and the product node.

[0060] In this embodiment, the smoothing parameter ∈ is introduced to smooth the actual interaction labels, which can prevent the model from being overconfident in extreme cases. When there are few samples or the samples are unbalanced, this helps prevent the model from overfitting some samples and improves the generalization ability of the model.

[0061] S38: According to the constructed loss function, the parameters of the global product recommendation model are tuned by the Adam optimizer to obtain the trained local product recommendation model.

[0062] S4: The central server uses the private key to decrypt the local product recommendation model and performance indicators encrypted by the client, and aggregates the local product recommendation models uploaded by each client according to the performance indicators to obtain the global product recommendation model for the next round of iterative training;

[0063] Preferably, the performance indicators include the number of samples in the client's training set, the average error of the client's local product recommendation model in the validation set, and the convergence speed of the client's local product recommendation model in the training set; and aggregating the local product recommendation models uploaded by each client according to the performance indicators includes:

[0064]

[0065] Among them, G represents the global product recommendation model for the next round of iterative training, and D i represents the number of training set samples of client i, g i represents the local product recommendation model uploaded by the i-th client, represents the weight of the i-th client, ε i represents the average error of the local product recommendation model uploaded by the i-th client in the validation set, η i It represents the convergence speed of the local product recommendation model uploaded by the i-th client in the training set. μ and θ are constants, 0<μ, θ<1. α and β are hyperparameters, α>1, 0<β<1.

[0066] In this embodiment, the performance index includes the number of training set samples of the client. In federated learning, the amount of data of different clients may vary greatly. By incorporating it into the weight calculation, clients with larger data volumes will obtain relatively higher weights in the global model aggregation. For example, if a client has rich training data, it means that it contains more diverse user-product interaction information, and its local model may contribute more to the global model. This helps the global model better capture the behavior patterns and product characteristics of different user groups, thereby improving the generalization ability of the model and the accuracy of recommendations for various users. The average error of the local product recommendation model in the validation set is an important indicator for measuring the accuracy of the model. In the weight calculation, it is in the denominator and appears in the form of an exponent, which makes the weight of the client with a smaller average error relatively higher. For example, if the client's model shows a lower error on the validation set, it means that the model has a stronger ability to fit and predict the data and is closer to the real user-product interaction relationship. By giving such clients higher weights, the global model can absorb more accurate model information during aggregation, which helps to improve the prediction accuracy of the global model and reduce the recommendation error. The convergence speed of the local product recommendation model in the training set reflects the efficiency and stability of the model training. Models with fast convergence speed may use data more effectively during training and reach a better performance state faster. Although the impact of convergence speed on weights is relatively small, it still plays a role in weight calculation. This encourages clients to optimize models during training and improve training efficiency. It also ensures that the global model does not rely too much on convergence speed and ignore other important factors when aggregating, ensuring that the model strikes a balance between accuracy and training efficiency. By comprehensively considering the three performance indicators of the number of training set samples, the average error of the validation set, and the convergence speed of the training set to calculate weights, and aggregating the local models of each client to obtain a global model, the advantages of different clients can be fully integrated. For example, one client may have a small sample size, but a high model accuracy and fast convergence speed; another client may have a large sample size, but a slightly lower accuracy. Through this weight calculation method, the contribution of each client can be reasonably weighed, so that the global model can not only utilize rich data information, but also ensure a certain accuracy and training efficiency, thereby continuously optimizing the global model and improving the overall performance of the product recommendation system.

[0067] S5: Repeat steps S2 to S5 until the global product recommendation model converges or reaches a preset number of iterations, thereby obtaining a trained product recommendation model.

[0068] In summary, under the premise of ensuring data privacy, the present invention promotes effective collaboration between the client and the central server through this aggregation method based on performance indicators. The client uploads the encrypted model and performance indicators, and the central server decrypts and aggregates them using the private key. The introduction of performance indicators makes the collaboration process more scientific and reasonable. The client can contribute to the global model based on its own data characteristics and model performance. At the same time, the central server aggregates these models and gradually optimizes the global model without exposing the original data, thus achieving the dual goals of privacy protection and model optimization.

[0069] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solution, which should be included in the scope of the claims of the present invention.

Claims

1. A product recommendation model training method based on horizontal federation and multi-representation, characterized in that: include: S1: The central server runs the key generation algorithm to generate its public key and private key, and sends the public key to the clients participating in this round of federated learning through a secure channel; S2: The central server sends the global product recommendation model to each client; S3: Each client uses local user-product interaction data to train the global product recommendation model to obtain a local product recommendation model; and encrypts the local product recommendation model and performance indicators with the public key of the central server and uploads them to the central server; S4: The central server uses the private key to decrypt the local product recommendation model and performance indicators encrypted by the client, and aggregates the local product recommendation models uploaded by each client according to the performance indicators to obtain the global product recommendation model for the next round of iterative training; S5: Repeat steps S2 to S5 until the global product recommendation model converges or reaches a preset number of iterations, thereby obtaining a trained product recommendation model.

2. A product recommendation model training method based on horizontal federation and multi-representation according to claim 1, characterized in that: The user-product interaction data is represented as is a continuous time graph, V represents the set of all nodes including user node v u and product node v p , ε(T) represents the interaction relationship between user nodes and commodity nodes at different time points, represents the feature set of all nodes, Represents the interaction label set between user nodes and product nodes. Represents user v u With product v p At time t, there is an interaction. Represents user v u With product v p There is no interaction at time t.

3. A product recommendation model training method based on horizontal federation and multivariate representation according to claim 1, characterized in that: The product recommendation model adopts the DA-TGN model; the training of the global product recommendation model using the local user-product interaction data includes: S21: Generate initial state vectors for each user node and product node respectively according to feature sets of user nodes and product nodes in user-product interaction data; S22: Calculate the message received by the user node and the commodity node when the interaction event occurs according to the interaction event between the user node and the commodity node; S23: Calculate the aggregate message of the user node or commodity node according to all messages received by each user node or commodity node; S24: updating the state vectors of the user node and the commodity node respectively according to the aggregated messages of the user node and the commodity node; S25: Obtain the final representation vector of the user node through the attention mechanism according to the updated state vector of the user node and the state vectors of the neighboring nodes of the user node; S26: Predict the probability score of the interaction between the user node and the product node through the inner product based on the final representation vector of the user node and the final representation vector of the product node: S27: Construct a cross entropy loss function based on the predicted probability scores of the user node and the product node interacting, and the interaction labels of the user node and the product node: S28: According to the constructed loss function, the parameters of the global product recommendation model are tuned by the Adam optimizer to obtain the trained local product recommendation model.

4. A product recommendation model training method based on horizontal federation and multi-representation according to claim 1, characterized in that: The messages received by the user node and the commodity node when an interaction event occurs include: m i (t)=s j (t - )||s i (t - )||Δt||e ij (t)) m j (t)=s i (t - )||s j (t - )||Δt||e ij (t)) Among them, m i (t) represents the message received by user node i at time t; m j (t) represents the message received by commodity node j at time t; s j (t - ) represents the state vector of commodity node j before time t; s i (t - ) represents the state vector of the user node before time t; Δt represents the time interval from time t to the last interaction event between user node i and commodity node j; e ij (t) represents the relevant information of the edge between user node i and product node j at time t; t represents the time when this interaction event occurs.

5. A product recommendation model training method based on horizontal federation and multivariate representation according to claim 1, characterized in that: The aggregated message of the user node or commodity node includes: in, Represents the aggregated message of user node i at time t; represents the aggregated message of commodity node j at time t; m i (t b ) indicates that user node i is in t b Messages received at the moment; m j (t b ) indicates that commodity node j is in t b The message received at time t; Δt represents the time interval from time t to the last interaction event between user node i and product node j; t1 represents the time when the first interaction event between user node i and product node j occurs; agg represents the averaging function.

6. A product recommendation model training method based on horizontal federation and multi-representation according to claim 1, characterized in that: The updating of the state vectors of the user node and the commodity node includes: Among them, s i (t) represents the updated state vector of user node i at time t; s j (t) represents the updated state vector of commodity node j at time t; s j (t - ) represents the state vector of commodity node j before time t; s i (t - ) represents the state vector of user node i before time t; GRU represents gated recurrent unit.

7. A product recommendation model training method based on horizontal federation and multi-representation according to claim 1, characterized in that: The calculating of the final representation vector of the user node comprises: Among them, z i (t) = emb(i, t) represents the final representation vector of user node i at time t, represents the set of k-hop neighbor nodes of user node i in the time interval [0, t]; || represents the splicing operation; e ij (t) represents the edge-related information between user node i and product node j at time t; attn(·) represents the attention weight.

8. A product recommendation model training method based on horizontal federation and multi-representation according to claim 1, characterized in that: The probability score of the user node and the commodity node interacting includes: Among them, z u (t) represents the final representation vector of user node u, z p (t) represents the final representation vector of the product node p, It represents the probability score of the interaction between user node u and product node p, and e represents the natural base.

9. A product recommendation model training method based on horizontal federation and multi-representation according to claim 1, characterized in that: The cross entropy loss function includes: Among them, y u,p represents the actual interaction label between user node u and product node p. represents the smoothed interaction label, ∈ is a small smoothing parameter with a value between [0,1]. Indicates the predicted probability score of the interaction between the user node and the product node.

10. A product recommendation model training method based on horizontal federation and multi-representation according to claim 1, characterized in that: The performance indicators include the number of training set samples of the client, the average error of the local product recommendation model of the client in the validation set, and the convergence speed of the local product recommendation model of the client in the training set; The local product recommendation model that aggregates the local product recommendations uploaded by each client according to the performance indicators includes: Among them, G represents the global product recommendation model for the next round of iterative training, and D i represents the number of training set samples of client i, g i represents the local product recommendation model uploaded by the i-th client, represents the weight of the i-th client, ε i represents the average error of the local product recommendation model uploaded by the i-th client in the validation set, η i It represents the convergence speed of the local product recommendation model uploaded by the i-th client in the training set. μ and θ are constants, 0<μ, θ<1. α and β are hyperparameters, α>1, 0<β<1.

Images