Model training method, recommendation method and system of cross-border recommendation graph neural network

Abstract

This application relates to the field of graph neural network technology, and particularly to a model training method, recommendation method, and system for cross-border recommendation graph neural networks. The model training method trains a cross-border recommendation model to generate feature representations of the target object and the target item. The cross-border recommendation model aggregates the first embedding vectors of the second-order item neighbors of the target object, where the second-order item neighbors are connected to the target object through implicit association objects. It also aggregates the second embedding vectors of the second-order object neighbors of the target item, where the second-order object neighbors are connected to the target item through implicit association objects. This application proposes a novel implicit relationship aggregation constraint, enhancing the ability to learn generalizable representations from sparse and isolated local data in cross-border scenarios where objects do not overlap. This enables various clients to form cross-border recommendation models with highly overlapping parameter space distributions, allowing the parameters of the global model aggregated on the server to converge.

Description

Technical Field

[0001] This application relates to the field of graph neural network technology, and in particular to model training methods, recommendation methods and systems for cross-border recommendation graph neural networks. Background Technology

[0002] Currently, graph neural networks are widely used in product recommendation systems. However, in cross-border recommendation scenarios, the sparsity of local graph data makes it difficult to train sufficient training samples for cross-border recommendation models.

[0003] To provide sufficient training data, a federated learning framework is typically used to distribute cross-border recommendation models to clients located around the world. The clients then train the models using their local graph data, and the server aggregates the model updates uploaded by each client to form a global model.

[0004] However, due to factors such as regional culture, policy restrictions, and consumption habits, although the graph structures constructed by each client overlap in the feature space, there are systematic differences in structure, semantics, and dynamism.

[0005] This systematic difference leads to significant differences in the distribution of local models in the parameter space for each client. When aggregating, the parameters of the global model are difficult to converge effectively in the same region, and the prediction accuracy of the global model is not significantly improved. Summary of the Invention

[0006] The main objective of this application is to propose a model training method, recommendation method, and system for cross-border recommendation graph neural networks, so as to enable the client's cross-border recommendation model to aggregate information from local graph data through a new aggregation paradigm, and form a cross-border recommendation model with overlapping distribution in the parameter space of each client, so that the global model parameters aggregated on the server can converge.

[0007] To achieve the above objectives, one aspect of this application proposes a model training method for a cross-border recommendation graph neural network, applied to a federated learning client, the method comprising:

[0008] Determine the object interaction sequence based on the target object's historical interaction records, and determine the project interaction sequence based on the target project's historical interaction records.

[0009] A dynamic bipartite graph is constructed using the object interaction sequence and the project interaction sequence. The dynamic bipartite graph includes bipartite graphs corresponding to N time steps, where N is an integer greater than or equal to 1.

[0010] A cross-border recommendation model is constructed or updated. During the information transmission process on the dynamic bipartite graph, the cross-border recommendation model aggregates the first embedding vector of the second-order item neighbors of the target object. The second-order item neighbors are connected to the target object through implicit association objects. The first embedding vector represents the items liked by the target object. A second embedding vector is also aggregated with the second-order object neighbors of the target object. The second-order object neighbors are connected to the target object through implicit association objects. The second embedding vector represents the objects attracted by the target object.

[0011] The cross-border recommendation model is trained based on the training triples and the dynamic bipartite graph, and the model update data is sent to the server so that the server can determine the global model based on the cross-border recommendation models of several clients.

[0012] In some embodiments, constructing a dynamic bipartite graph through the object interaction sequence and the project interaction sequence specifically includes:

[0013] Each interaction behavior in the object interaction sequence is converted into an object dense vector, and the object dense vector is embedded according to time decay to obtain the embedded representation of the target object;

[0014] Each interaction behavior in the project interaction sequence is converted into a project dense vector, and the project dense vector is embedded according to time decay to obtain the embedded representation of the target project;

[0015] The dynamic bipartite graph is constructed based on the embedding representation of the target object and the embedding representation of the target item.

[0016] In some embodiments, the cross-border recommendation model is provided with a GNN module and a perception module;

[0017] The training of the cross-border recommendation model based on the training triples and the dynamic bipartite graph specifically includes:

[0018] The training triples are encoded using interactive scoring labels, and the adjacency matrix is ​​determined using the dynamic bipartite graph.

[0019] The encoded training triples and the adjacency matrix are input into the cross-border recommendation model; wherein, the GNN module aggregates the first embedding vectors of the second-order item neighbors of the target object along the first meta-path to obtain the target object feature representation; the GNN module aggregates the second embedding vectors of the second-order object neighbors of the target item along the second meta-path to obtain the target item feature representation; the perception module predicts the interaction score based on the target object feature representation and the target item feature representation;

[0020] The cross-border recommendation model is updated based on the interaction rating labels and the prediction loss determined by the interaction ratings.

[0021] In some embodiments, the GNN module aggregates the first embedding vectors of the second-order item neighbors of the target object along the first metapath to obtain the feature representation of the target object, specifically including:

[0022] Determine common following objects based on the target object and its first-order object neighbors;

[0023] The first-order object neighbors of the common following object are determined as the implicit associated objects of the target object, and the embedding vectors of the first-order item neighbors of the implicit associated objects are passed as the first embedding vector.

[0024] By aggregating all the first embedding vectors, the feature representation of the target object is obtained.

[0025] In some embodiments, the GNN module aggregates the second embedding vectors of the second-order object neighbors of the target item along the second metapath to obtain the feature representation of the target item, specifically including:

[0026] The first-order project neighbors of the target project are determined based on the common rating object set; the first-order project neighbors are projects that have interaction records with objects in the common rating object set.

[0027] The first-order project neighbors of the target project are determined as the implicitly associated project, and the embedding vectors of the first-order object neighbors of the implicitly associated project are passed as the second embedding vector.

[0028] Aggregate all the second embedding vectors to obtain the feature representation of the target item.

[0029] In some embodiments, determining the first-order project neighbors of the target project based on a set of common scoring objects specifically includes:

[0030] A common scoring object set is determined by the historical interaction objects of the target project and the candidate project;

[0031] Calculate the interaction strength of each object in the set of common scoring objects with respect to the target item and the candidate items;

[0032] The candidate item is determined as a first-order item neighbor of the target item based on the interaction strength.

[0033] In some embodiments, calculating the interaction strength of each object in the common rating object set with respect to the target item and the candidate items specifically includes:

[0034] The scale confidence level is determined based on the number of objects in the common scoring object set;

[0035] Obtain the first rating of each object in the common rating object for the target item, and the second rating of each object for the candidate item, and calculate the distance between the first rating and the second rating to determine the evaluation similarity;

[0036] The interaction strength is determined based on the scale confidence and the evaluation similarity.

[0037] To achieve the above objectives, this application also provides a method for training a cross-border recommendation graph neural network model, applied to a federated learning server, the method comprising:

[0038] Get the model update data of cross-border recommendation models uploaded by K clients, where K is an integer greater than or equal to 1;

[0039] The global model is determined based on the model update data of the cross-border recommendation model.

[0040] To achieve the above objectives, this application also provides a cross-border object recommendation method. The global model obtained by the model training method described in this application recommends target items to target objects, specifically including:

[0041] The interaction triple is input into the global model to obtain the interaction score; the interaction triple includes the target object, the target item, and the time step, and the interaction score represents the degree of interest of the target object in the target item.

[0042] If the interaction score is greater than a preset interaction score threshold, the target item is pushed to the target object.

[0043] To achieve the above objectives, this application also provides a cross-border object recommendation system, including a processor and a memory, wherein the memory stores computer-readable executable code, which, when executed by the processor, implements the method described in this application.

[0044] The embodiments of this application include at least the following beneficial effects:

[0045] This application provides a model training method, recommendation method, and system for cross-border recommendation graph neural networks. The scheme constructs object interaction sequences and item interaction sequences based on historical interaction records of target objects and target items. A dynamic bipartite graph is then constructed using these sequences to build or update a cross-border recommendation model. The model is trained based on training triples and the dynamic bipartite graph, and the updated model data is sent to a server. During information transmission on the dynamic bipartite graph, the cross-border recommendation model aggregates the first embedding vectors of the target object's second-order item neighbors. These second-order item neighbors are connected to the target object through implicit association objects. The first embedding vector represents the items the target object likes. The application also aggregates the second embedding vectors of the target items' second-order object neighbors. The first embedding vector represents the characteristics of the items the target object likes, and the second embedding vector represents the enthusiast characteristics of the target items. This application uses these implicit relationships for aggregation constraints, enhancing the ability to learn generalizable representations from sparse and isolated local data in cross-border scenarios where objects do not overlap, resulting in a more convergent parameter space distribution among various cross-border recommendation models. Attached Figure Description

[0046] Figure 1 A flowchart of the training method provided in this embodiment;

[0047] Figure 2 This is a flowchart of step 104 of the training method shown in the embodiment of this application;

[0048] Figure 3 The dynamic bipartite graph constructed in step 102 of the training method shown in this embodiment;

[0049] Figure 4 A flowchart of another training method provided in this embodiment;

[0050] Figure 5 A flowchart of the recommended method provided in this embodiment. Detailed Implementation

[0051] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit it. In the following description, when referring to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with those of this application; they are merely examples of apparatuses and methods consistent with some aspects of the embodiments of this application as detailed in the appended claims.

[0052] It is understood that the terms “first,” “second,” etc., used in this application may be used herein to describe various concepts, but unless otherwise stated, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another. For example, without departing from the scope of the embodiments of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the words “if,” “when,” or “in response to a determination” as used herein may be interpreted as “when…” or “when…” or “in response to a determination.”

[0053] As used in this application, the terms "at least one", "multiple", "each", "any", etc., "at least one" includes one, two or more, "multiple" includes two or more, "each" refers to each of the corresponding multiples, and "any" refers to any one of the multiples.

[0054] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0055] In related technologies, graph neural network-based recommendation systems are a type of AI-optimized operating system. The operating system acquires interaction data to provide personalized product recommendations to users. Graph neural networks predict the relationships between users and products based on graph-structured data, thereby improving the accuracy of product recommendations.

[0056] In cross-border recommendation scenarios, the sharing of cross-border data is subject to legal regulations. To achieve accurate service, recommendation systems need to process large amounts of data containing sensitive information about objects (such as geographical location, transaction records, and credit status) to train graph neural networks. Graph data is highly susceptible to privacy breaches during cross-border flows. Therefore, to address the problem of local data sparsity, using a federated learning framework to train graph neural network models using cross-border data has become an effective solution.

[0057] Cross-border recommendation scenarios typically involve multi-source homogeneous data. "Homogeneous" means that the business models and feature spaces of each participant (client) are highly similar. For example, bank branches in different countries may maintain customer profile features (such as age and income level) and financial product features (such as product type and risk level) with the same structure.

[0058] Due to factors such as regional culture, policy restrictions, and consumption habits, although the graph structures constructed by each client overlap in the feature space, there are systematic differences in structure, semantics, and dynamism.

[0059] In cross-border recommendation scenarios, systemic differences mainly refer to:

[0060] 1. Heterogeneity of user-project interaction patterns: The user-project bipartite graphs constructed by different clients have significant differences in edge type, edge weight, and node degree distribution.

[0061] 1.1 The frequency and density of user interaction with the project may vary significantly across different countries or regions. For example, some regions have high user activity and frequent interactions, resulting in a dense graph structure; while other regions have sparse interactions and a simpler graph structure.

[0062] 1.2 Differences in Interaction Preferences: Although the business models are similar (e.g., both are bank recommendation systems), users in different regions may have drastically different preferences for financial products. For example, users in region A are more interested in investment products; users in region B prefer credit products; and users in region C are more interested in insurance products.

[0063] 2. Inconsistent graph topology.

[0064] 2.1 Uneven Node Distribution: The number, type, and distribution of users and projects are inconsistent across different clients. For example, client A has 100,000 users and 10,000 projects; client B has 50,000 users and 20,000 projects; and client C has very little overlap between users and projects, or even completely independent ones.

[0065] 2.2 Subgraph Structure Differences: Due to factors such as regional culture, policy restrictions, and market maturity, the local graph structures (such as connected components, clustering coefficients, path lengths, etc.) formed by different clients may vary greatly.

[0066] 3. Asynchronous behavior in temporal evolution

[0067] 3.1 Different patterns of user behavior evolution over time: For example, user preferences change rapidly in some regions, and the graph structure is highly dynamic; while in other regions, the graph structure is relatively stable and changes slowly.

[0068] 3.2 Inconsistent trends in project popularity: The popularity cycle of the same financial product may differ in different regions, resulting in inconsistent changes in the importance of nodes in the time slice chart over time.

[0069] In summary, in cross-border recommendation scenarios, the graph structures of different clients have local differences, which leads to different data convergence directions of local models in their respective parameter spaces, resulting in the global model failing to converge effectively.

[0070] In view of this, embodiments of this application provide a model training method, recommendation method and system for cross-border recommendation graph neural networks. This scheme enables the cross-border recommendation model of the federated client to focus on the common features of the fusion data from the local graph data through a new meta-path aggregation paradigm, ignoring the specific features of the data, forming a cross-border recommendation model with highly overlapping parameter space distribution, so that the global model parameters aggregated by the server can converge.

[0071] The cross-border recommendation model training method and cross-border recommendation method provided in this application relate to the field of information technology. The cross-border recommendation model training method and cross-border recommendation method provided in this application can be applied to a terminal, a server, or software running on a terminal or server. In some embodiments, the terminal can be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, or in-vehicle terminal, etc., but is not limited to these; the server can be configured as an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The server can also be a node server in a blockchain network; the software can be an application implementing the cross-border recommendation model training method and cross-border recommendation method, etc., but is not limited to the above forms.

[0072] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics devices, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0073] It should be noted that in various specific embodiments of this application, when processing data related to the identity or characteristics of an object, such as object information, object behavior data, object historical data, and object location information, the object's permission or consent is obtained first. Furthermore, the collection, use, and processing of this data comply with relevant laws, regulations, and standards. In addition, when embodiments of this application require obtaining sensitive personal information of an object, separate permission or consent from the object is obtained through pop-ups or redirection to a confirmation page. Only after obtaining the object's separate permission or consent is the necessary object-related data required for the proper functioning of the embodiments of this application obtained.

[0074] Figure 1 This is an optional flowchart of a cross-border recommendation graph neural network model training method provided in the embodiments of this application.

[0075] See Figure 1 The model training method shown in the embodiments of this application may include, but is not limited to, steps 101 to 104.

[0076] 101. Determine the object interaction sequence based on the historical interaction records of the target object, and determine the project interaction sequence based on the historical interaction records of the target project.

[0077] 102. Construct a dynamic bipartite graph through the object interaction sequence and the project interaction sequence. The dynamic bipartite graph includes bipartite graphs corresponding to N time steps, where N is an integer greater than or equal to 1.

[0078] 103. Construct or update a cross-border recommendation model, wherein during the information transmission process on the dynamic bipartite graph, the cross-border recommendation model aggregates the first embedding vector of the second-order item neighbors of the target object, wherein the second-order item neighbors are connected to the target object through implicit association objects, and aggregates the second embedding vector of the second-order object neighbors of the target object, wherein the second-order object neighbors are connected to the target object through implicit association objects;

[0079] 104. The cross-border recommendation model is trained based on the training triples and the dynamic bipartite graph, and the model update data is sent to the server so that the server can determine the global model based on the cross-border recommendation models of several clients.

[0080] In steps 101 to 105 of this embodiment, during the information transmission process on the dynamic bipartite graph through the cross-border recommendation model, the first embedding vector of the second-order item neighbors of the target object is aggregated. These second-order item neighbors are connected to the target object through implicit association objects. The first embedding vector represents the items liked by the target object. A second embedding vector is also aggregated from the second-order object neighbors of the target items. The first embedding vector represents the characteristics of the items liked by the target object. The second embedding vector represents the enthusiast characteristics of the target items. This application uses the above implicit relationships for aggregation constraints, enhancing the ability to learn generalizable representations from sparse and isolated local data in cross-border scenarios where objects do not overlap, making the parameter space distribution of the cross-border recommendation models of various clients more convergent.

[0081] It is understood that the cross-border recommendation model includes a GNN module and a perception module. The GNN module is used for graph data information aggregation to obtain feature representations; the perception module is used for predicting the input feature representations. The training triples include the target object, the target item, and the time step, with true labels between the target object and the target item.

[0082] The GNN module is a graph neural network, and the perception module is a multilayer perceptron. To address the lack of direct collaborative signals caused by the complete isolation of object groups in cross-border scenarios, the GNN module aims to indirectly establish preference associations between different object groups that are geographically or legally isolated by mining and fusing implicit semantic paths within the federated parameter space.

[0083] Figure 2 This is a flowchart of step 104 of the model training method shown in the embodiment of this application.

[0084] See Figure 2 In some embodiments, step 104 includes, but is not limited to:

[0085] 201. Encode the training triples using interactive scoring labels, and determine the adjacency matrix using the dynamic bipartite graph;

[0086] 202. Input the encoded training triples and the adjacency matrix into the cross-border recommendation model; wherein, the GNN module aggregates the first embedding vectors of the second-order item neighbors of the target object along the first meta-path to obtain the target object feature representation; the GNN module aggregates the second embedding vectors of the second-order object neighbors of the target item along the second meta-path to obtain the target item feature representation; the perception module predicts the interaction score based on the target object feature representation and the target item feature representation;

[0087] 203. Update the cross-border recommendation model based on the interaction rating labels and the prediction loss determined by the interaction ratings.

[0088] In most cases, explicit social relationships are sparse and biased, which greatly limits the effective use of social information about objects. This embodiment uses steps 201 to 203 to uncover potential implicit relationships between objects.

[0089] This embodiment assumes that there is a common following object between the target object and its second-order neighbors, and therefore, there is a similarity-based preference between them. Based on this idea, this embodiment proposes a first-order path mining method to uncover implicit object relationships.

[0090] For example, for the target object Implicit object relations The definition is as follows:

[0091] ,

[0092] in, For target object The set of neighbors of a second-order object. Representation Object Clearly define who to follow , Representation Object Clearly define who to follow , Representation and target object Having a common target of followers A collection of objects; This represents the cutoff threshold, which is used to determine the number of objects in a collection. For user groups.

[0093] In this embodiment, the target object is defined using the above method. The implicit object relationship is determined as a set. ,gather Each object in the list has a common follower object. Second-order object neighbors and target objects have similar preferences, and the representation of aggregated second-order object neighbors can reflect the degree of preference of common enthusiasts for the target item.

[0094] In some embodiments, the first meta-path aggregation in step 202 specifically includes, but is not limited to:

[0095] 301. Determine common following objects based on the target object and its first-order object neighbors;

[0096] 302. The first-order object neighbors of the common following object are determined as the implicit associated objects of the target object, and the embedding vectors of the first-order item neighbors of the implicit associated objects are passed as the first embedding vector.

[0097] 303. Aggregate all the first embedding vectors to obtain the feature representation of the target object.

[0098] Traditional item similarity metrics (such as the Pearson correlation coefficient) may overlook the scale of shared rating objects. This embodiment assumes that two items that multiple objects have interacted with with similar ratings are more likely to be similar, and identifies items that the same object has interacted with with similar interaction strengths as second-order item neighbors. Based on this, a second-order path is proposed to mine implicit relationships between items.

[0099] In some embodiments, a new similarity metric is constructed based on a common set of rating objects and their rating scales, thereby defining implicit item relationships. This allows for a more robust capture of potential connections between projects.

[0100] In some embodiments, the second meta-path aggregation in step 202 specifically includes, but is not limited to:

[0101] 401. Determine the first-order project neighbors of the target project based on the common rating object set; the first-order project neighbors are projects that have interaction records with objects in the common rating object set;

[0102] 402. Determine the first-order project neighbors of the target project as the implicitly associated project, and pass the embedding vector of the first-order object neighbor of the implicitly associated project as the second embedding vector.

[0103] 403. Aggregate all the second embedding vectors to obtain the feature representation of the target item.

[0104] In some embodiments, step 401 includes, but is not limited to, the following:

[0105] 501. Determine the common scoring object set through the historical interaction objects of the target project and the candidate projects;

[0106] 502. Calculate the interaction strength of each object in the common scoring object set with respect to the target item and the candidate items;

[0107] 503. Determine whether to identify the candidate item as a first-order item neighbor of the target item based on the interaction strength.

[0108] In this embodiment, the common scoring object set is defined as:

[0109] ,

[0110] in, For a common set of rating objects, For object For the project Interactive rating, For object For the project Interactive rating.

[0111] In some embodiments, step 502 includes:

[0112] 601. Determine the scale confidence level based on the number of objects in the aforementioned common scoring object set;

[0113] 602. Obtain the first rating of each object in the common rating objects for the target item, and the second rating of each object for the candidate item, and calculate the distance between the first rating and the second rating to determine the evaluation similarity;

[0114] 603. Determine the interaction strength based on the scale confidence and the evaluation similarity.

[0115] This embodiment proposes a new interaction strength that takes into account the proximity of the size confidence of the common object set to the rating scale.

[0116] In step 601, for example, the formula for calculating the interaction strength is:

[0117] ,

[0118] in, The interaction strength between the target project and the candidate project. For scale confidence, It is the size of the set of objects that are jointly rated. It is a smoothing hyperparameter used to avoid... When the value is very small, this value is too low. The more common objects there are, the closer this value is to 1, and the more reliable the similarity result is.

[0119] In this embodiment, To evaluate similarity, For optional object weights, similarity is evaluated using a Gaussian kernel-based transformation to measure the difference in object scores. For object The squared difference of the scores for the two items, As a preset parameter used to control the rate of decay of similarity, the transformation maps the score difference to a similarity value between 0 and 1, with the smaller the difference, the closer the value is to 1.

[0120] Figure 3 The dynamic bipartite graph constructed in step 102 of the training method shown in this embodiment.

[0121] In one embodiment, after aggregating the embedding vectors along the activated meta-paths, this embodiment employs a multi-head attention mechanism to automatically learn the importance weights of different meta-paths and different neighbors under the same path, thereby reducing the interference of unreliable or noisy paths.

[0122] For example, the multi-head attention mechanism is defined as follows:

[0123] ,

[0124] in, For nodes The final characteristic representation, For nodes The input feature vector, For concatenation functions, The number of heads for multi-head attention. The number of the attention head. It is the first Node in the head For nodes Attention weights For the first The linear transformation matrix of the size, For nodes The neighborhood group, This is the activation function.

[0125] Based on the multi-head attention mechanism described above, in order to obtain node representation learning with stronger expressive power, this embodiment expects that when a node updates its own feature representation vector, it can not only aggregate messages from first-order neighbors, but also aggregate messages from k-order neighbors.

[0126] For example, node The aggregation method is as follows:

[0127] ,

[0128] in, For the current node at the th Layer from The message vector obtained by order neighbor aggregation, For message functions (such as mean pooling or weighted summation). For nodes exist Layer representation, represent Connected components, The index of the connected component. For nodes of Neighbor set of order, For nodes and neighboring nodes The edge, For time steps, Let the order of the neighbors be . The total order of the neighbors;

[0129] ,

[0130] in, For nodes In the The first layer of aggregation The intermediate representation generated after the messages from the next-order neighbor is based on the node. In the The layer representation aggregation comes from the first layer. Generate messages from neighboring units; This is the state update function (usually a linear transformation plus nonlinear activation). For nodes exist Layer representation;

[0131] ,

[0132] in, For nodes In the The final representation of the layer, For the first Layer A fusion function (e.g., concatenation or weighted average) representing intermediate orders.

[0133] For example, following the message aggregation method shown above, targeting the target object The Layer, with and As input features, then output features It can be represented as:

[0134] ,

[0135] in, This indicates the number of network layers in the GNN model. For target object Neighbors For target object The neighborhood group, It is an aggregate function. Represents the target object Its own node characteristics, This represents the characteristics of neighboring nodes.

[0136] The embeddings obtained from each layer are combined to form the first embedding vector of the target object. :

[0137] ,

[0138] Similarly, the second embedding vector of the target item :

[0139] ,

[0140] in, This represents the total number of aggregation levels for the messages of the target object. This represents the total number of aggregation layers for messages from the target project. Indicates the serial number.

[0141] Given an interactive triple This allows for the prediction of the target object. With the target project Predicted score

[0142] ,

[0143] in, Predictive scores for target objects and target projects. This represents a multilayer perceptron. For target object At the current time step Embedded representation, For the target project At the current time step Embedded representation, The first embedding vector of the target object is output by the GNN module. The second embedding vector of the target item is output by the GNN module.

[0144] In some embodiments, step 102 includes:

[0145] 701. Convert each interaction behavior in the object interaction sequence into an object dense vector, and embed the object dense vector according to time decay to obtain the embedded representation of the target object;

[0146] 702. Convert each interaction behavior in the project interaction sequence into a project dense vector, and embed the project dense vector according to time decay to obtain the embedded representation of the target project;

[0147] 703. Construct the dynamic bipartite graph based on the embedding representation of the target object and the embedding representation of the target item.

[0148] See Figure 3 In cross-border business, object preferences and project popularity evolve over time. This embodiment captures and quantifies this dynamic evolution pattern derived from its private data through steps 701 to 703, and constructs a corresponding time-series graph representation.

[0149] ①The embedded representation of the target object:

[0150] In step 701, in At any given moment, the sequence of interactions between the object and the project. Then the object Historical Behavioral Sequence for:

[0151] ,

[0152] ,

[0153] in, Indicates the length of the object's behavior sequence; Representative object The t-th historical action can contain various auxiliary information; Representative object The k-th auxiliary information of the i-th action is generally the material ID, category, and time of the action.

[0154] ②For objects Historical Behavioral Sequence Transform each behavior of an object into a dense vector:

[0155] ,

[0156] By feeding each interactive item along with a time decay in the sequence into the embedding layer, the embedded sequence of object behavior can be represented as:

[0157] ,

[0158] Similarly, the project Historical Behavioral Sequence .

[0159] in, Indicates the length of the item's behavior sequence; For auxiliary messages, Representative projects The Secondary behavior Auxiliary information.

[0160] Historical behavioral sequence of the project Each behavior of an item is transformed into a dense vector:

[0161] ,

[0162] By feeding each interacting object along with the time decay in the sequence into the embedding layer, the embedded sequence of item behaviors can be represented as:

[0163] ,

[0164] Therefore, the resulting embedding is used as the time-aware zero-layer input. , .

[0165] Therefore, each client can transform raw, timestamped interaction data into a structured graph embedding representation rich in dynamic information, while strictly protecting user privacy. This lays the foundation for the next stage of cross-client high-order semantic reasoning in the parameter space (rather than the data space).

[0166] In one embodiment, the loss function of the cross-border recommendation model is defined as:

[0167] ,

[0168] in, For model parameters, Given the number of ratings, This represents the model's prediction score for the training samples. Represents objects in the training set For the project The actual rating label at the current time step.

[0169] Figure 4 This is another flowchart of the training method shown in this embodiment.

[0170] Please see Figure 4 This application also provides a model training method for a cross-border recommendation graph neural network, which is applied to the server side of federated learning to aggregate the cross-border recommendation model obtained by the model training method of the client side.

[0171] like Figure 4 As shown, the model training method illustrated in this embodiment includes, but is not limited to, the following:

[0172] 801. Obtain the model update data of the cross-border recommendation model uploaded by K clients, where K is an integer greater than or equal to 1;

[0173] 802. Determine the global model based on the model update data of the cross-border recommendation model.

[0174] In one embodiment, step 801 may further include: obtaining gradient update and training information of the K clients in the current update round; the training information includes the number of client samples and the client prior confidence.

[0175] In one embodiment, step 802 may further include: determining the aggregate weight of the Kth client based on the training information; aggregating the gradient updates of the Kth client based on the aggregate weight of the Kth client respectively; and distributing the updated global model to the K clients.

[0176] Specifically, after receiving updates from each client, the server does not directly perform a weighted average based on data volume, but rather dynamically calculates the contribution weight of each client based on multi-dimensional indicators. .

[0177] For example, the contribution weight is:

[0178] ,

[0179] in, Indicates the client The sample size. Although the number of client users / projects varies, the aggregate weight takes into account the overall data volume. The quality of the graph structure ensures the fairness of the contribution. The prior confidence level for the local graph structure on the client side can be set based on metrics such as graph density and connectivity. and This is a hyperparameter used to balance the weights between data volume, update stability, and structural quality. The function is for client-side stability.

[0180] Understandable The function is an evaluation function used to quantify the stability of cross-border recommendation model updates for clients. Its core function is to determine whether a client provides reliable and consistent parameter updates during federated learning training, thereby assigning it appropriate weights during aggregation.

[0181] In one embodiment, The function can be designed based on the consistency of the update magnitude to calculate the cosine similarity or Euclidean distance change rate of the parameter update vectors in consecutive rounds of client updates. The smaller the change, the more stable the update.

[0182] For example, The function is defined as follows:

[0183] ,

[0184] in, For the client number; This represents the current training round (or the length of the time window being considered). Indicates the client In the In each training round, relative to the previous round of the global model The parameter update vector; Similarity functions, such as cosine similarity, are used to measure the consistency of update directions between two rounds.

[0185] In this embodiment, The closer the value is to 1, the more stable the client updates are.

[0186] In one embodiment, The function can be designed based on the stability of the loss decrease. If the client's loss function value decreases smoothly and without drastic fluctuations, it is considered stable.

[0187] For example, The function is defined as follows:

[0188] ,

[0189] in, For the client In recent The sequence of loss values ​​during rounds of training; It is a variance function that measures the degree of fluctuation in the loss value. This represents the average value of the loss over that time period.

[0190] In this embodiment, the smaller the output loss fluctuation, the smaller the variance, and the closer the exponential term is to 1, the higher the stability.

[0191] Figure 5 This is a flowchart of the cross-border object recommendation method shown in this embodiment.

[0192] Please see Figure 5 After obtaining the cross-border recommendation model, this application embodiment also provides a cross-border object recommendation method, including:

[0193] 901. Input the interaction triples into the global model to obtain the interaction score; the interaction triples include the target object, the target item, and the time step, and the interaction score represents the degree of interest of the target object in the target item;

[0194] 902. If the interaction score is greater than the preset interaction score threshold, the target item is pushed to the target object.

[0195] Specifically, the interaction triple encodes the ID of the target object, the target item, and the current time step;

[0196] This application also provides a cross-border object recommendation system, including a processor and a memory, wherein the memory stores computer-readable executable code, which, when executed by the processor, implements the method described in this application.

[0197] This cross-border object recommendation system can be used on any smart terminal, including tablets, in-vehicle computers, etc.

[0198] It is understood that the content of the above method embodiments is applicable to this device embodiment. The specific functions implemented by this device embodiment are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0199] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0200] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0201] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0202] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0203] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0204] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0205] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0206] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0207] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0208] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0209] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. A method for training a cross-border recommendation graph neural network model, characterized in that, The method, applied to a client for federated learning, includes: Determine the object interaction sequence based on the target object's historical interaction records, and determine the project interaction sequence based on the target project's historical interaction records. A dynamic bipartite graph is constructed using the object interaction sequence and the project interaction sequence. The dynamic bipartite graph includes bipartite graphs corresponding to N time steps, where N is an integer greater than or equal to 1. Construct or update a cross-border recommendation model, wherein during the information transmission process on the dynamic bipartite graph, the cross-border recommendation model aggregates the first embedding vector of the second-order item neighbors of the target object, wherein the second-order item neighbors are connected to the target object through implicit association objects, and aggregates the second embedding vector of the second-order object neighbors of the target object, wherein the second-order object neighbors are connected to the target object through implicit association objects; The cross-border recommendation model is trained based on the training triples and the dynamic bipartite graph, and the model update data is sent to the server so that the server can determine the global model based on the cross-border recommendation models of several clients. Constructing a dynamic bipartite graph through the object interaction sequence and the project interaction sequence specifically includes: Each interaction behavior in the object interaction sequence is converted into an object dense vector, and the object dense vector is embedded according to time decay to obtain the embedded representation of the target object; Each interaction behavior in the project interaction sequence is converted into a project dense vector, and the project dense vector is embedded according to time decay to obtain the embedded representation of the target project; The dynamic bipartite graph is constructed based on the embedding representation of the target object and the embedding representation of the target item; The cross-border recommendation model is equipped with a GNN module and a perception module; The training of the cross-border recommendation model based on the training triples and the dynamic bipartite graph specifically includes: The training triples are encoded using interactive scoring labels, and the adjacency matrix is ​​determined using the dynamic bipartite graph. The encoded training triples and the adjacency matrix are input into the cross-border recommendation model; wherein, the GNN module aggregates the first embedding vectors of the second-order item neighbors of the target object along the first meta-path to obtain the target object feature representation; the GNN module aggregates the second embedding vectors of the second-order object neighbors of the target item along the second meta-path to obtain the target item feature representation; the perception module predicts the interaction score based on the target object feature representation and the target item feature representation; The cross-border recommendation model is updated based on the interaction rating labels and the prediction loss determined by the interaction ratings.

2. The model training method as described in claim 1, characterized in that, The GNN module aggregates the first embedding vectors of the second-order item neighbors of the target object along the first metapath to obtain the feature representation of the target object, specifically including: Determine common following objects based on the target object and its first-order object neighbors; The first-order object neighbors of the common following object are determined as the implicit associated objects of the target object, and the embedding vectors of the first-order item neighbors of the implicit associated objects are passed as the first embedding vector. By aggregating all the first embedding vectors, the feature representation of the target object is obtained.

3. The model training method as described in claim 1, characterized in that, The GNN module aggregates the second embedding vectors of the second-order object neighbors of the target item along the second-order path to obtain the feature representation of the target item, specifically including: The first-order project neighbors of the target project are determined based on the common rating object set; the first-order project neighbors are projects that have interaction records with objects in the common rating object set. The first-order project neighbors of the target project are determined as the implicitly associated project, and the embedding vectors of the first-order object neighbors of the implicitly associated project are passed as the second embedding vector. Aggregate all the second embedding vectors to obtain the feature representation of the target item.

4. The model training method as described in claim 3, characterized in that, The first-order project neighbors of the target project are determined based on the set of common scoring objects, specifically including: A common scoring object set is determined by the historical interaction objects of the target project and the candidate project; Calculate the interaction strength of each object in the set of common scoring objects with respect to the target item and the candidate items; The candidate item is determined as a first-order item neighbor of the target item based on the interaction strength.

5. The model training method as described in claim 4, characterized in that, Calculating the interaction strength of each object in the common rating object set with respect to the target item and the candidate items specifically includes: The scale confidence level is determined based on the number of objects in the common scoring object set; Obtain the first rating of each object in the common rating object for the target item, and the second rating of each object for the candidate item, and calculate the distance between the first rating and the second rating to determine the evaluation similarity; The interaction strength is determined based on the scale confidence and the evaluation similarity.

6. A method for training a cross-border recommendation graph neural network model, characterized in that, A server-side application for federated learning, wherein the server-side aggregates cross-border recommendation models obtained by the method according to any one of claims 1 to 5, the method comprising: Obtain the model update data of the cross-border recommendation model uploaded by K clients, where K is an integer greater than or equal to 1; The global model is determined based on the model update data of the cross-border recommendation model.

7. A cross-border object recommendation method, characterized in that, The global model obtained based on the model training method described in claim 6 recommends target items to the target object, specifically including: The interaction triple is input into the global model to obtain the interaction score; the interaction triple includes the target object, the target item, and the time step, and the interaction score represents the degree of interest of the target object in the target item. If the interaction score is greater than a preset interaction score threshold, the target item is pushed to the target object.

8. A cross-border object recommendation system, characterized in that, The method includes a processor and a memory, wherein the memory stores computer-readable executable code that, when executed by the processor, implements the method according to any one of claims 1 to 7.

Images