A recommendation model training method and device

Abstract

The embodiment of the specification provides a training method and device of a recommendation model, the model comprising a first graph neural network, N expert networks arranged in parallel, and a plurality of gating networks and prediction networks corresponding to a plurality of recommendation tasks respectively, the method comprising: obtaining a two-part graph and a label set; processing the two-part graph by using the first graph neural network to obtain a first user representation of a target user and a first object representation of a target object; processing user features of the target user and object features of the target object in parallel by using the expert networks to obtain expert output representations; determining inputs of the prediction networks through a combination operation, so that the prediction networks generate prediction data corresponding to the recommendation tasks; the combination operation comprises combining the expert output representations, the first user representation and the first object representation based on outputs of the corresponding gating networks; and training the recommendation model by using the prediction data corresponding to each recommendation task and label data corresponding to each recommendation task in the label set.

Description

Technical Field

[0001] This specification relates to the field of graph neural network technology, and in particular to a method and apparatus for training a recommendation model. Background Technology

[0002] Multi-task learning-based neural network architectures are widely used in various industrial applications, from entity detection to online advertising and product recommendations. However, the accuracy of recommendation models built on current multi-task learning-based neural network architectures is still not high enough.

[0003] Therefore, how to provide a recommendation model with high accuracy in recommending results has become an urgent problem to be solved. Summary of the Invention

[0004] This specification provides one or more embodiments of a method and apparatus for training a recommendation model, so as to achieve a recommendation model with high accuracy in training recommendation results.

[0005] According to the first aspect, a method for training a recommendation model is provided, the recommendation model comprising a first graph neural network, N parallel expert networks, and several gating networks and prediction networks corresponding to each recommendation task, the method comprising:

[0006] Obtain a bipartite graph and a tag set, wherein each node in the bipartite graph corresponds to a user or object, and the first edge represents the association between the user and the object; the tag set includes several tag data corresponding to the several recommendation tasks between the target user and the target object connected by the first edge.

[0007] Using a first graph neural network, the bipartite graph is processed to obtain a first user representation of the target user and a first object representation of the target object;

[0008] By utilizing various expert networks, the user characteristics of the target user and the object characteristics of the target object are processed in parallel to obtain the output representations of each expert.

[0009] The inputs to each prediction network are determined by combining operations, so that each prediction network generates prediction data for the corresponding recommendation task; the combining operations include combining the expert output representations, the first user representation, and the first object representation based on the outputs of the corresponding gating networks.

[0010] The recommendation model is trained using the prediction data and label data corresponding to each recommendation task.

[0011] According to the second aspect, a training device for a recommendation model is provided, the recommendation model comprising a first graph neural network, N parallel expert networks, and several gating networks and prediction networks corresponding to each recommendation task, the device comprising:

[0012] The first acquisition module is configured to acquire a bipartite graph and a tag set, wherein each node in the bipartite graph corresponds to a user or object, and the first edge represents the association relationship between the user and the object; the tag set includes several tag data corresponding to the several recommendation tasks between the target user and the target object connected by the first edge.

[0013] The first processing module is configured to use a first graph neural network to process the bipartite graph to obtain a first user representation of the target user and a first object representation of the target object.

[0014] The second processing module is configured to utilize each expert network to process the user features of the target user and the object features of the target object in parallel, and obtain the output representations of each expert.

[0015] The first determining module is configured to determine the input of each prediction network through a combination operation, so that each prediction network generates prediction data for the corresponding recommendation task; the combination operation includes combining the expert output representation, the first user representation, and the first object representation based on the output of the corresponding gating network.

[0016] The training module is configured to train the recommendation model using the prediction data and label data corresponding to each recommendation task.

[0017] According to a third aspect, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed in a computer, causes the computer to perform the method described in the first aspect.

[0018] According to a fourth aspect, a computing device is provided, including a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, it implements the method described in the first aspect.

[0019] According to the training method and apparatus for the recommendation model provided in the embodiments of this specification, the recommendation model includes a first graph neural network, N parallel expert networks, and several gating networks and prediction networks corresponding to each recommendation task. After obtaining a bipartite graph representing the relationship between users and objects, and a label set including several label data corresponding to several recommendation tasks between target users and target objects connected by a first edge, the first graph neural network is used to process the bipartite graph to obtain a first user representation of the target user aggregated with its own features and the features of its neighbors, and a first object representation of the target object aggregated with its own features and the features of its neighbors. Then, each expert network is used to extract the output representations of each expert network containing the information they value from the user features of the target user and the object features of the target object. Through a combination operation, that is, based on the output of the corresponding gating network of each prediction network, the expert output representations, the first user representations and the first object representations are combined to obtain the inputs of each prediction network, so that each prediction network generates prediction data corresponding to the recommendation task. Then, the recommendation model is trained based on the prediction data and label data of each recommendation task. In the above process, by combining a graph neural network that can obtain more accurate representations of users and objects with an expert network that can obtain expert output representations containing information that each party values, the recommendation task is completed in a collaborative manner by introducing the corresponding expert network based on learning the deep interaction context between users and objects. This improves the accuracy of the recommendation results of each recommendation task to a certain extent. Attached Figure Description

[0020] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are merely some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.

[0021] Figure 1A A flowchart illustrating a training method for a recommendation model provided in an embodiment;

[0022] Figure 1B An exemplary structural diagram of the recommendation model provided in this embodiment;

[0023] Figure 1C A partial schematic diagram of the bipartite diagram provided for an embodiment;

[0024] Figure 2A A flowchart illustrating a training method for a recommendation model provided in an embodiment;

[0025] Figure 2B A schematic diagram of yet another exemplary structure of the recommendation model provided in this embodiment;

[0026] Figure 3A A flowchart illustrating a training method for a recommendation model provided in an embodiment;

[0027] Figure 3B A schematic diagram of yet another exemplary structure of the recommendation model provided in this embodiment;

[0028] Figure 4A A flowchart illustrating a training method for a recommendation model provided in an embodiment;

[0029] Figure 4B A schematic diagram of yet another exemplary structure of the recommendation model provided in this embodiment;

[0030] Figure 5 A schematic block diagram of a training apparatus for a recommended model provided in an embodiment. Detailed Implementation

[0031] The technical solutions of the embodiments of this specification will now be described in detail with reference to the accompanying drawings.

[0032] The training method of the recommended model provided in this specification will be described in detail below with reference to specific embodiments.

[0033] Figure 1A A flowchart illustrating a training method for a recommended model in one embodiment of this specification is shown. This method can be implemented using any device, equipment, platform, device cluster, etc., with computing and processing capabilities. Figure 1B As shown, the recommendation model may include a first graph neural network, N parallel expert networks, and several gating networks and prediction networks corresponding to each recommendation task. Figure 1B (as well as Figure 2B , Figure 3B and Figure 4B As shown in the diagram, there are N expert networks, including expert network 1, expert network 2, ..., expert network N; several recommendation tasks are as follows: Figure 1B (as well as Figure 2B , Figure 3B and Figure 4B The example can include recommendation task 1 and recommendation task 2. Accordingly, recommendation task 1 corresponds to gating network 1 and prediction network 1, and recommendation task 2 corresponds to gating network 2 and prediction network 2.

[0034] The training process of this recommendation model, such as Figure 1A As shown, the method includes the following steps S110-S150:

[0035] In step S110, a bipartite graph and a tag set are obtained, wherein each node in the bipartite graph corresponds to a user or object, and the first edge represents the relationship between the user and the object; the tag set includes several tag data between the target user and the target object connected by the first edge, corresponding to several recommendation tasks.

[0036] In one implementation, the bipartite graph This can also be called a user-object bipartite graph. This bipartite graph can be constructed based on the relationships between users and objects within the target platform. It can include multiple nodes and edges; for clarity, each edge in the bipartite graph is referred to as the first edge. Each node in the bipartite graph corresponds to a user or object, and the first edge represents the relationship between users and objects, which is related to several recommendation tasks. A partial schematic diagram is shown below. Figure 1C As shown, the nodes represented by circles are the nodes corresponding to the users, the nodes represented by squares are the nodes corresponding to the objects, the neighboring nodes of the user's node are all the nodes corresponding to the objects, and the neighboring nodes of the object's node are all the nodes corresponding to the users.

[0037] This tag set includes: several tag data corresponding to the recommendation tasks between the target users and target objects connected by the first edge. A set of tags for users and objects (e.g., target users and target objects) connected by the first edge can be represented as (u, o, y). u，o ), where u represents a user, belonging to the user set U of the target platform; o represents an object, belonging to the object set O of the target platform, and y u，o This represents the label data for user u and object o, y u，o Belongs to {0,1} K K represents the number of recommendation tasks. The first value, e.g., 1, represents the label data between user u and object o corresponding to a specific recommendation task, indicating an association between them. The second value, e.g., 0, represents the absence of such an association. The target user and target object can be any pair of user-object pairs with label data corresponding to the recommendation tasks involved in the bipartite graph.

[0038] Each user and object involved in the bipartite graph is also associated with its own initial attribute features, where the initial attribute features of a user can be represented as x. u The initial attribute characteristics of an object can be represented as x. o .

[0039] In one exemplary scenario, the target platform can be an e-commerce platform, and correspondingly, the object can be a merchant or a product. In one case, if the object is a merchant, the recommendation tasks can include clicks and conversions, such as converting a user into a member of the merchant; the relationship between the user and the object can be clicks or conversions. If the object is a product, the recommendation tasks can include clicks and purchases; the relationship between the user and the object can be clicks or purchases. The initial attribute characteristics of the user can include, but are not limited to, the following dimensions: age, hobbies, occupation, and user type tags, etc.; the initial attribute characteristics of the object can include, but are not limited to, the following dimensions: the category of the included product (or the product itself), brand, purpose, etc.

[0040] In another exemplary scenario, the target platform can be a media platform, and correspondingly, the object can be an advertisement (or other news article). The recommendation tasks can be: clicks and conversions (or clicks and favorites, follows), where the conversion can refer to purchasing the product involved in the advertisement, and the relationship between the user and the object can be clicks or conversions (or clicks, favorites, or follows). The initial attribute characteristics of the user can include, but are not limited to, the following dimensions of attribute characteristics: age, hobbies, occupation, and user type tags, etc.; the initial attribute characteristics of the object can include, but are not limited to, the following dimensions of attribute characteristics: the category, brand, and purpose of the product involved (or the type and content description of the news article involved), etc.

[0041] In another exemplary scenario, the target platform can be a short-video platform, and the corresponding object can be a user. The recommendation tasks can be: clicking and following (or adding friends), and the relationship between the user and the object can be clicking or following (or adding friends). The initial attribute features corresponding to the user can include, but are not limited to, the following dimensions of attribute features: age, hobbies, occupation, and user type tags, etc.; the initial attribute features corresponding to the object can include, but are not limited to, the following dimensions of attribute features: age, hobbies, and the video field it belongs to (e.g., food, beauty, travel, etc.).

[0042] In one embodiment, the recommendation model may further include a transformation layer that can transform the user's initial attribute features x u and the initial attribute characteristics x of the object o This is mapped to low-dimensional dense features (as vectors, i.e., the user features of users and the object features of objects mentioned later), thereby enhancing the expressive power of the features and facilitating subsequent processing by the first graph neural network, n expert networks, and several gating networks. For example... Figure 1BAs shown, this transformation layer is placed before the first graph neural network. This transformation layer can include user feature transformation matrices and object feature transformation matrices. Specifically, it can first obtain the initial attribute features x of each user (including the target user) involved in the bipartite graph (or the target subgraph mentioned later). u and the initial attribute characteristics x of each object (including the target object) o The initial attribute features x of each user are processed using the user feature transformation matrix. u The user characteristics p of each user (including the target user) are obtained. u The initial attribute features x of each object (including the target object) are processed using the object feature transformation matrix. o To obtain the object characteristics p of each object o Then, the user characteristics p of each user will be... u and the object characteristics p of each object o , which serves as the input to the first graph neural network, N expert networks, and several gated networks.

[0043] The user feature transformation matrix and the object feature transformation matrix in this transformation layer can be pre-set or trainable.

[0044] Next, in step S120, the first graph neural network is used to process the bipartite graph to obtain the first user representation of the target user and the first object representation of the target object. In this step, processing the bipartite graph using the first graph neural network means that in the bipartite graph, the node corresponding to the target user and the node corresponding to the target object are respectively taken as target nodes. Using the first graph neural network, based on the characteristics of the target node and its neighboring node set, the representations of the target nodes are aggregated to obtain the first user representation of the target user. and the first object representation of the target object When the target node is the node corresponding to the target user, its feature is the user feature p of the target user. u When the target node is the node corresponding to the target object, its feature is the object feature p of the target object. o The characteristics of neighboring nodes are similar and will not be repeated here. Among them, the obtained first user representation... It can integrate the characteristics of the target user and the characteristics of its neighbors in the bipartite graph. The resulting first object representation It can integrate the characteristics of the target object itself and the characteristics of its neighbors in the bipartite graph.

[0045] In one implementation, a target subgraph corresponding to the target node (the node corresponding to the target user or target object) can first be extracted from the bipartite graph. This target subgraph includes the target node and its set of neighboring nodes, which can include the neighboring nodes within L hops of the target node. Then, a first graph neural network is used to process the target subgraph corresponding to the target user and the target subgraph corresponding to the target object, respectively, to obtain the first user representation of the target user and the first object representation of the target object, thus saving computational resources. The value of L is related to the number of aggregation layers in the first graph neural network.

[0046] Accordingly, in one implementation, the first graph neural network may include L aggregation layers;

[0047] The method of using a first graph neural network to represent and aggregate the target node based on the characteristics of the target node and its neighboring nodes may include the following steps: sequentially performing L-level vector aggregation for the target node in L aggregation layers, wherein each level of vector aggregation includes performing neighbor aggregation on the aggregation vector of the previous level of each neighboring node in the set of neighboring nodes of the target node, and determining the current level aggregation vector of the target node based on the neighbor aggregation result and the aggregation vector of the previous level of the target node, wherein the primary aggregation vector of each node is determined based on the characteristics of the corresponding user or object (if the node corresponds to a user, the primary aggregation vector is determined based on the user characteristics of the corresponding user; if the node corresponds to an object, the primary aggregation vector is determined based on the object characteristics of the corresponding object).

[0048] Specifically, in order to perform L-level vector aggregation for the target node v in L aggregation layers sequentially, firstly, the primary aggregation vector of the target node v is determined based on the characteristics of the target node v. And based on the characteristics of each neighbor node j in the neighbor node set, determine the primary aggregation vector of each neighbor node j. Then based on the primary aggregation vector and primary aggregate vector Perform L-level vector aggregation on the target node v to obtain the aggregation vector of the Lth aggregation layer. This aggregated representation serves as the aggregate representation corresponding to the target node, where the target node corresponds to the target user. That is, the first user representation The target node corresponds to the target object, and this aggregation vector That is, the first object representation

[0049] In each level of vector aggregation, each neighbor node j in the aforementioned neighbor node set is a node connected to the target node v. It can be understood that at each aggregation layer, in addition to performing vector aggregation on the target node, vector aggregation is also required on each neighbor node to obtain the current-level aggregated vector for each neighbor node. When performing vector aggregation on each neighbor node, it is necessary to utilize the previous-level aggregated vector of that neighbor node and the previous-level aggregated vector of its neighbor nodes, and so on, so that in the l-th aggregation layer, the current-level aggregated vector of the target node includes the features of the nodes in the l-th order of the target node's neighbors. Thus, after L aggregation layers, the aggregated vector of the target node... The aggregate contains the characteristics of the target node v and its L-hop neighbors.

[0050] In one embodiment, the above determination of the primary aggregation vector The process can be as follows: encoding the features (user features or object features) of the target node v, and using the encoded vector as the primary aggregation vector. Alternatively, the encoded vector can be subjected to linear or nonlinear transformations, and the transformed vector can be used as the primary aggregate vector.

[0051] In one implementation, for the l-th aggregation layer, it can perform neighbor aggregation on the (l-1)-level aggregation vectors of the neighboring nodes of the target node v based on an attention mechanism. Then, according to the corresponding aggregation function, the aggregation result of the neighbor aggregation and the (l-1)-level aggregation vector of the target node v are used to determine the current (l)-level aggregation vector of the target node v. This can be expressed by the following formula (1):

[0052]

[0053] in, This represents the aggregation vector of the l-th layer of the target node v. Represents the set of neighboring nodes of the target node v The aggregation vector of the j-th neighbor node at the (l-1)-th layer. α represents the aggregation vector of the (l-1)th layer of the target node v. v←j The attention value of the j-th neighbor node relative to the target node v can be determined based on any attention mechanism algorithm in related technologies; W l The parameters represent the neighbor aggregation process in the l-th aggregation layer, f(.) represents the aggregation function of the l-th aggregation layer, and σ(.) represents the activation function, such as the LeakyReLU activation function.

[0054] It is understood that the above aggregation process is merely an exemplary example and does not constitute a limitation on the aggregation process of the embodiments described herein.

[0055] The aggregation operation of the L-th aggregation layer can obtain the aggregation vector of the L-th aggregation layer of the target node, that is, the first user representation of the target user corresponding to the target node or the first object representation of the target object corresponding to the target node.

[0056] Then, in step S130, the user features of the target user and the object features of the target object are processed in parallel using various expert networks to obtain the output representations of each expert. In one embodiment, in this step, the user features p of the target user can be concatenated first. u and the object characteristics p of the target object o To obtain splicing features Next, the splicing features will be... Each expert network is input separately, and each expert network performs splicing feature analysis. The process is performed to obtain the expert output representations of each expert network. The expert output representation of the nth expert network can be expressed as: n is an integer from 1 to N. "||" represents the concatenation operation.

[0057] In one implementation, the nth expert network f n (.) This can be implemented using a multilayer perceptron with a LeakyReLU activation function. The specific number N of expert networks can be set according to the actual situation, for example, it can be no less than 2. Each expert network can separately extract features from the splicing. Extract information from the perspectives they deem important, so that each recommendation task can make recommendations accordingly.

[0058] Understandable. Figure 1A The training process of the recommendation model shown is only a schematic diagram. Step S130 can also be executed before step S120, or in parallel with step S120.

[0059] Then in step S140, the input of each prediction network is determined by a combination operation, so that each prediction network generates prediction data for the corresponding recommendation task; the combination operation includes combining the expert output representation, the first user representation and the first object representation based on the output of the corresponding gating network.

[0060] In one implementation, the splicing features can be first... Input each gating network separately to obtain the output of each gating network (also called the gating output result). The output of the gating network for the k-th recommendation task can be expressed as: It can represent the prediction participation weight of each expert network in n expert networks participating in the prediction of the k-th recommendation task. The value of k is an integer from 1 to K. In one case, the gating network g of the k-th recommendation task... k (.) can be implemented using a linear transformation layer with a softmax activation function.

[0061] Next, a combination operation is performed. Specifically, based on the outputs of the gating networks corresponding to each prediction network, the expert output representations, the first user representation, and the first object representation are combined. In one implementation, the expert output representations can be determined first using the first user representation and the first object representation. (i.e., the associated representations of each expert network) (for example, the associated representation of the nth expert network can be represented as Z) n Then, each subordinate representation is fused with the corresponding expert output representation of the expert network to obtain the comprehensive expert representation of each expert network. This fusion can be performed in various ways, such as pooling or concatenation. Typically, the fusion operation here uses concatenation. That is, the subordinate representation corresponding to the nth expert output representation is concatenated to the nth expert output representation to obtain the nth concatenated representation. This serves as the expert synthesis representation for the nth expert network. Subsequently, based on the outputs of the gating networks corresponding to each prediction network, the expert synthesis representations are combined.

[0062] Among them, the first user representation is used and the first object representation The process of determining the associated representations for each expert network can be achieved by processing the first user representation using the interaction function κ(·,·). and the first object representation Obtain the corresponding interactive representation This interaction representation These serve as auxiliary representations for each expert network. The interaction function κ(·,·) can be set as an element-wise multiplication operation, an element-wise addition operation, a concatenation operation, or a multilayer perceptron (the parameters of which are trainable).

[0063] The process of combining the comprehensive representations of various experts based on the outputs of the gating networks corresponding to each prediction network can be as follows: For the prediction network of the k-th recommendation task, the output of the k-th gating network is used. The comprehensive representations of various experts are combined to obtain the input (data) of the prediction network for the k-th recommendation task, which can be represented by the following formula (2):

[0064]

[0065] in, This represents the output of the k-th gated network. The corresponding prediction participation weight of the nth expert network. This represents the input to the prediction network for the k-th recommendation task.

[0066] Using the aforementioned method, the input (data) of the prediction network for each recommendation task is obtained. Then, the input (data) of the prediction network for each recommendation task is input into each prediction network to obtain the output of each prediction network, i.e., the prediction data for each recommendation task. The process of obtaining the output of the k-th prediction network can be represented by the following formula (3):

[0067]

[0068] in, This represents the output of the k-th prediction network, i.e., the prediction data for the k-th recommendation task. v represents the parameters of the k-th prediction network. k The transpose of is a trainable parameter.

[0069] After obtaining the prediction data for each recommendation task using the aforementioned method, in step S150, the recommendation model is trained using the prediction data and label data corresponding to each recommendation task, i.e., the model parameters of the recommendation model are adjusted.

[0070] In one embodiment, step S150 may specifically include steps 11-12: In step 11, the total prediction loss is determined using the prediction data and label data corresponding to each recommendation task.

[0071] In one implementation, the total prediction loss can be determined based on a preset loss function, using the prediction data and label data corresponding to each recommendation task. This preset loss function can be the cross-entropy loss function. In one case, the process of determining the total prediction loss can be represented by the following formula (4):

[0072]

[0073] in, This can represent the total predicted loss. as well as Both represent the prediction loss corresponding to the k-th recommendation task (i.e., the initial prediction loss for subsequent tasks). Represents the cross-entropy loss function. This represents the set of label data corresponding to the k-th recommendation task, representing a set of user-object pairs (e.g., target user and target object) connected by the first edge. This represents a set of user object pairs that correspond to a set of tags (users and objects connected by a first side, such as target user and target object).

[0074] In another implementation, considering the inherent uncertainty in the predicted data for each recommendation task, a corresponding uncertainty index can be learned for each recommendation task during the training process, based on the principle of task-dependent uncertainty. This allows for adaptive balancing of the prediction loss for each recommendation task during training, leading to better adjustment of the model parameters. Accordingly, step 11 may include steps 111-113. In step 111, the initial prediction loss for any recommendation task is determined using the predicted and label data. Specifically, this step can be based on a preset loss function, where the preset loss function can be the cross-entropy loss function.

[0075] Next, in step 112, based on the uncertainty index corresponding to each recommendation task and the initial prediction loss corresponding to each recommendation task, the adjusted prediction loss corresponding to each recommendation task is determined, wherein the uncertainty index is a trainable parameter. In this step, the uncertainty index corresponding to each recommendation task can characterize the uncertainty (i.e., possibility and probability) of the recommendation task predicting a certain prediction data. The determination process of the adjusted prediction loss corresponding to the k-th recommendation task can be represented by the following formula (5):

[0076]

[0077] in, Let σ represent the adjusted prediction loss for the k-th recommendation task. k This represents the uncertainty index corresponding to the k-th recommendation task.

[0078] Then, in step 113, the total prediction loss is determined based on the adjusted prediction loss corresponding to each recommendation task.

[0079] In one implementation, the sum of the adjusted prediction losses for each recommendation task can be directly used to determine the total prediction loss. Accordingly, the total prediction loss can be expressed as the following formula (6):

[0080]

[0081] in, This represents the total predicted loss. This represents the set of model parameters for the recommendation model. This represents the set of uncertainty indicators corresponding to each recommendation task.

[0082] In one implementation, to ensure the stability of the recommendation model's training process, the sum of the adjusted prediction losses and their regularization terms for each recommendation task can be determined as the total prediction loss. The regularization terms for each recommendation task can be determined based on the corresponding uncertainty index. Accordingly, the total prediction loss can be expressed as the following formula (7):

[0083]

[0084] in, This represents the total predicted loss. This represents the regularization term for the k-th recommendation task.

[0085] In another implementation, considering that the recommendation model includes N parallel expert networks, to ensure that the N parallel expert networks cooperate to assist in the prediction of each recommendation task, and to avoid a particular recommendation task selectively choosing the output of one or more expert networks (for example, for a certain recommendation task, when making recommendation predictions, it always relies on the output of one or more expert networks (with the corresponding prediction participation weight always being large)), the collaborative loss between the recommendation tasks is added when determining the total prediction loss to balance the load of each expert network (i.e., the proportion of each expert network participating in the recommendation prediction of each recommendation task). Step 113 may include: constructing the collaborative loss based on the gating output results of each gating network; and determining the total prediction loss based on the adjusted prediction loss and collaborative loss corresponding to each recommendation task.

[0086] For each recommendation task, the output of the corresponding gating network includes the prediction participation weights of each of the n expert networks participating in the prediction of the corresponding recommendation task. In step 113, for each recommendation task, the task loss corresponding to the recommendation task can be determined by the sum of the products of the prediction participation weights of each of the two adjacent expert networks corresponding to the recommendation task, and then the collaborative loss can be determined by the task loss corresponding to each recommendation task. The collaborative loss can be expressed by the following formula (8):

[0087]

[0088] in, This represents the prediction participation weight of the nth expert network corresponding to the kth recommendation task. The prediction participation weight of the m-th expert network corresponding to the k-th recommendation task.

[0089] Next, the total prediction loss is determined based on the adjusted prediction loss and collaborative loss corresponding to each recommendation task. This determination can be achieved by calculating the sum of the adjusted prediction loss and its regularization term for each recommendation task, then summing the product of the collaborative loss and its corresponding weight with this sum, and finally determining the total prediction loss based on the summed result. In one implementation, the summed result can be used as the total prediction loss, specifically expressed by the following formula (9):

[0090]

[0091] in, This represents the total predicted loss, and β represents the weight value corresponding to the collaborative loss. It can be a preset value and can be adjusted according to the actual situation.

[0092] In another implementation, the sum of the accumulated results and the regularization term corresponding to the recommendation model can be used to determine the total prediction loss. Specifically, this can be expressed by the following formula (10):

[0093]

[0094] in, This represents the total predicted loss, where λ is a preset value that can be adjusted according to actual conditions. ||Θ|| 2 This represents the regularization term corresponding to the recommendation model, which is determined based on the set of model parameters of the recommendation model.

[0095] After determining the total prediction loss through the aforementioned implementation methods, in step 12, the recommendation model is trained with the goal of minimizing the total prediction loss. In this step, based on the total prediction loss, the gradient of the recommendation model's parameters is determined using the backpropagation algorithm. Using the determined gradient of the model parameters and the current values ​​of each model parameter, the updated values ​​of the recommendation model's parameters are determined. Then, based on the updated values ​​of the model parameters, the values ​​of the recommendation model's parameters are adjusted. The determination of the gradient of the recommendation model's parameters is achieved with the goal of minimizing the total prediction loss.

[0096] In this embodiment, the high-order connectivity information between users and objects can be determined through the first graph neural network and the bipartite graph. Based on this, a set of expert networks is introduced to obtain expert output representations containing important information of different dimensions. Then, recommendations for several recommendation tasks are achieved in a collaborative manner, which improves the accuracy of the recommendation results of each recommendation task to a certain extent.

[0097] To improve the accuracy of recommendation results from recommendation models, it is necessary not only to determine the higher-order connectivity information between users and objects through first-order graph neural networks and bipartite graphs, but also to construct (derive) user relationship graphs representing collaborative similarity between users and / or object relationship graphs representing system similarity between objects based on graph neural network bipartite graphs. This combination of higher-order connectivity information between users and objects, as well as collaborative similarity between users and / or collaborative similarity between objects, can further improve the accuracy of recommendation results from recommendation models.

[0098] Accordingly, in one embodiment, such as Figure 2A As shown, a flowchart illustrating the training method of the recommended model in another embodiment of this specification is illustrated. Figure 2B As shown, the recommendation model may include a first graph neural network, a second graph neural network, N parallel expert networks, and several gating networks and prediction networks corresponding to each recommendation task. The training process of this recommendation model is as follows: Figure 2A As shown, steps S210-S270 may be included:

[0099] In step S210, a bipartite graph and a tag set are obtained, wherein each node in the bipartite graph corresponds to a user or object, and the first edge represents the association between the user and the object; the tag set includes several tag data between the target user and the target object connected by the first edge, corresponding to several recommendation tasks.

[0100] In step S220, a user relationship graph constructed based on the bipartite graph is obtained, wherein each node corresponds to each user involved in the bipartite graph, and the second edge indicates that the users corresponding to the two connected nodes satisfy the first condition.

[0101] In step S230, the first graph neural network is used to process the bipartite graph to obtain the first user representation of the target user and the first object representation of the target object.

[0102] In step S240, the second graph neural network is used to process the user relationship graph to obtain the second user representation of the target user.

[0103] In step S250, the user features of the target user and the object features of the target object are processed in parallel using each expert network to obtain the output representation of each expert.

[0104] In step S260, the input of each prediction network is determined by a combination operation, so that each prediction network generates prediction data for the corresponding recommendation task; the combination operation includes combining the expert output representation, the first user representation, the first object representation and the second user representation based on the output of the corresponding gating network.

[0105] In step S270, the recommendation model is trained using the prediction data and label data corresponding to each recommendation task.

[0106] Understandable. Figure 2A The process shown is an exemplary flowchart. In one embodiment, step S230 may be executed before step S220, or may be executed in parallel with step S240; step S250 may also be executed before step S220, step S230, or step S240, or may be executed in parallel with step S220, step S230, or step S240, etc.

[0107] In this embodiment, after obtaining the bipartite graph, a user relationship graph constructed based on the bipartite graph can also be obtained. The user relationship graph can be obtained by reading it from a preset storage location or by constructing it based on the bipartite graph.

[0108] This user relationship diagram Including multiple nodes and multiple edges (referred to as second edges for clarity), each node in this user relationship graph corresponds to a user involved in the bipartite graph, and the second edge indicates that the users corresponding to the two connected nodes satisfy the first condition.

[0109] In one implementation, the first condition includes: the first number of identical objects related to two users exceeds a first threshold. Understandably, the first number of identical objects related to two users exceeding the first threshold indicates a high degree of user collaborative similarity between the two users (e.g., user A and user B), suggesting a high probability that objects related to user A are also related to user B. This user collaborative similarity can characterize the similarity of preferences between users.

[0110] For example, when the association represents clicks or purchases, the first condition could include the number of times two users have clicked and purchased the same objects exceeding a first threshold. This first threshold can be set as needed. Wherein, if the sum of the number of times two users have clicked and purchased the same objects exceeds the first threshold, it can be considered that the two users have a high degree of similarity in their shopping preferences.

[0111] The first and second graph neural networks in the recommendation model are set up in parallel, such as... Figure 2B As shown. Additionally, as... Figure 2B As shown, a transformation layer is included before the first and second graph neural networks. This transformation layer can transform the user's initial attribute features x. u and the initial attribute characteristics x of the object oThe transformation layer maps the data to low-dimensional dense features (vectors), thereby enhancing the expressive power of the features and facilitating subsequent processing by the first graph neural network, the second graph neural network, n expert networks, and several gated networks. In this embodiment, this transformation layer not only reduces the computational cost of data during training but also avoids over-parameterization of the recommendation model, limiting the first graph neural network and the second graph neural network to share the user's initial representation (i.e., user features).

[0112] Specifically, after obtaining the bipartite graph, label set, and user relationship graph, a first-graph neural network can be used to process the bipartite graph. We obtain the first user representation of the target user and the first object representation of the target object, and then use a second graph neural network to process the user relationship graph. Obtain the second user representation of the target user Among them, the second graph neural network is used to process the user relationship graph. For the process, please refer to the process of using the first graph neural network to process the bipartite graph, which will not be repeated here.

[0113] By utilizing various expert networks, user features of the target user and object features of the target object are processed in parallel to obtain the output representations of each expert. Then, a combination operation is used to determine the input (data) of each prediction network. The input (data) of each prediction network is then fed into each prediction network to obtain its output, which is the prediction data for each recommendation task. This combination operation includes: combining the expert output representations, the first user representation, the first object representation, and the second user representation based on the output of the corresponding gating network.

[0114] The process of the combined operation in this embodiment is described below.

[0115] In one embodiment, the combined operation may include steps 21-23: In step 21, based on the first user representation, the first object representation, and the second user representation, the auxiliary representation corresponding to each expert network is determined. In one implementation, step 21 may include steps 211-213. Specifically, in step 211, the first user representation may be used as the basis for determining the auxiliary representation. First object representation Second user representation Determine each user object representation group, wherein the first user representation and the first object representation can be considered as a user object representation group. The second user representation and the first object representation can be considered as a set of user object representations. Next, in step 212, the interaction function κ(·,·) is used to process each user object representation group to obtain each interaction representation, where the interaction representation can be obtained. and interactive representation In one implementation, the interaction function can be set as an element-wise multiplication operation, an element-wise addition operation, a concatenation operation, or a multilayer perceptron (whose parameters are trainable).

[0116] Next, in step 213, the associated representations for each expert network are determined using the interaction representations. In one implementation, the interaction representations can be... and interactive representation These serve as supplementary representations corresponding to the output representations of each expert; for example, given N expert output representations, the first a expert output representations (expert network) correspond to the interaction representations. The interaction representations corresponding to the last b expert output representations (expert network) Where a + b = N. Alternatively, it could be: representing the interaction. and interactive representation Perform fusion operations (e.g., pooling, concatenation, element-wise multiplication, element-wise addition, etc.) to obtain a fused interactive representation, and use this fused interactive representation as an auxiliary representation for each expert network.

[0117] After determining the associated representations for each expert network using the above method, in step 22, each associated representation is fused with the corresponding expert output representation of the expert network to obtain the comprehensive expert representation of each expert network. This fusion can be performed in various ways, such as pooling or concatenation. Typically, the fusion operation here uses concatenation. That is, the associated representation corresponding to the nth expert output representation is concatenated to the nth expert output representation to obtain the nth concatenated representation. This serves as the expert comprehensive representation for the nth expert network. Then, in step 23, the expert comprehensive representations are combined based on the outputs of the gating networks corresponding to each prediction network. Specifically, the aforementioned formula (2) can be used to combine the expert comprehensive representations based on the outputs of the gating networks corresponding to each prediction network.

[0118] Understandably, the implementation process of step S210 can be found in [reference needed]. Figure 1A The implementation process of step S110 and step S230 can be found in [reference needed]. Figure 1A The implementation process of step S120 and step S250 can be found in [reference needed]. Figure 1A The implementation process of step S130 and step S270 can be found in [reference needed]. Figure 1A The implementation process of step S150 is not described in detail here.

[0119] In yet another embodiment, such as Figure 3A As shown, a flowchart illustrating the training method of the recommended model in another embodiment of this specification is illustrated. Figure 3BAs shown, the recommendation model may include a first graph neural network, a third graph neural network, N parallel expert networks, and several gating networks and prediction networks corresponding to each recommendation task. The training process of this recommendation model is as follows: Figure 3A As shown, steps S310-S370 may be included:

[0120] In step S310, a bipartite graph and a tag set are obtained, wherein each node in the bipartite graph corresponds to a user or object, and the first edge represents the association between the user and the object; the tag set includes several tag data between the target user and the target object connected by the first edge, corresponding to several recommendation tasks.

[0121] In step S320, an object relationship graph constructed based on the bipartite graph is obtained, wherein each node corresponds to each object involved in the bipartite graph, and the third edge indicates that the objects corresponding to the two connected nodes satisfy the second condition.

[0122] In step S330, the first graph neural network is used to process the bipartite graph to obtain the first user representation of the target user and the first object representation of the target object.

[0123] In step S340, the object relationship graph is processed using the third graph neural network to obtain the second object representation of the target object.

[0124] In step S350, the user features of the target user and the object features of the target object are processed in parallel using each expert network to obtain the output representation of each expert.

[0125] In step S360, the input of each prediction network is determined by a combination operation, so that each prediction network generates prediction data for the corresponding recommendation task; the combination operation includes combining expert output representations, first user representations, first object representations and second object representations based on the outputs of the corresponding gating networks.

[0126] In step S370, the recommendation model is trained using the prediction data and label data corresponding to each recommendation task.

[0127] Understandable. Figure 3A The process shown is an exemplary flowchart. In one embodiment, step S330 may be executed before step S320, or may be executed in parallel with step S340; step S350 may also be executed before step S320, step S330, or step S340, or may be executed in parallel with step S320, step S330, or step S340, etc.

[0128] In this embodiment, after obtaining the bipartite graph, an object relationship graph constructed based on the bipartite graph can also be obtained. The object relationship diagram can be obtained by reading it from a preset storage location or by constructing it based on the bipartite graph.

[0129] This object relationship diagram It includes multiple nodes and multiple edges (referred to as third edges for clarity). Each node in this object relationship graph corresponds to an object involved in the bipartite graph, and the third edge indicates that the objects corresponding to two nodes satisfy the second condition.

[0130] In one implementation, the second condition includes: the second number of identical users with a relationship between two objects exceeds a second threshold. It can be understood that the second number of identical users with a relationship between two objects exceeding the second threshold indicates a high degree of object collaborative similarity between the two objects (e.g., object C and object D). It can be considered that a user with a relationship with object C is more likely (probably) to also have a relationship with object D. Here, object collaborative similarity can characterize whether the two objects need to be used together, whether the two objects belong to the same brand, and / or whether the two objects have similar attributes (or similar content, such as: needing to use products together, belonging to the same brand, and / or having similar attributes), etc.

[0131] For example, the association represents clicks or purchases, and the second condition could include the number of times the same users have clicked and purchased from two objects exceeding a second threshold (i.e., two objects have been clicked by a first number of the same users and purchased from a second number of the same users, and the number of these same users (the sum of the first and second numbers) exceeds the second threshold). This second threshold can be set as needed.

[0132] The first and third graph neural networks in the recommendation model are set up in parallel, such as... Figure 3B As shown. Additionally, as... Figure 3B As shown, a transformation layer is included before the first and third graph neural networks. This transformation layer can transform the user's initial attribute features x. u and the initial attribute characteristics x of the object o The transformation layer maps the data to low-dimensional dense features (vectors), thereby enhancing the expressive power of the features and facilitating subsequent processing by the first graph neural network, the third graph neural network, n expert networks, and several gated networks. In this embodiment, this transformation layer not only reduces the computational cost of data during training but also avoids over-parameterization of the recommendation model, limiting the initial representation (i.e., object features) of the shared objects between the first graph neural network and the third graph neural network.

[0133] Specifically, after obtaining the bipartite graph, label set, and object relationship graph, a first-graph neural network can be used to process the bipartite graph. We obtain the first user representation of the target user and the first object representation of the target object, and then use a third-graph neural network to process the object relationship graph. Obtain the second object representation of the target user Among them, the third graph neural network is used to process object relationship graphs. For the process, please refer to the process of using the first graph neural network to process the bipartite graph, which will not be repeated here.

[0134] By utilizing various expert networks, user features of the target user and object features of the target object are processed in parallel to obtain the output representations of each expert. Then, a combination operation is used to determine the inputs of each prediction network. The inputs (data) of each prediction network are then fed into each prediction network to obtain their outputs, which are the prediction data for each recommendation task. This combination operation includes: combining the expert output representations, the first user representation, the first object representation, and the second object representation based on the outputs of the corresponding gating networks.

[0135] In this embodiment, the specific implementation process of the combination operation can be found in the previous embodiment. In this embodiment, the combination object is changed from combining expert output representations, the first user representation, the first object representation, and the second user representation based on the output of the corresponding gating network, to combining expert output representations, the first user representation, the first object representation, and the second object representation based on the output of the corresponding gating network.

[0136] Understandably, the implementation process of step S310 can be found in [reference needed]. Figure 1A The implementation process of step S110 and the implementation process of step S330 can be found in [reference needed]. Figure 1A The implementation process of step S120 and step S350 can be found in [reference needed]. Figure 1A The implementation process of step S130 and step S370 can be found in [reference needed]. Figure 1A The implementation process of step S150 is not described in detail here.

[0137] In yet another embodiment, such as Figure 4A As shown, a flowchart illustrating the training method of the recommended model in another embodiment of this specification is illustrated. Figure 4B As shown, the recommendation model may include a first graph neural network, a second graph neural network, a third graph neural network, N parallel expert networks, and several gating networks and prediction networks corresponding to each recommendation task. The training process of this recommendation model is as follows: Figure 4A As shown, steps S410-S470 may be included:

[0138] In step S410, a bipartite graph and a tag set are obtained, wherein each node in the bipartite graph corresponds to a user or object, and the first edge represents the association between the user and the object; the tag set includes several tag data corresponding to several recommendation tasks between the target user and the target object connected by the first edge.

[0139] In step S420, a user relationship graph constructed based on the bipartite graph is obtained, wherein each node corresponds to each user involved in the bipartite graph, and the second edge indicates that the users corresponding to the two connected nodes satisfy the first condition.

[0140] In step S430, an object relationship graph constructed based on the bipartite graph is obtained, wherein each node corresponds to each object involved in the bipartite graph, and the third edge indicates that the objects corresponding to the two connected nodes satisfy the second condition.

[0141] In step S440, the first graph neural network is used to process the bipartite graph to obtain the first user representation of the target user and the first object representation of the target object.

[0142] In step S450, the user relationship graph is processed using the second graph neural network to obtain the second user representation of the target user.

[0143] In step S460, the object relationship graph is processed using the third graph neural network to obtain the second object representation of the target object.

[0144] In step S470, the user features of the target user and the object features of the target object are processed in parallel using each expert network to obtain the output representation of each expert.

[0145] In step S480, the input of each prediction network is determined by a combination operation, so that each prediction network generates prediction data for the corresponding recommendation task; the combination operation includes combining expert output representations, first user representations, first object representations, second user representations and second object representations based on the output of the corresponding gating network.

[0146] In step S490, the recommendation model is trained using the prediction data and label data corresponding to each recommendation task.

[0147] In this embodiment, the first graph neural network, the second graph neural network, and the third graph neural network are configured in parallel, as follows: Figure 4B As shown. Additionally, as... Figure 4B As shown, a transformation layer is included before the first, second, and third graph neural networks. This transformation layer can transform the user's initial attribute features x. u and the initial attribute characteristics x of the object oThe transformation layer maps the data to low-dimensional dense features (vectors), thereby enhancing the expressive power of the features and facilitating subsequent processing by the first graph neural network, the second graph neural network, the third graph neural network, n expert networks, and several gated networks. In this embodiment, this transformation layer not only reduces the computational cost of data during training but also avoids over-parameterization of the recommendation model, limiting the first and second graph neural networks to share the initial representation of users (i.e., user features), and the first and third graph neural networks to share the initial representation of objects (i.e., object features).

[0148] In this embodiment, the combined operation may include steps 31-33: In step 31, based on the first user representation, the first object representation, the second user representation, and the second object representation, the corresponding auxiliary representation of each expert network is determined.

[0149] In one implementation, step 31 may include steps 311-313. Specifically, in step 311, the process may begin based on the first user representation. First object representation Second User Characteristic Second object representation Determine each user object representation group, wherein the first user representation and the first object representation can be considered as a user object representation group. The first user representation and the second object representation can be considered as a set of user object representations. The second user representation and the first object representation can be considered as a set of user object representations. The second user representation and the second object representation can be considered as a set of user object representations.

[0150] Next, in step 312, the interaction function κ(·,·) is used to process each user object representation group to obtain each interaction representation, wherein the interaction representation can be obtained. Interactive representation Interactive representation and interactive representation In one implementation, the interaction function can be set as an element-wise multiplication operation, an element-wise addition operation, a concatenation operation, or a multilayer perceptron (whose parameters are trainable).

[0151] That is, the interactive representation set can be obtained.

[0152] Next, in step 313, the auxiliary representations corresponding to each expert network are determined using the interaction representations. In one implementation, the auxiliary representation for each expert network is determined by an interaction representation; for example, for N expert networks, the interaction representations... The associated representations corresponding to the 1st to cth expert networks (expert output representations) are identified, and the interaction representations are... The associated representations corresponding to the (c+1)th to dth expert networks (expert output representations) are identified, and the interaction representations are... The associated representations are identified as those corresponding to the (d+1)th to the eth expert networks (expert output representations); the interactive representations are... The associated representations are determined as those corresponding to the (e+1)th to Nth expert networks (expert output representations). Where 1 ≤ c ≤ d ≤ e ≤ N. In one case, when the number of expert networks N is 4, the interactive representations can be... Interactive representation Interactive representation and interactive representation These serve as auxiliary representations corresponding to the expert output representations of the four expert networks.

[0153] In another implementation, the subordinate representations corresponding to each expert network are determined by at least two interaction representations. Accordingly, the aforementioned four interaction representations can be combined according to a specified relationship to obtain interaction representation groups corresponding to each expert network (expert network output). Then, for each expert network's corresponding interaction representation group, a fusion operation (e.g., pooling, concatenation, element-wise multiplication, element-wise addition, etc.) is performed on the interaction representations within that group to obtain the subordinate representations corresponding to each expert network. The subordinate representations corresponding to each expert network can be the same or different. For example, in one case, the four interaction representations can be fused, and the resulting representations can be used as the subordinate representations corresponding to each expert network.

[0154] After determining the associated representations for each expert network using the above method, in step 32, each associated representation is fused with the corresponding expert output representation of the expert network to obtain the comprehensive expert representation of each expert network. This fusion can be performed in various ways, such as pooling or concatenation. Typically, the fusion operation here uses concatenation. That is, the associated representation corresponding to the nth expert output representation is concatenated to the nth expert output representation to obtain the nth concatenated representation. This serves as the expert comprehensive representation for the nth expert network. Then, in step 33, the expert comprehensive representations are combined based on the outputs of the gating networks corresponding to each prediction network. Specifically, the aforementioned formula (2) can be used to combine the expert comprehensive representations based on the outputs of the gating networks corresponding to each prediction network.

[0155] Understandably, the implementation process of step S410 can be found in [reference needed]. Figure 1A The implementation process of step S110 and step S420 can be found in [reference needed]. Figure 2A The implementation process of step S220 and step S430 can be found in [reference needed]. Figure 3AThe implementation process of step S320 and step S440 can be found in [reference needed]. Figure 1A The implementation process of step S120 and step S450 can be found in [reference needed]. Figure 2A The implementation process of step S240 and step S460 can be found in [reference needed]. Figure 3A The implementation process of step S340 and step S470 can be found in [reference needed]. Figure 1A The implementation process of step S130 and step S490 can be found in [reference needed]. Figure 1A The implementation process of step S150 is not described in detail here.

[0156] This embodiment provides a multi-channel graph neural network that utilizes a bipartite graph and its derived user relationship graph and object relationship graph to jointly model high-order interaction information between user objects. Specifically, the bipartite graph determines a first user representation and a first object representation that integrate high-order connectivity information between user objects; the user relationship graph determines a second user representation that integrates collaborative information between users with high similarity; and the object relationship graph determines a second object representation that integrates collaborative information between objects with high similarity. The high-order interaction information (i.e., deep contextual relationships) between user objects is jointly constructed using the first user representation, the first object representation, the second user representation, and the second object representation, which can help alleviate the data sparsity problem in the platform to some extent. Based on this, a set of expert networks is introduced, and they collaboratively complete several recommendation tasks, which can improve the accuracy of the recommendation results for each task. A Mixture of Graph Enhanced Expert Networks (MOGE) is designed for recommendation of several tasks.

[0157] The foregoing description describes specific embodiments of this specification; other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than those shown in the embodiments, and the desired result may still be achieved. Furthermore, the processes depicted in the drawings do not necessarily need to follow the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0158] Corresponding to the above method embodiments, this specification provides a training device 500 for a recommendation model, the schematic block diagram of which is shown below. Figure 5 As shown, the recommendation model includes a first graph neural network, N parallel expert networks, and several gating networks and prediction networks corresponding to each recommendation task. The device includes:

[0159] The first acquisition module 510 is configured to acquire a bipartite graph and a tag set, wherein each node in the bipartite graph corresponds to a user or object, and the first edge represents the association relationship between the user and the object; the tag set includes several tag data corresponding to the several recommendation tasks between the target user and the target object connected by the first edge.

[0160] The first processing module 520 is configured to use a first graph neural network to process the bipartite graph to obtain a first user representation of the target user and a first object representation of the target object.

[0161] The second processing module 530 is configured to utilize each expert network to process the user features of the target user and the object features of the target object in parallel, and obtain the output representations of each expert.

[0162] The first determining module 540 is configured to determine the input of each prediction network through a combination operation, so that each prediction network generates prediction data for the corresponding recommendation task; the combination operation includes combining the expert output representation, the first user representation, and the first object representation based on the output of the corresponding gating network.

[0163] The training module 550 is configured to train the recommendation model using the prediction data and label data corresponding to each recommendation task.

[0164] In one alternative implementation, the recommendation model further includes a second graph neural network, and the apparatus further includes:

[0165] The second acquisition module (not shown in the figure) is configured to acquire a user relationship graph constructed based on the bipartite graph, wherein each node corresponds to each user involved in the bipartite graph, and the second edge indicates that the users corresponding to the two connected nodes satisfy the first condition.

[0166] The third processing module (not shown in the figure) is configured to use the second graph neural network to process the user relationship graph and obtain the second user representation of the target user.

[0167] The first determining module 540 is specifically configured as follows:

[0168] Based on the output of the corresponding gating network, the input of each prediction network is determined by combining the expert output representation, the first user representation, the first object representation, and the second user representation.

[0169] In one alternative implementation, the recommendation model further includes a third graph neural network, and the apparatus further includes:

[0170] The third acquisition module (not shown in the figure) is configured to acquire an object relationship graph constructed based on the bipartite graph, wherein each node corresponds to each object involved in the bipartite graph, and the third edge indicates that the objects corresponding to the two connected nodes satisfy the second condition.

[0171] The fourth processing module (not shown in the figure) is configured to use the third graph neural network to process the object relationship graph and obtain the second object representation of the target object;

[0172] The first determining module 540 is specifically configured as follows:

[0173] Based on the output of the corresponding gating network, the input of each prediction network is determined by combining the expert output representation, the first user representation, the first object representation, the second user representation, and the second object representation.

[0174] In one alternative implementation, the first condition includes: the first number of identical objects that are associated between two users exceeds a first threshold; the second condition includes: the second number of identical users that are associated between two objects exceeds a second threshold.

[0175] In one alternative implementation, the first determining module 540 includes:

[0176] The first determining unit (not shown in the figure) is configured to determine the auxiliary representations corresponding to each expert network based on the first user representation, the first object representation, the second user representation, and the second object representation.

[0177] The fusion unit (not shown in the figure) is configured to fuse the auxiliary representation with the corresponding expert output representation to obtain the corresponding expert comprehensive representation;

[0178] The combined unit (not shown in the figure) is configured to combine the comprehensive representations of various experts based on the output of the corresponding gating network.

[0179] In one optional implementation, the first determining unit is specifically configured to determine each user object representation group based on the first user representation, the first object representation, the second user representation, and the second object representation.

[0180] Using interactive functions, process each user object representation group to obtain each interactive representation;

[0181] Based on each interaction representation, the corresponding auxiliary representations for each expert network are determined.

[0182] In one alternative implementation, the fusion unit is specifically configured to splice together the expert output representations and their corresponding auxiliary representations.

[0183] In one alternative implementation, it further includes:

[0184] The fifth processing module (not shown in the figure) is configured to process the user features of the target user and the object features of the target object in parallel using each gating network before determining the input of each prediction network through the combination operation, so as to obtain the output of each gating network.

[0185] In one optional implementation, the second processing module 530 is specifically configured to concatenate the user features and the object features to obtain concatenated features;

[0186] The splicing features are input into each expert network to obtain the expert output representation of each expert network.

[0187] In one alternative implementation, the recommendation model further includes a transformation layer; and further includes:

[0188] The fourth acquisition module (not shown in the figure) is configured to acquire the initial attribute features of the target user and the initial attribute features of the target object before the user features of the target user and the object features of the target object are processed in parallel using each expert network;

[0189] The sixth processing module (not shown in the figure) is configured to process the initial attribute features of the target user using the user feature transformation matrix of the transformation layer to obtain the user features of the target user;

[0190] The seventh processing module (not shown in the figure) is configured to process the initial attribute features of the target object using the object feature transformation matrix of the transformation layer to obtain the object features of the target object.

[0191] In one alternative implementation, the object is a merchant or an advertisement, and the plurality of recommendation tasks include clicks and conversions; or, the object is a product, and the plurality of recommendation tasks include clicks and purchases.

[0192] In one alternative implementation, the training module 550 includes:

[0193] The second determining unit (not shown in the figure) is configured to determine the total prediction loss using the prediction data and label data corresponding to each recommendation task.

[0194] The training unit (not shown in the figure) is configured to train the model parameters of the recommendation model with the objective of minimizing the total prediction loss.

[0195] In one optional embodiment, the second determining unit includes:

[0196] The first determining submodule (not shown in the figure) is configured to use the prediction data and label data corresponding to any recommendation task to determine the initial prediction loss corresponding to the recommendation task.

[0197] The second determining submodule (not shown in the figure) is configured to determine the adjusted prediction loss for each recommendation task based on the uncertainty index corresponding to each recommendation task and the initial prediction loss corresponding to each recommendation task, wherein the uncertainty index is a trainable parameter.

[0198] The third determining submodule (not shown in the figure) is configured to adjust the prediction loss based on each recommendation task to determine the total prediction loss.

[0199] In one optional implementation, the third determining submodule is specifically configured to construct a collaborative loss based on the gating output results of each gating network; and determine the total prediction loss based on the adjusted prediction loss corresponding to each recommendation task and the collaborative loss.

[0200] The above-described apparatus embodiments correspond to the method embodiments, and detailed descriptions can be found in the description of the method embodiments section, which will not be repeated here. The apparatus embodiments are derived based on the corresponding method embodiments and have the same technical effects as the corresponding method embodiments; detailed descriptions can be found in the corresponding method embodiments.

[0201] This specification also provides a computer-readable storage medium storing a computer program that, when executed in a computer, causes the computer to perform the training method for the recommended model provided in this specification.

[0202] This specification also provides a computing device, including a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, it implements the training method of the recommended model provided in this specification.

[0203] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the embodiments for storage media and computing devices are basically similar to the method embodiments, so they are described more simply; relevant parts can be referred to the descriptions of the method embodiments.

[0204] Those skilled in the art will recognize that the functions described in the embodiments of the present invention in one or more of the above examples can be implemented using hardware, software, firmware, or any combination thereof. When implemented in software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or code on a computer-readable medium.

[0205] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the embodiments of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made based on the technical solutions of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for training a recommendation model, the recommendation model comprising a first graph neural network, N parallel expert networks, and several gating networks and prediction networks corresponding to each recommendation task, the method comprising: Obtain a bipartite graph and a tag set, wherein each node in the bipartite graph corresponds to a user or object, and the first edge represents the association between the user and the object; the tag set includes several tag data corresponding to the several recommendation tasks between the target user and the target object connected by the first edge. Using a first graph neural network, the bipartite graph is processed to obtain a first user representation of the target user and a first object representation of the target object; By utilizing various expert networks, the user characteristics of the target user and the object characteristics of the target object are processed in parallel to obtain the output representations of each expert. The inputs to each prediction network are determined by a combination operation, so that each prediction network generates prediction data for the corresponding recommendation task; the combination operation includes combining the expert output representations and the first user representation and the first object representation obtained by the first graph neural network based on the output of the corresponding gating network. Using the prediction data and label data corresponding to each recommendation task, determine the initial prediction loss for each recommendation task. Based on the uncertainty index corresponding to each recommendation task and the initial prediction loss corresponding to each recommendation task, the adjusted prediction loss corresponding to each recommendation task is determined. The uncertainty index is a trainable parameter, and the uncertainty index corresponding to each recommendation task represents the uncertainty of the prediction data predicted by the recommendation task. The total prediction loss is determined based on the adjusted prediction loss corresponding to each recommendation task. The model parameters of the recommendation model are trained with the goal of minimizing the total prediction loss.

2. The method of claim 1, wherein the recommendation model further comprises a second graph neural network, and the method further comprises: Obtain a user relationship graph constructed based on the bipartite graph, wherein each node corresponds to each user involved in the bipartite graph, and the second edge indicates that the users corresponding to the two connected nodes satisfy the first condition; The second graph neural network is used to process the user relationship graph to obtain a second user representation of the target user. The combined operation includes: Based on the output of the corresponding gating network, combine the expert output representation, the first user representation, the first object representation, and the second user representation.

3. The method of claim 2, wherein the recommendation model further includes a third graph neural network, and the method further includes: Obtain the object relationship graph constructed based on the bipartite graph, wherein each node corresponds to each object involved in the bipartite graph, and the third edge indicates that the objects corresponding to the two connected nodes satisfy the second condition; Using a third-graph neural network, the object relationship graph is processed to obtain a second object representation of the target object; The combined operation includes: Based on the output of the corresponding gating network, combine the expert output representation, the first user representation, the first object representation, the second user representation, and the second object representation.

4. The method of claim 3, wherein, The first condition includes: the first number of identical objects that are associated between two users exceeds a first threshold; the second condition includes: the second number of identical users that are associated between two objects exceeds a second threshold.

5. The method of claim 3, wherein, The combined operation includes: Based on the first user representation, the first object representation, the second user representation, and the second object representation, the corresponding auxiliary representations of each expert network are determined. The auxiliary representation is fused with the corresponding expert output representation to obtain the corresponding expert comprehensive representation; Based on the output of the corresponding gating network, the comprehensive representations of various experts are combined.

6. The method of claim 5, wherein, The determination of the associated representations for each expert network includes: Based on the first user representation, the first object representation, the second user representation, and the second object representation, each user object representation group is determined. Using interactive functions, process each user object representation group to obtain each interactive representation; Based on each interaction representation, the corresponding auxiliary representations for each expert network are determined.

7. The method of claim 5, wherein, The process of fusing the auxiliary representation with the corresponding expert output representation includes: The output representations of each expert and their corresponding auxiliary representations are spliced ​​together.

8. The method of claim 1, further comprising, before determining the inputs of each prediction network through the combination operation: By utilizing each gating network, the user characteristics of the target user and the object characteristics of the target object are processed in parallel to obtain the output of each gating network.

9. The method of claim 1, wherein, The method of utilizing various expert networks to process the user features of the target user and the object features of the target object in parallel, and obtaining the output representations of each expert, includes: By concatenating the user features and the object features, a concatenated feature is obtained; The splicing features are input into each expert network to obtain the expert output representation of each expert network.

10. The method of claim 1, wherein the recommendation model further comprises a transformation layer; and prior to processing the user features of the target user and the object features of the target object in parallel using each expert network, the method further comprises: Obtain the initial attribute characteristics of the target user and the initial attribute characteristics of the target object; The initial attribute features of the target user are processed using the user feature transformation matrix of the transformation layer to obtain the user features of the target user; The initial attribute features of the target object are processed using the object feature transformation matrix of the transformation layer to obtain the object features of the target object.

11. The method of claim 1, wherein, The object is a merchant or advertisement, and the several recommendation tasks include clicks and conversions; or... The object is a product, and the recommendation tasks include clicking and purchasing.

12. The method of claim 1, wherein, The step of adjusting the prediction loss based on each recommendation task to determine the total prediction loss includes: Based on the gating outputs of each gating network, construct the collaborative loss; The total prediction loss is determined based on the adjusted prediction loss for each recommended task and the collaborative loss.

13. A training apparatus for a recommendation model, the recommendation model comprising a first graph neural network, N parallel expert networks, and several gating networks and prediction networks corresponding to each recommendation task, the apparatus comprising: The first acquisition module is configured to acquire a bipartite graph and a tag set, wherein each node in the bipartite graph corresponds to a user or object, and the first edge represents the association relationship between the user and the object; the tag set includes several tag data corresponding to the several recommendation tasks between the target user and the target object connected by the first edge. The first processing module is configured to use a first graph neural network to process the bipartite graph to obtain a first user representation of the target user and a first object representation of the target object. The second processing module is configured to utilize each expert network to process the user features of the target user and the object features of the target object in parallel, and obtain the output representations of each expert. The first determining module is configured to determine the input of each prediction network through a combination operation, so that each prediction network generates prediction data for the corresponding recommendation task; the combination operation includes combining the output representations of each expert and the first user representation and the first object representation obtained by the first graph neural network based on the output of the corresponding gating network. The training module is configured to: determine the initial prediction loss for each recommendation task using the prediction data and label data corresponding to each recommendation task; determine the adjusted prediction loss for each recommendation task based on the uncertainty index and the initial prediction loss, wherein the uncertainty index is a trainable parameter and represents the uncertainty of the prediction data for each recommendation task; determine the total prediction loss based on the adjusted prediction loss for each recommendation task; and train the model parameters of the recommendation model with the objective of minimizing the total prediction loss.

14. A computing device comprising a memory and a processor, wherein, The memory stores executable code, and when the processor executes the executable code, it implements the method of any one of claims 1-12.

Images