Recommendation system and method based on meta-learning graph neural network for cold start

Abstract

The application discloses a recommendation system and method based on meta-learning graph neural network for cold start, and belongs to the technical field of network recommendation. The technical problem to be solved by the application is how to overcome the defect that the cold start user interaction data is sparse and cannot accurately dynamically model user preferences. The technical solution adopted is that the system comprises a two-part graph construction layer, a graph embedding layer, a sequence coding layer and a meta-learning layer. The method is as follows: obtaining interaction data in a data set, constructing a user-item two-part graph using the interaction data; using a graph convolution network based on the two-part graph to construct high-order relationships between items in sequences, and generating accurate embedding representations of users and items; using the sequence coding layer to learn the transfer information of items within the sequence, and combining an attention mechanism to generate a dynamic interest representation for the user; and using the meta-learning layer to quickly adapt to the cold start recommendation task.

Description

Technical Field

[0001] This invention relates to the field of network recommendation technology, specifically a cold-start-oriented recommendation system and method based on meta-learning graph neural networks. Background Technology

[0002] As deep learning continues to develop in the recommendation field, the performance of recommendation algorithms is also gradually improving. When new users join, due to a lack of sufficient interaction information, recommendation methods cannot accurately model user preferences. Researchers leverage the advantages of meta-learning in few-shot learning, treating cold-start user recommendation as a new task. Through gradient descent, they enable the model to quickly adapt to the cold-start task and provide recommendations that match user preferences.

[0003] Currently, meta-learning has become a mainstream approach to address the cold-start problem in recommendation algorithms. For example, MeLU uses demographic and item information to predict cold-start user preferences, achieving excellent performance. However, its performance significantly degrades in scenarios where auxiliary information is unavailable. Later, researchers shifted their focus to methods that do not use auxiliary information. MetaTL, for instance, uses only sequences of limited interactions to dynamically model cold-start user preferences. However, these methods ignore the correlations between items in different sequences. The correlations between items in sequences can capture general user preferences and alleviate the data sparsity problem.

[0004] Therefore, overcoming the shortcomings of the sparse user interaction data during cold starts and failing to accurately generate user item embedding representations, and dynamically modeling user preferences, is a technical problem that urgently needs to be solved. Summary of the Invention

[0005] The technical objective of this invention is to provide a recommendation system and method based on meta-learning graph neural networks for cold starts, in order to address the problem that sparse user interaction data during cold starts makes it impossible to accurately and dynamically model user preferences.

[0006] The technical objective of this invention is achieved as follows: a cold-start-oriented recommendation system based on meta-learning graph neural networks, comprising,

[0007] The bipartite graph construction layer is used to obtain interaction data from the dataset and use the interaction data to construct a user-item bipartite graph.

[0008] The graph embedding layer is used to model the high-order relationships between items in a sequence based on the user-item bipartite graph through a graph convolutional network, and generate accurate user and item embedding representations.

[0009] Sequence encoding layer, used to model the dynamic changes of items within a sequence and user preferences through attention network;

[0010] The meta-learning layer is used to quickly adapt to cold-start recommendation tasks.

[0011] As a preferred embodiment, the user-item bipartite graph includes two types of nodes: user node u and item node v.

[0012] If a user interacts with an item, an edge exists between the user and the item; a path connecting multiple item nodes and user nodes captures the correlation between item nodes located at the beginning and end of the path.

[0013] More preferably, the embedded layer is as follows:

[0014] The formula for aggregating the neighbor information of the target node is as follows:

[0015]

[0016]

[0017] in, This means that the item node v at level k is obtained by applying an aggregation function to all its neighbors at level k-1. i neighborhood representation and Let the weight matrix and bias vector of the k-th layer be respectively, and This means that the item node u at level k is obtained by applying an aggregation function to all its neighbors at level k-1. i neighborhood representation and Let the weight matrix and bias vector of the k-th layer be respectively, and The corresponding k-th hidden representation is User node u i The hidden representation of the k-th layer is User-Item Two-Part Diagram and Target v i Adjacent nodes are defined as 1-hop neighbors of the target node; n(v i ) represents node v i The set of 1-hop neighbors; the initial embedding of item node v is represented as x. v The initial embedding representation of user node u is x. u In graph convolutional networks (GCNs), node embeddings depend on both the node's own information and the information of its surrounding neighboring nodes.

[0018] The neighborhood information is combined with the target node representation to form the user and item embedding representation for this layer, as shown in the following formula:

[0019]

[0020]

[0021] in, and Let the transformation weight matrix and bias vector of the k-th layer be... γ represents the pooling function; σ represents the activation function; [;] represents concatenation;

[0022] Since embeddings at different layers have different semantics, in order to make the embedding representations applied to cold start scenarios more comprehensive and richer, the embedding representations at different layers are weighted and combined to obtain the final embedding representation. The formula is as follows:

[0023]

[0024]

[0025] Among them, a k Let a represent the weight matrix of the k-th layer, used to indicate the importance of embeddings from other layers in the embedding composition of the k-th layer. k These are either manually adjusted parameters or automatically optimized model parameters; to reduce training complexity, a... k Set to 1 / (K+1) uniformly; Embedded representations for users and items.

[0026] More preferably, the sequence coding layer is specifically as follows:

[0027] After the graph embedding layer, the user u's interaction sequence is represented as follows:

[0028] With user sequence s u As input, each item v q Generate an interactive item set based on the interaction order of the items that precede it. In each interaction sequence, the first item, having no prior interactions, has its set of interactive item pairs defined as follows:

[0029] Each item pair obtains the mapping relationship between the two items through an L-layer neural network, as follows:

[0030]

[0031] r l =ReLU(W l r l-1 +b l );

[0032]

[0033] in, This indicates that vectors m and n are connected; v i With v q Represent the head entity and tail entity respectively; L is the number of neural network layers, l∈{1,…,L-1}; W l and b l This represents the weight matrix and bias vector of the l-th layer. Represents the final item v i With item v q The transfer information; v represents the item; q represents the tail entity; r represents the concatenated item vector; the superscript of r indicates the layer number;

[0034] Since each item pair contains multiple item pairs, when generating an item transfer representation for each item, all transfer information is aggregated, as shown in the following formula:

[0035]

[0036] After obtaining the transformation information of each item within the sequence, the correlation information between items in the sequence is combined with the transition information between items within the sequence to generate the final user interest representation;

[0037] To capture each user's evolving interests, an attention network is used to generate a weight for the latent state of each input. This weight represents the impact of the user's current behavior on the final preference. Since the last item in a user's interaction sequence often plays a crucial role in the next interaction, the last interaction item v is considered when generating the attention weights. n The effect is shown in the following formula:

[0038]

[0039]

[0040] in, Represents item v q The weights in user u's final decision; W2, W3, W4, b2, b2 are the parameters of the multilayer sensing network. express The influence weight of item vq;

[0041] User interests, considering the order correlation of items within a sequence and time dynamics, are represented as follows:

[0042]

[0043] After obtaining the representation of the user's current interests, the classic matrix factorization method is used to infer the user's preferences for items;

[0044] User u on item vp The predicted score is the user interest I u and item embedding The inner product of these two factors, the probability that a user will interact with an item, is defined as the predicted score:

[0045]

[0046] The parameters are learned using Bayesian Personalized Ranking (BPR) loss, which estimates the probability of observed interactions as higher than that of unobserved items. The BPR loss function is defined as follows:

[0047]

[0048] Among them, T i T represents the training samples for the i-th task. i ={(u,v,v - )|(u,v)∈R + ,(u,v - )∈R -};R + A collection of items that interact with the user; R - θ represents the set of items that have no interaction history with the user; θ represents the learnable parameters; λ is the L2 regularization parameter used to alleviate the overfitting problem.

[0049] More preferably, the meta-learning layer is as follows:

[0050] Optimize all parameters θ of the graph embedding network using a gradient-based MAML method. g and all parameters θ of the sequence coding layer t ;

[0051] The training task is divided into an inner loop and an outer loop. The inner loop is used to optimize the user interest model, and the outer loop is used to update the parameters through the meta-learning layer.

[0052] Since user and item representations are globally shared, the item embedding representation will not be updated within the inner loop; the inner loop uses the support set D. s During training, the sequence encoding layer is updated using gradient descent, with the parameters updated to Θ'. t The formula is as follows:

[0053]

[0054] The outer loop optimization process uses query set D. q To optimize all parameters:

[0055]

[0056]

[0057] After the training phase of the meta-learning layer, good initial embeddings of users and items are obtained, along with the parameters required for the bipartite graph construction layer, graph embedding layer, sequence coding layer, and meta-learning layer.

[0058] During the testing phase of the meta-learning layer, a small amount of new user data was used to fine-tune the bipartite graph construction layer, graph embedding layer, sequence coding layer, and meta-learning layer, enabling rapid adaptation to recommendation tasks for cold-start users.

[0059] A cold-start-oriented recommendation method based on meta-learning graph neural networks is proposed, as follows:

[0060] Obtain interaction data from the dataset and use the interaction data to construct a user-item bipartite graph;

[0061] Based on bipartite graphs, graph convolutional networks are used to construct high-order relationships between items in sequences, generating accurate embedded representations of users and items;

[0062] By utilizing sequence encoding layers to learn the transition information of items within a sequence, and combining this with an attention mechanism, dynamic interest representations are generated for users.

[0063] The training data is divided into a support set and a query set using a meta-learning layer. The parameters of the sequence encoding layer are optimized using the support set in the inner loop, and all parameters are updated using the query set in the outer loop. In the cold start task, several gradient descent steps are used to obtain accurate recommendation content, thus enabling rapid adaptation to the cold start recommendation task.

[0064] As a preferred embodiment, the user-item bipartite graph includes two types of nodes: user node u and item node v.

[0065] If a user interacts with an item, an edge exists between the user and the item; a path connecting multiple item nodes and user nodes captures the correlation between item nodes located at the beginning and end of the path;

[0066] Based on bipartite graphs, graph convolutional networks are used to construct high-order relationships between items in sequences, generating accurate embedding representations of users and items, as detailed below:

[0067] The formula for aggregating the neighbor information of the target node is as follows:

[0068]

[0069]

[0070] in, This means that the item node v at level k is obtained by applying an aggregation function to all its neighbors at level k-1. i neighborhood representation and Let the weight matrix and bias vector of the k-th layer be respectively, and This means that the item node u at level k is obtained by applying an aggregation function to all its neighbors at level k-1. i neighborhood representation and Let the weight matrix and bias vector of the k-th layer be respectively, and The corresponding k-th hidden representation is User node u i The hidden representation of the k-th layer is User-Item Two-Part Diagram and Target v i Adjacent nodes are defined as 1-hop neighbors of the target node; n(v i ) represents node v i The set of 1-hop neighbors; the initial embedding of item node v is represented as x. v The initial embedding representation of user node u is x. u In graph convolutional networks (GCNs), node embeddings depend on both the node's own information and the information of its surrounding neighboring nodes.

[0071] The neighborhood information is combined with the target node representation to form the user and item embedding representation for this layer, as shown in the following formula:

[0072]

[0073]

[0074] in, and Let the transformation weight matrix and bias vector of the k-th layer be... γ represents the pooling function; σ represents the activation function; [;] represents concatenation;

[0075] Since embeddings at different layers have different semantics, in order to make the embedding representations applied to cold start scenarios more comprehensive and richer, the embedding representations at different layers are weighted and combined to obtain the final embedding representation. The formula is as follows:

[0076]

[0077]

[0078] Among them, a k Let a represent the weight matrix of the k-th layer, used to indicate the importance of embeddings from other layers in the embedding composition of the k-th layer. k These are either manually adjusted parameters or automatically optimized model parameters; to reduce training complexity, a... kSet to 1 / (K+1) uniformly; Embedded representations for users and items.

[0079] More optimally, the sequence encoding layer is used to learn the transition information of items within a sequence, and combined with an attention mechanism to generate dynamic interest representations for users, as follows:

[0080] After the graph embedding layer, the user u's interaction sequence is represented as follows:

[0081] With user sequence s u As input, each item v q Generate an interactive item set based on the interaction order of the items that precede it. In each interaction sequence, the first item, having no prior interactions, has its set of interactive item pairs defined as follows:

[0082] Each item pair obtains the mapping relationship between the two items through an L-layer neural network, as follows:

[0083]

[0084] r l =ReLU(W l r l-1 +b l );

[0085]

[0086] in, This indicates that vectors m and n are connected; v i With v q Represent the head entity and tail entity respectively; L is the number of neural network layers, l∈{1,…,L-1}; W l and b l This represents the weight matrix and bias vector of the l-th layer. Represents the final item v i With item v q The transfer information; v represents the item; q represents the tail entity; r represents the concatenated item vector; the superscript of r indicates the layer number;

[0087] Since each item pair contains multiple item pairs, when generating an item transfer representation for each item, all transfer information is aggregated, as shown in the following formula:

[0088]

[0089] After obtaining the transformation information of each item within the sequence, the correlation information between items in the sequence is combined with the transition information between items within the sequence to generate the final user interest representation;

[0090] To capture each user's evolving interests, an attention network is used to generate a weight for the latent state of each input. This weight represents the impact of the user's current behavior on the final preference. Since the last item in a user's interaction sequence often plays a crucial role in the next interaction, the last interaction item v is considered when generating the attention weights. n The effect is shown in the following formula:

[0091]

[0092]

[0093] in, Represents item v q The weights in user u's final decision; W2, W3, W4, b2, b2 are the parameters of the multilayer sensing network. express The influence weight of item vq;

[0094] User interests, considering the order correlation of items within a sequence and time dynamics, are represented as follows:

[0095]

[0096] After obtaining the representation of the user's current interests, the classic matrix factorization method is used to infer the user's preferences for items;

[0097] User u on item v p The predicted score is the user interest I u and item embedding The inner product of these two factors, the probability that a user will interact with an item, is defined as the predicted score:

[0098]

[0099] The parameters are learned using Bayesian Personalized Ranking (BPR) loss, which estimates the probability of observed interactions as higher than that of unobserved items. The BPR loss function is defined as follows:

[0100]

[0101] Among them, T i T represents the training samples for the i-th task. i ={(u,v,v - )|(u,v)∈R + ,(u,v -)∈R -};R + A collection of items that interact with the user; R - θ represents the set of items that have no interaction history with the user; θ represents the learnable parameters; λ is the L2 regularization parameter, used to alleviate the overfitting problem;

[0102] The training data is divided into a support set and a query set using a meta-learning layer. The parameters of the sequence encoding layer are optimized using the support set in the inner loop, and all parameters are updated using the query set in the outer loop. Accurate recommendation content is obtained by using several gradient descent steps in the cold start task, as detailed below:

[0103] Optimize all parameters θ of the graph embedding network using a gradient-based MAML method. g and all parameters θ of the sequence coding layer t ;

[0104] The training task is divided into an inner loop and an outer loop. The inner loop is used to optimize the user interest model, and the outer loop is used to update the parameters through the meta-learning layer.

[0105] Since user and item representations are globally shared, the item embedding representation will not be updated within the inner loop; the inner loop uses the support set D. s During training, the sequence encoding layer is updated using gradient descent, with the parameters updated to Θ'. t The formula is as follows:

[0106]

[0107] The outer loop optimization process uses query set D. q To optimize all parameters:

[0108]

[0109]

[0110] After the training phase of the meta-learning layer, good initial embeddings of users and items are obtained, along with the parameters required for the bipartite graph construction layer, graph embedding layer, sequence coding layer, and meta-learning layer.

[0111] During the testing phase of the meta-learning layer, a small amount of new user data was used to fine-tune the bipartite graph construction layer, graph embedding layer, sequence coding layer, and meta-learning layer, enabling rapid adaptation to recommendation tasks for cold-start users.

[0112] An electronic device includes: a memory and at least one processor;

[0113] The memory contains computer programs;

[0114] The at least one processor executes the computer program stored in the memory, causing the at least one processor to perform the cold-start-oriented recommendation method based on meta-learning graph neural networks as described above.

[0115] A computer-readable storage medium storing a computer program that can be executed by a processor to implement the cold-start-oriented recommendation method based on meta-learning graph neural networks as described above.

[0116] The cold-start-oriented recommendation system and method based on meta-learning graph neural networks of the present invention have the following advantages:

[0117] (i) Without using other auxiliary information, this invention generates interest representations for cold-start users by modeling the relationship between items in sequences and the transfer relationship between items within sequences, combined with an attention mechanism. The parameter optimization adopts a gradient descent method based on meta-learning, which significantly improves the recommendation performance in the cold-start scenario.

[0118] (ii) To address the problem that sparse user interaction data during cold start cannot accurately generate user-item embedding representations, this invention constructs user-item bipartite graphs from user interaction sequences and uses graph networks to model high-order relationships between items in the sequences to generate accurate embedding representations.

[0119] (iii) This invention captures the transfer relationships between items through a sequence coding layer, and models changes in user preferences in scenarios without using other auxiliary information;

[0120] (iv) For cold start scenarios, this invention uses a meta-learning layer for training. Through one or several gradient descents, it can accurately model the interests and preferences of cold start users and provide personalized recommended content.

[0121] (v) This invention utilizes both inter-sequence item relationships and intra-sequence item relationships to solve the cold start problem, improving the accuracy of recommended content and enabling rapid adaptation to cold start recommendation tasks. Attached Figure Description

[0122] The invention will be further described below with reference to the accompanying drawings.

[0123] Appendix Figure 1 This is a block diagram of a cold-start-oriented recommendation system based on meta-learning graph neural networks.

[0124] Appendix Figure 2 The flowchart shows a cold-start-oriented recommendation method based on meta-learning graph neural networks.

[0125] Appendix Figure 3 To model the high-order relationships between items in a sequence in a user-item bipartite graph using a graph convolutional network, a schematic diagram of accurate user and item embedding representations is generated. Detailed Implementation

[0126] The following detailed description of the cold-start-oriented recommendation system and method based on meta-learning graph neural networks of the present invention is provided with reference to the accompanying drawings and specific embodiments.

[0127] Example 1:

[0128] As attached Figure 1 As shown, this embodiment provides a cold-start-oriented recommendation system based on meta-learning graph neural networks. The system includes...

[0129] The bipartite graph construction layer is used to obtain interaction data from the dataset and use the interaction data to construct a user-item bipartite graph.

[0130] The graph embedding layer is used to model the high-order relationships between items in a sequence based on the user-item bipartite graph through a graph convolutional network, and generate accurate user and item embedding representations.

[0131] Sequence encoding layer, used to model the dynamic changes of items within a sequence and user preferences through attention network;

[0132] The meta-learning layer is used to quickly adapt to cold-start recommendation tasks.

[0133] As attached Figure 3 As shown, the user-item bipartite graph in this embodiment includes two types of nodes: user node u and item node v. If a user interacts with an item, there is an edge between the user and the item. Paths connecting multiple item nodes and user nodes capture the correlation between item nodes located at the beginning and end of the path.

[0134] The embedded layer in this embodiment is as follows:

[0135] (1) Aggregate the neighbor information of the target node, using the following formula:

[0136]

[0137]

[0138] in, This means that the item node v at level k is obtained by applying an aggregation function to all its neighbors at level k-1. i neighborhood representation and Let the weight matrix and bias vector of the k-th layer be respectively, and This means that the item node u at level k is obtained by applying an aggregation function to all its neighbors at level k-1. i neighborhood representation and Let the weight matrix and bias vector of the k-th layer be respectively, and The corresponding k-th hidden representation is User node u i The hidden representation of the k-th layer is User-Item Two-Part Diagram and Target v i Adjacent nodes are defined as 1-hop neighbors of the target node; n(v i ) represents node v i The set of 1-hop neighbors; the initial embedding of item node v is represented as x. v The initial embedding representation of user node u is x. u In graph convolutional networks (GCNs), node embeddings depend on both the node's own information and the information of its surrounding neighboring nodes.

[0139] (2) The neighborhood information is combined with the target node representation to form the user and item embedding representation of this layer, as shown in the following formula:

[0140]

[0141]

[0142] in, and Let the transformation weight matrix and bias vector of the k-th layer be... γ represents the pooling function; σ represents the activation function; [;] represents concatenation;

[0143] (3) Since the embeddings of different layers have different semantics, in order to make the embedding expression applied to the cold start scenario more comprehensive and richer, the embedding representations of different layers are weighted and combined to obtain the final embedding representation. The formula is as follows:

[0144]

[0145]

[0146] Among them, a k Let a represent the weight matrix of the k-th layer, used to indicate the importance of embeddings from other layers in the embedding composition of the k-th layer. k These are either manually adjusted parameters or automatically optimized model parameters; to reduce training complexity, a... k Set to 1 / (K+1) uniformly; Embedded representations for users and items.

[0147] The sequence coding layer in this embodiment is as follows:

[0148] (1) The interaction sequence of user u after the graph embedding layer is represented as follows:

[0149] (2) Using user sequence s u As input, each item v q Generate an interactive item set based on the interaction order of the items that precede it. In each interaction sequence, the first item, having no prior interactions, has its set of interactive item pairs defined as follows:

[0150] (3) For each item pair, the mapping relationship between the two items is obtained through an L-layer neural network, as follows:

[0151]

[0152] r l =ReLU(W l r l-1 +b l );

[0153]

[0154] in, This indicates that vectors m and n are connected; v i With v q Represent the head entity and tail entity respectively; L is the number of neural network layers, l∈{1,…,L-1}; W l and b l This represents the weight matrix and bias vector of the l-th layer. Represents the final item v i With item v q The transfer information; v represents the item; q represents the tail entity; r represents the concatenated item vector; the superscript of r indicates the layer number;

[0155] (4) Since there are multiple item pairs in each item pair set, when generating an item transfer representation for each item, all the transformation information is aggregated, as shown in the following formula:

[0156]

[0157] (5) After obtaining the transformation information of each item in the sequence, the correlation information between items in the sequence is combined with the transfer information between items in the sequence to generate the final user interest representation;

[0158] (6) To capture each user's ever-changing interests, an attention network is used to generate a weight for the latent state of each input. This weight represents the influence of the user's current behavior on the final preference. The last item in the user interaction sequence often plays an important role in the next interaction item, so the last interaction item v is considered when generating the attention weight. n The effect is shown in the following formula:

[0159]

[0160]

[0161] in, Represents item v q The weights in user u's final decision; W2, W3, W4, b2, b2 are the parameters of the multilayer sensing network. express The influence weight of item vq;

[0162] (7) User interests, considering the order correlation of items within a sequence and time dynamics, are represented as follows:

[0163]

[0164] (8) After obtaining the representation of the user's current interests, the classic matrix factorization method is used to infer the user's preference for items.

[0165] (9) User u to item v p The predicted score is the user interest I u and item embedding The inner product of these two factors, the probability that a user will interact with an item, is defined as the predicted score:

[0166]

[0167] (10) The parameters are learned using Bayesian Personalized Ranking (BPR) loss, which makes the probability estimate of the interaction items observed by the user higher than that of the items they did not observe. The BPR loss function is defined as:

[0168]

[0169] Among them, T i T represents the training samples for the i-th task. i ={(u,v,v - )|(u,v)∈R + ,(u,v - )∈R -};R + A collection of items that interact with the user; R -θ represents the set of items that have no interaction history with the user; θ represents the learnable parameters; λ is the L2 regularization parameter used to alleviate the overfitting problem.

[0170] The meta-learning layer in this embodiment is as follows:

[0171] (1) Optimize all parameters θ of the graph embedding network using the gradient-based MAML method. g and all parameters θ of the sequence coding layer t ;

[0172] The training task is divided into an inner loop and an outer loop. The inner loop is used to optimize the user interest model, and the outer loop is used to update the parameters through the meta-learning layer.

[0173] (2) Since the user and item representations are globally shared, the item embedding representation will not be updated in the inner loop; the inner loop uses the support set D. s During training, the sequence encoding layer is updated using gradient descent, with the parameters updated to Θ'. t The formula is as follows:

[0174]

[0175] The outer loop optimization process uses query set D. q To optimize all parameters:

[0176]

[0177]

[0178] (3) After the training phase of the meta-learning layer, good initial embeddings of users and items are obtained, as well as the parameters required for the bipartite graph construction layer, graph embedding layer, sequence coding layer, and meta-learning layer.

[0179] (4) During the testing phase of the meta-learning layer, a small amount of new user data is used to fine-tune the bipartite graph construction layer, graph embedding layer, sequence coding layer and meta-learning layer to quickly adapt to the recommendation task for cold-start users.

[0180] Example 2:

[0181] As attached Figure 2 As shown in this embodiment, a recommendation method for cold starts based on meta-learning graph neural networks is described in detail below:

[0182] S1. Obtain the interaction data from the dataset and use the interaction data to construct a user-item bipartite graph;

[0183] S2. Based on bipartite graphs, graph convolutional networks are used to construct high-order relationships between items in sequences, generating accurate embedded representations of users and items;

[0184] S3. Utilize the sequence coding layer to learn the transition information of items within the sequence, and combine it with the attention mechanism to generate dynamic interest representations for users;

[0185] S4. The training data is divided into a support set and a query set using a meta-learning layer. The parameters of the sequence encoding layer are optimized using the support set in the inner loop, and all parameters are updated using the query set in the outer loop. In the cold start task, several gradient descent steps are used to obtain accurate recommendation content, thereby enabling rapid adaptation to the cold start recommendation task.

[0186] The specific method in step S1 of this embodiment is as follows:

[0187] The goal of the meta-learning layer is to learn a recommendation method that is general enough for all users, meaning it can be fine-tuned to adapt to specific users, with the training task T... train Defined as selecting n general users U train ={u1,u2,…,u n}, as attached Figure 1 As shown, K1 consecutive item sets are selected from the interaction sequence of each user u as the support set. K2 items are used as the query set The interaction sequences are constructed into a user-item bipartite graph so that the representations of the nodes in the graph can be aggregated using a graph neural network. (See attached image) Figure 3 As shown, the user-item bipartite graph contains two types of nodes: user nodes u and item nodes v. If a user interacts with an item, an edge exists between the user and the item. Paths connecting multiple item nodes and user nodes can capture the correlation between item nodes located at the beginning and end of the path.

[0188] As attached Figure 3 As shown, in step S2 of this embodiment, the construction of high-order relationships between items in sequences using a graph convolutional network based on a bipartite graph to generate accurate embedded representations of users and items is as follows:

[0189] S201. Aggregate the neighbor information of the target node, using the following formula:

[0190]

[0191]

[0192] in, This means that the item node v at level k is obtained by applying an aggregation function to all its neighbors at level k-1. i neighborhood representation and Let the weight matrix and bias vector of the k-th layer be respectively, and This means that the item node u at level k is obtained by applying an aggregation function to all its neighbors at level k-1. i neighborhood representation and Let the weight matrix and bias vector of the k-th layer be respectively, and The corresponding k-th hidden representation is User node u i The hidden representation of the k-th layer is User-Item Two-Part Diagram and Target v i Adjacent nodes are defined as 1-hop neighbors of the target node; n(v i ) represents node v i The set of 1-hop neighbors; the initial embedding of item node v is represented as x. v The initial embedding representation of user node u is x. u In graph convolutional networks (GCNs), node embeddings depend on both the node's own information and the information of its surrounding neighboring nodes.

[0193] S202. The neighborhood information is combined with the target node representation to form the user and item embedding representation of this layer, as shown in the following formula:

[0194]

[0195]

[0196] in, and Let the transformation weight matrix and bias vector of the k-th layer be... γ represents the pooling function; σ represents the activation function; [;] represents concatenation;

[0197] S203. Since embeddings at different layers have different semantics, in order to make the embedding representations applied to the cold start scenario more comprehensive and richer, the embedding representations at different layers are weighted and combined to obtain the final embedding representation. The formula is as follows:

[0198]

[0199]

[0200] Among them, a k Let a represent the weight matrix of the k-th layer, used to indicate the importance of embeddings from other layers in the embedding composition of the k-th layer. k These are either manually adjusted parameters or automatically optimized model parameters; to reduce training complexity, a... k Set to 1 / (K+1) uniformly; Embedded representations for users and items.

[0201] As attached Figure 1 As shown, in step S3 of this embodiment, the use of a sequence coding layer to learn the transition information of items within a sequence, combined with an attention mechanism, to generate a dynamic interest representation for the user is as follows:

[0202] S301, The interaction sequence of user u after the graph embedding layer is represented as follows:

[0203] S302, with user sequence s u As input, each item v q Generate an interactive item set based on the interaction order of the items that precede it. In each interaction sequence, the first item, having no prior interactions, has its set of interactive item pairs defined as follows:

[0204] S303. For each item pair, the mapping relationship between the two items is obtained through an L-layer neural network, as follows:

[0205]

[0206] r l =ReLU(W l r l-1 +b l );

[0207]

[0208] in, This indicates that vectors m and n are connected; v i With v q Represent the head entity and tail entity respectively; L is the number of neural network layers, l∈{1,…,L-1}; W l and b l This represents the weight matrix and bias vector of the l-th layer. Represents the final item v i With item v q The transfer information; v represents the item; q represents the tail entity; r represents the concatenated item vector; the superscript of r indicates the layer number;

[0209] S304. Since each item pair contains multiple item pairs, when generating an item transfer representation for each item, all transformation information is aggregated, as shown in the following formula:

[0210]

[0211] S305. After obtaining the transformation information of each item in the sequence, the correlation information between items in the sequence is combined with the transition information between items in the sequence to generate the final user interest representation.

[0212] S307. To capture each user's ever-changing interests, an attention network is used to generate a weight for the latent state of each input. This weight represents the influence of the user's current behavior on the final preference. The last item in the user interaction sequence often plays an important role in the next interaction item; therefore, the last interaction item v is considered when generating the attention weight. n The effect is shown in the following formula:

[0213]

[0214]

[0215] in, Represents item v q The weights in user u's final decision; W2, W3, W4, b2, b2 are the parameters of the multilayer sensing network. express The influence weight of item vq;

[0216] S308. User interests, considering the order correlation of items within a sequence and time dynamics, are represented as follows:

[0217]

[0218] S309. After obtaining the representation of the user's current interests, the classic matrix factorization method is used to infer the user's preferences for items.

[0219] S310, User u to item v p The predicted score is the user interest I u and item embedding The inner product of these two factors, the probability that a user will interact with an item, is defined as the predicted score:

[0220]

[0221] S311. Utilize Bayesian Personalized Ranking (BPR) loss to learn parameters such that the probability estimate of the interaction items observed by the user is higher than that of the items they did not observe. The BPR loss function is defined as:

[0222]

[0223] Among them, T i T represents the training samples for the i-th task. i ={(u,v,v - )|(u,v)∈R+ ,(u,v - )∈R -};R + A collection of items that interact with the user; R - θ represents the set of items that have no interaction history with the user; θ represents the learnable parameters; λ is the L2 regularization parameter used to alleviate the overfitting problem.

[0224] As attached Figure 1 As shown, in step S4 of this embodiment, the training data is divided into a support set and a query set using a meta-learning layer. The parameters of the sequence encoding layer are optimized using the support set in the inner loop, and all parameters are updated using the query set in the outer loop. The accurate recommendation content is obtained by using several gradient descent steps in the cold start task, as detailed below:

[0225] S401. Optimize all parameters θ of the graph embedding network using the gradient-based MAML method. g and all parameters θ of the sequence coding layer t ;

[0226] S402. Divide the training task into an inner loop and an outer loop. The inner loop is used to optimize user interest modeling, and the outer loop is used to update parameters through the meta-learning layer.

[0227] S403. Since user and item representations are globally shared, the item embedding representation will not be updated in the inner loop; the inner loop uses the support set D. s During training, the sequence encoding layer is updated using gradient descent, with the parameters updated to Θ'. t The formula is as follows:

[0228]

[0229] The outer loop optimization process uses query set D. q To optimize all parameters:

[0230]

[0231]

[0232] S404. After the training phase of the meta-learning layer, good initial embeddings of users and items are obtained, along with the parameters required for the bipartite graph construction layer, graph embedding layer, sequence coding layer, and meta-learning layer.

[0233] S405. During the testing phase of the meta-learning layer, a small amount of new user data is used to fine-tune the bipartite graph construction layer, graph embedding layer, sequence coding layer, and meta-learning layer to quickly adapt to recommendation tasks for cold-start users.

[0234] Example 3:

[0235] This invention also provides an electronic device, including: a memory and a processor;

[0236] The memory stores the instructions executed by the computer.

[0237] The processor executes computer execution instructions stored in the memory, causing the processor to execute the cold-start-oriented recommendation method based on meta-learning graph neural networks in any embodiment of the present invention.

[0238] The processor can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), off-the-shelf programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The processor can be a microprocessor or any conventional processor.

[0239] Memory is used to store computer programs and / or modules. The processor implements various functions of the electronic device by running or executing the computer programs and / or modules stored in the memory, and by accessing data stored in the memory. Memory can mainly include a program storage area and a data storage area. The program storage area can store the operating system, at least one application program required for a function, etc.; the data storage area can store data created based on the use of the terminal, etc. In addition, memory can also include high-speed random access memory, and can also include non-volatile memory, such as hard disks, RAM, plug-in hard disks, smart memory cards (SMC), secure digital cards (SD cards), flash memory cards, at least one disk storage device, flash memory devices, or other volatile solid-state storage devices.

[0240] Example 4:

[0241] This invention also provides a computer-readable storage medium storing a plurality of instructions, which are loaded by a processor to cause the processor to execute the cold-start-oriented recommendation method based on meta-learning graph neural networks in any embodiment of this invention. Specifically, a system or apparatus equipped with a storage medium may be provided, on which software program code implementing the functions of any of the above embodiments is stored, and the computer (or CPU or MPU) of the system or apparatus may read and execute the program code stored in the storage medium.

[0242] In this case, the program code read from the storage medium can itself implement the function of any of the above embodiments, and therefore the program code and the storage medium storing the program code constitute part of the present invention.

[0243] Storage media embodiments for providing program code include floppy disks, hard disks, magneto-optical disks, optical disks (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RYM, DVD-RW, DVD+RW), magnetic tapes, non-volatile memory cards, and ROMs. Alternatively, program code can be downloaded from a server computer via a communication network.

[0244] Furthermore, it should be clear that not only can the program code read by the computer be executed, but also the operating system or other components operating on the computer can be instructed based on the program code to perform some or all of the actual operations, thereby realizing the function of any of the embodiments described above.

[0245] Furthermore, it is understood that the program code read from the storage medium is written to the memory set in the expansion board inserted into the computer or to the memory set in the expansion unit connected to the computer. Then, based on the instructions of the program code, the CPU or other components installed on the expansion board or expansion unit execute some and all of the actual operations, thereby realizing the function of any of the embodiments described above.

[0246] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A meta-learning graph neural network-based recommendation system for cold start, characterized in that, The system comprises, a bipartite graph construction layer, configured to obtain interaction data in a data set, and construct a user-item bipartite graph by using the interaction data; a graph embedding layer, configured to model high-order relationships between items in sequences by a graph convolution network based on the user-item bipartite graph, and generate accurate user and item embedding representations; a sequence encoding layer, configured to model dynamic changes of items in sequences and user preferences by an attention network; a meta-learning layer, configured to quickly adapt to a cold start recommendation task; wherein the sequence encoding layer is specifically as follows: The interaction sequence of user u after the graph embedding layer is represented as With the user sequence s u As input, each item v q Generate the interaction item pair set with the previous item in the interaction order Where the first item in each interaction sequence will be the first item due to no other interaction items before it, the interaction item pair set of the first item is defined as Each item pair passes through an L-layer neural network to obtain a mapping relationship between the two items, and is specifically as follows: r l = ReLU(W l + b l-1 ); and l ) wherein, denotes the concatenation of vector m with vector n; v i denotes the concatenation of vector m with vector n; v q denote the head and tail entities, respectively; L is the number of neural network layers, l e {1,..., L-1}; W l and b l denote the weight matrix and bias vector of the l-th layer, denotes the transition information of the final item v i and item v q ; v denotes the item; q denotes the tail entity; r denotes the concatenated item vector; the superscript of r denotes the number of layers. Since multiple item pairs exist in each item pair, when generating item transition representations for each item, all transition information is aggregated, and the formula is as follows: After obtaining the transition information of each item in the sequence, the item correlation information between sequences and the transition information between items in the sequence are combined to generate a final user interest representation; To capture the changing interests of each user, an attention network is used to generate a weight for each input hidden state, with the weight representing the influence of the current behavior of the user on the final preference. The last item in the user interaction sequence plays an important role in the next interaction item, so the influence of the last interaction item v n is considered when generating the attention weight, with the formula as follows: wherein, representing the item v q weights on the final decision of the user u; W2, W3, W4, b2, b2 are parameters of the multi-layer perception network, representing the influence weight of the item vq; Considering the sequence correlation and time dynamics of items in the sequence, the user interest is represented as: After obtaining the representation of the current interest of the user, a classical matrix factorization method is used to infer the user's preference for items; User u on item v p The predicted score is the user interest I u and item embedding The inner product of these two factors, the probability that a user will interact with an item, is defined as the predicted score: A Bayesian personalized ranking loss is used to learn the parameters, so that the probability estimate of the observed interaction items of the user is higher than that of the unobserved items, and the BPR loss function is defined as: where T i represents the training sample of the i-th task, i.e., T i = {(u, v, v - ) | (u, v) ∈ R + , (u, v - ) ∈ R -}; R + represents the set of items that the user has interacted with; R - represents the set of items that the user has no interaction history with; θ represents the learnable parameters; λ is the L2 regularization parameter to mitigate overfitting.

2. The meta-learning graph neural network-based recommendation system for cold start according to claim 1, wherein, The user-item bipartite graph includes two types of nodes, user nodes u and item nodes v; If a user interacts with an item, there is an edge between the user and the item; the path connecting multiple item nodes and user nodes captures the correlation between the item nodes located at the beginning and end of the path.

3. The meta-learning graph neural network-based recommendation system for cold start according to claim 1 or 2, characterized in that, The embedding graph layer is specifically as follows: The neighbor information of the target node is aggregated, and the formula is as follows: in, This means that the item node v at level k is obtained by applying an aggregation function to all its neighbors at level k-1. i neighborhood representation and Let the weight matrix and bias vector of the k-th layer be respectively, and This means that the item node u at level k is obtained by applying an aggregation function to all its neighbors at level k-1. i neighborhood representation and Let the weight matrix and bias vector of the k-th layer be respectively, and The corresponding k-th hidden representation is User node u i The hidden representation of the k-th layer is User-Item Two-Part Diagram and Target v i Adjacent nodes are defined as 1-hop neighbors of the target node; n(v i ) represents node v i The set of 1-hop neighbors; the initial embedding of item node v is represented as x. v The initial embedding representation of user node u is x. u In graph convolutional networks, node embedding depends on both the node's own information and its surrounding neighboring nodes. The neighbor information and the target node representation are combined as the user and item embedding representations of this layer, and the formula is as follows: in, and Let the transformation weight matrix and bias vector of the k-th layer be... γ represents the pooling function; σ represents the activation function; [;] represents concatenation; combining the embedding representations of different layers to obtain a final embedding representation The formula is as follows: wherein a k denotes the weight matrix of the k-th layer, which is used to represent the importance of other layer embeddings in the embedding construction of the k-th layer, a k is a manually adjusted parameter or an automatically optimized model parameter; a k is uniformly set to 1 / (K+1); is the user, item embedding representation.

4. The meta-learning graph neural network-based recommendation system for cold start according to claim 3, wherein, The meta-learning layer is specifically as follows: optimizing all parameters θ of the graph embedding network using the gradient-based MAML method g and all parameters θ of the sequence encoding layer t ; The training task is divided into an inner loop and an outer loop, the inner loop is used to optimize user interest modeling, and the outer loop is used to update parameters through the meta-learning layer; Inner loop uses support set D s Training, update the sequence encoding layer by gradient descent, the parameter update is Θ' t The formula is as follows: An outer loop optimization process, using query set D q to optimize all parameters: After the training stage of the meta-learning layer, good initial embeddings of users and items are obtained, and parameters required by the bipartite graph construction layer, the graph embedding layer, the sequence encoding layer and the meta-learning layer are obtained; In the test stage of the meta-learning layer, the bipartite graph construction layer, the graph embedding layer, the sequence encoding layer and the meta-learning layer are fine-tuned using new user data, and quickly adapt to the recommendation task of cold start users.

5. A recommendation method based on meta-learning graph neural network for cold start, characterized in that, The method is specifically as follows: Obtain interaction data in a data set, and construct a user-item bipartite graph by using the interaction data; Model high-order relationships between items in sequences by a graph convolution network based on the bipartite graph, and generate accurate user and item embedding representations; Learn transition information of items in sequences by using the sequence encoding layer, and generate dynamic interest representations for users by using an attention mechanism; Divide the training data into a support set and a query set by using the meta-learning layer, use the support set to optimize the parameters of the sequence encoding layer in the inner loop, use the query set to update all parameters in the outer loop, and obtain accurate recommendation content in the cold start task by using several gradient descent methods, to quickly adapt to the cold start recommendation task; Wherein, the sequence coding layer is used to learn the transition information of items in the sequence, and the attention mechanism is used to generate a dynamic interest representation for the user, as follows: The interaction sequence of user u after the graph embedding layer is represented as With the user sequence s u As input, each item v q The interaction item pair set is generated in the interaction order with the previous item Wherein, the first item in each interaction sequence will be the first item without other interaction items, the interaction item pair set of the first item is defined as Each item pair is used to obtain the mapping relationship between two items through an L-layer neural network, as follows: r l =ReLU(W l r l-1 +b l ); in, This indicates that vectors m and n are connected; v i With v q Represent the head entity and tail entity respectively; L is the number of neural network layers, l∈{1,…,L-1}; W l and b l This represents the weight matrix and bias vector of the l-th layer. Represents the final item v i With item v q The transfer information; v represents the item; q represents the tail entity; r represents the concatenated item vector; the superscript of r indicates the layer number; Since multiple item pairs exist in each item pair, when generating the item transition representation for each item, all transition information is aggregated, as follows: After obtaining the transition information of each item in the sequence, the inter-sequence item correlation information is combined with the inter-sequence item transition information to generate the final user interest representation. To capture the changing interests of each user, an attention network is used to generate a weight for each input hidden state, with the weight representing the influence of the current behavior of the user on the final preference. The last item in the user interaction sequence plays an important role in the next interaction item, so the influence of the last interaction item v n is considered when generating the attention weight, with the formula as follows: wherein, representing the item v q weights on the final decision of the user u; W2, W3, W4, b2, b2 are parameters of the multi-layer perception network, representing the influence weight of the item vq; Considering the sequence order correlation and time dynamics of user interest, the user interest is represented as: After obtaining the representation of the current interest of the user, a classic matrix factorization method is used to infer the user's preference for items. User u on item v p The predicted score is the user interest I u and item embedding The inner product of these two factors, the probability that a user will interact with an item, is defined as the predicted score: The Bayesian personalized ranking loss is used to learn the parameters, so that the probability estimate of the observed interaction items is higher than that of the unobserved items, and the BPR loss function is defined as: where T i represents the training samples of the i-th task, i.e., T i = {(u, v, v - ) | (u, v) ∈ R + , (u, v - ) ∈ R -}; R + is the set of items that the user has interacted with; R - represents the set of items that the user has no interaction history with; θ represents the learnable parameters; λ is the L2 regularization parameter to mitigate overfitting issues; The meta-learning layer is used to divide the training data into a support set and a query set, the support set is used in the inner loop to optimize the parameters of the sequence coding layer, and the query set is used in the outer loop to update all parameters, and in the cold start task, several gradient descent methods are used to obtain accurate recommended content, as follows: optimizing all parameters θ of the graph embedding network using the gradient-based MAML method g and all parameters θ of the sequence encoding layer t ; The training task is divided into an inner loop and an outer loop, the inner loop is used to optimize the user interest modeling, and the outer loop is used to update the parameters through the meta-learning layer; Inner loop uses support set D s Training, update the sequence encoding layer by gradient descent, the parameter update is Θ' t The formula is as follows: An outer loop optimization process uses the query set D to optimize all parameters: q ​ After the training stage of the meta-learning layer, the user and item are obtained Good initial embedding and the parameters required by the bipartite graph construction layer, graph embedding layer, sequence coding layer and meta-learning layer are obtained. In the test stage of the meta-learning layer, the bipartite graph construction layer, the graph embedding layer, the sequence coding layer and the meta-learning layer are fine-tuned using new user data to quickly adapt to the recommendation task of cold start users.

6. The meta-learning graph neural network-based recommendation method for cold start according to claim 5, characterized in that, The user-item bipartite graph includes two types of nodes: user nodes u and item nodes v. If a user interacts with an item, there is an edge between the user and the item; the path connecting multiple item nodes and user nodes captures the correlation between the item nodes located at the beginning and end of the path; Based on the bipartite graph, the graph convolution network is used to construct the high-order relationship between the items in the sequence to generate accurate user and item embedding representations, as follows: The neighbor information of the target node is aggregated, as follows: in, This means that the item node v at level k is obtained by applying an aggregation function to all its neighbors at level k-1. i neighborhood representation and Let the weight matrix and bias vector of the k-th layer be respectively, and This means that the item node u at level k is obtained by applying an aggregation function to all its neighbors at level k-1. i neighborhood representation and Let the weight matrix and bias vector of the k-th layer be respectively, and The corresponding k-th hidden representation is User node u i The hidden representation of the k-th layer is User-Item Two-Part Diagram and Target v i Adjacent nodes are defined as 1-hop neighbors of the target node; n(v i ) represents node v i The set of 1-hop neighbors; the initial embedding of item node v is represented as x. v The initial embedding representation of user node u is x. u In graph convolutional networks, node embedding depends on both the node's own information and its surrounding neighboring nodes. The neighbor information and the target node representation are combined as the user and item embedding representation of this layer, as follows: wherein and denote the transform weight matrix and bias vector of the kth layer and denote the transform weight matrix and bias vector of the kth layer and denote the transform weight matrix and bias vector of the kth layer and denote the transform weight matrix and bias vector of the kth layer and denote the transform weight matrix and bias vector of the kth layer and denote the transform weight matrix and bias vector of the kth layer and combining the embedding representations of different layers to obtain a final embedding representation The formula is as follows: wherein a k denotes the weight matrix of the k-th layer, used to represent the importance of other layer embeddings in the embedding composition of the k-th layer, a k is a manually adjusted parameter or an automatically optimized model parameter; a k is uniformly set to 1 / (K+1); is the user, item embedding representation.

7. An electronic device, comprising: It includes: A memory and at least one processor; The memory has a computer program stored thereon; The at least one processor executes the computer program stored in the memory, so that the at least one processor executes the recommendation method based on the meta-learning graph neural network for cold start as claimed in claim 5 or 6.

8. A computer-readable storage medium, characterized in that, The computer program stored in the computer readable storage medium can be executed by the processor to implement the recommendation method based on the meta-learning graph neural network for cold start as claimed in claim 5 or 6.

Images