Recommender Systems Using Bayesian Graph Convolutional Networks

Abstract

A system and method for processing an observed bipartite graph having a plurality of user nodes, a plurality of item nodes, and an observed graph topology, wherein the observed graph topology defines edges connecting at least some of the user nodes to some of the item nodes such that at least some of the nodes have node neighborhoods that include edge connections to one or more other nodes. A plurality of random graph topologies are derived by replacing the node neighborhoods of at least some of the nodes with the node neighborhoods of other nodes, the plurality of random graph topologies being implementations of the observed graph topology. A nonlinear function is trained using the plurality of user nodes, the plurality of item nodes, and the plurality of random graph topologies to learn user node embeddings and item node embeddings for the plurality of user nodes and the plurality of item nodes, respectively.

Description

[0001] Cross-reference to related applications

[0002] This application claims the benefit of priority to U.S. patent application No. 16 / 789,325, filed on February 12, 2020, entitled “RECOMMENDER SYSTEM USING BAYSESIAN GRAPH CONVOLUTIONNETWORKS,” the entire contents of which are incorporated herein by reference. Technical Field

[0003] The present invention generally relates to processing graph-based data using machine learning techniques, particularly in the context of recommender systems. Background Art

[0004] An information filtering system is a system that removes redundant or unnecessary information from the information stream provided to human users to manage information overload. A recommender system (RS) is a subclass of information filtering systems that attempts to predict users' ratings or preferences for items. In commercial applications, RS is often used to guide users to find content that they are actually interested in among the ever-increasing amount of online information.

[0005] Personalized RS plays an important role in many online services. Accurate personalized RS can benefit users as well as content publishers and platform providers. RS is used in various business fields to provide personalized recommendations to users, for example: providing video or music recommendations for streaming and downloading content provider platforms; providing product recommendations for online retailer platforms; providing application recommendations for app store platforms; providing content recommendations for social media platforms; and recommending news articles for mobile news applications or online news websites.

[0006] RS usually adopts one or both of collaborative filtering (CF) and content-based filtering. Both filtering methods adopt personality-based methods to recommend personalized products or services based on different historical behaviors of different users.

[0007] CF methods typically build predictive models or functions based on the past behavior of the target user or active user (e.g., previously purchased or selected items and / or numerical ratings given to these items) and the past behavior of other users with similar behavioral histories to the active user. In contrast, content-based filtering methods utilize a series of pre-labeled discrete features (item attributes) of items to recommend other items with similar attributes. However, content-based filtering methods may suffer from the fact that a large number of items have a very limited number of associated item attributes.

[0008] Some RSs integrate content-based filtering methods into CF methods to create hybrid systems. However, due to the lack of appropriate item attributes in the surging number of items available through online platforms, most RSs still rely heavily on CF methods for recommendations based on user historical behaviors.

[0009] CF methods can generally be summarized as: step 1) find users who share the same interaction patterns as the active user (the user making the prediction); step 2) use the ratings / interactions of those users with similar intent found in step 1 to calculate the prediction of the active user. Finding users who share the same interaction patterns requires identifying similar users or similar items. The process of deriving similar users and similar items involves embedding each user and each item in the created low-dimensional space so that similar users are nearby and similar items are nearby. In this regard, an embedding is a mapping of discrete-categorical-variables to vectors of continuous numbers.

[0010] A classic and successful CF method is matrix factorization (MF). MF algorithms represent items and users by vectors in the same space inferred from the observed historical user-item interaction entries. MF algorithms work by decomposing the user-item interaction matrix into the product of two low-dimensional rectangular matrices, with the goal of representing users and items in a low-dimensional latent space (also called embedding representation in the context of deep learning algorithms). Recently, deep learning models have been introduced in various applications, which have significantly improved the performance compared to traditional models. However, due to the lack of explicit encoding of latent collaborative information from user-item interactions and over-reliance on explicit feedback from users (which is usually sparse), deep learning models are not sufficient to produce optimal user / item embeddings.

[0011] Therefore, researchers turned to the emerging field of graph convolution neural network (GCNN) and applied GCNN for recommendation by modeling the user-item interaction history as a bipartite graph. GCNN is trained to learn user and item representations of user and item nodes in a graph structure, and the user-item interaction history is modeled as connecting edges between nodes. The vector representation of a node is learned by iteratively combining the embedding of the node itself (i.e., mapping a discrete variable to a continuous number vector) with the embedding of the node in its local neighborhood. In the context of neural networks, an embedding is a learned low-dimensional continuous vector representation of a discrete variable. Neural network embeddings are useful because they can reduce the dimensionality of categorical variables and meaningfully represent categories in the transformed space.

[0012] Most existing methods divide the process of learning node vector representations (i.e., embeddings) into two steps: neighborhood aggregation, in which an aggregation function operates on a collection of vectors to aggregate neighborhood embeddings; and central neighborhood combination, which combines the aggregated neighborhood vectors with the central node embedding. These methods learn node embeddings on graphs in a convolutional manner by representing nodes as functions of their surrounding neighborhoods.

[0013] Existing bipartite graph RS schemes based on GCNN regard the observed graph as the ground truth description of the relations, thus treating the observed graph as very powerful prior knowledge. However, due to data sparsity, the bipartite user-item interaction graph actually often lacks many edges and thus reflects very limited information.

[0014] Learning fixed and incomplete graphs ignores all potential preferences of users and thus suffers from deficiencies in diversity and effectiveness in RS applications. This may lead to degraded recommendation performance when learning graphs. Accordingly, there is a need for a GCNN-based recommender system that can compensate for the data sparsity inherent in environments where the number of users and the amount of content are growing rapidly. Summary of the invention

[0015] According to one aspect of the present invention, there is provided a computer-implemented method for processing an observed bipartite graph having a plurality of user nodes, a plurality of item nodes, and an observed graph topology, wherein the observed graph topology defines connecting at least some of the user nodes to some of the item nodes such that at least some of the nodes have edges of node neighborhoods including edge connections to one or more other nodes. The method comprises: deriving a plurality of random graph topologies by replacing the node neighborhoods of at least some of the nodes with the node neighborhoods of other nodes, the plurality of random graph topologies being implementations of the observed graph topology; using the plurality of user nodes, the plurality of item nodes, and the plurality of random graph topologies to learn a nonlinear function to generate user node embeddings and item node embeddings for the plurality of user nodes and the plurality of item nodes, respectively.

[0016] According to the above aspect, the node neighborhoods of at least some nodes are replaced with the node neighborhoods of other nodes according to probabilistic sampling based on calculated similarities between pairs of nodes.

[0017] According to any one of the above aspects, the at least some nodes and the other nodes are respectively corresponding user nodes, and deriving the multiple random graph topologies includes: defining a user-user distance matrix, the user-user distance matrix specifies the distance of each node pair of the user nodes; replacing the node neighborhoods of at least some nodes with the node neighborhoods of other nodes includes: sampling the other nodes according to probability values ​​corresponding to the distances specified in the user-user distance matrix; wherein, compared with a larger distance, a smaller distance indicates a greater similarity between user nodes and corresponds to a larger probability value.

[0018] According to any one of the above aspects, the specified distance of each pair of user nodes is based on a plurality of item nodes in a union of node neighborhoods of the pair of user nodes and a plurality of item nodes in an intersection of node neighborhoods of the pair of user nodes.

[0019] According to any one of the above aspects, the specified distance of each pair of user nodes is also based on the heat of any item node in the intersection of the node neighborhoods of the pair of user nodes.

[0020] According to any one of the above aspects, learning the nonlinear function includes: using the user nodes and the item nodes to generate a corresponding set of user node embeddings and item node embeddings for each of the random graph topologies; and determining a set of final user node embeddings and item node embeddings based on the generated multiple sets of user node embeddings and item node embeddings.

[0021] According to any one of the above aspects, determining the set of end-user node embeddings and project node embeddings includes: calculating the average value of the user node embeddings generated for each user node in all the random graph topologies; calculating the average value of the project node embeddings generated for each project node in all the random graph topologies.

[0022] According to any of the above aspects, a graph convolution neural network (GCCN) is pre-trained to generate a set of initial user node embeddings and item node embeddings based on the observed bipartite graph, wherein learning the non-linear function includes: for each of the random graph topologies, retraining the pre-trained GCCN separately to generate the user node embeddings and the item node embeddings for the random graph topology.

[0023] According to any one of the above aspects, the multiple user nodes of the observed bipartite graph are respectively represented as corresponding user feature vectors, and the corresponding user feature vectors represent the attributes of the corresponding users; the multiple project nodes are respectively represented as corresponding project node feature vectors, and the corresponding project node feature vectors represent the attributes of the corresponding projects, and the observed graph topology is defined in an adjacency matrix; the user node embedding is a low-dimensional representation of the corresponding user feature vector, and the project node embedding is a low-dimensional representation of the corresponding project feature vector.

[0024] According to any of the above aspects, the method includes: determining user-specific item recommendations based on the user node embeddings and the item node embeddings.

[0025] According to another aspect of the present invention, there is provided a graph processing system for processing an observed bipartite graph, the observed bipartite graph comprising a plurality of user nodes, a plurality of project nodes, and a graph topology, the graph topology defining edges that connect at least some of the user nodes to some of the project nodes so that at least some of the nodes have node neighborhoods that include connections to one or more other nodes. The system comprises: a graph generation module for deriving a plurality of random graph topologies by replacing the node neighborhoods of at least some nodes with the node neighborhoods of other nodes, the plurality of random graph topologies being implementations of the observed graph topology; an embedding module for learning a nonlinear function using the plurality of user nodes, the plurality of project nodes, and the random graph topology to generate user node embeddings and project node embeddings for the plurality of user nodes and the plurality of project nodes, respectively.

[0026] According to another aspect above, the graph generation module is used to replace the node neighborhoods of at least some nodes with the node neighborhoods of other nodes based on probability sampling based on calculated similarities between node pairs.

[0027] According to any one of the other aspects above, the at least some nodes and the other nodes are user nodes, and the graph generation module includes: a similarity operation for defining a user-user distance matrix, wherein the user-user distance matrix specifies the distance of each node pair of the user nodes; a graph sampling operation for replacing the node neighborhoods of at least some user nodes with the node neighborhoods of other user nodes by: sampling the other user nodes according to probability values ​​corresponding to the distances specified in the user-user distance matrix; wherein, compared with a larger distance, a smaller distance indicates a greater similarity between the user nodes and corresponds to a larger probability value.

[0028] According to any one of the above-mentioned other aspects, the specified distance of each pair of user nodes is based on a plurality of item nodes in a union of node neighborhoods of the pair of user nodes and a plurality of item nodes in an intersection of node neighborhoods of the pair of user nodes.

[0029] According to any one of the above-mentioned other aspects, the specified distance of each pair of user nodes is also based on the heat of any item node in the intersection of the node neighborhoods of the pair of user nodes.

[0030] According to any one of the other aspects above, the embedding module is used to learn the nonlinear function in the following manner: for the nonlinear function, generating a corresponding set of user node embeddings and item node embeddings for each of the random graph topologies; and determining a set of final user node embeddings and item node embeddings based on the generated multiple sets of user node embeddings and item node embeddings.

[0031] According to any one of the other aspects above, the embedding module includes an average calculation operation for determining the set of final user node embeddings and project node embeddings in the following manner: calculating the average of the user node embeddings generated for each user node in all the random graph topologies; calculating the average of the project node embeddings generated for each project node in all the random graph topologies.

[0032] According to any one of the other aspects above, the nonlinear function is implemented using a graph convolution neural network (GCCN), and the embedding module 108 is used to: pre-train the GCCN to generate a set of initial user node embeddings and item node embeddings based on the user nodes, the item nodes and the observed graph topology of the observed bipartite graph; and learn the nonlinear function by retraining the pre-trained GCCN separately to generate the user node embeddings and the item node embeddings for each of the random graph topologies.

[0033] According to any one of the other aspects above, the multiple user nodes of the observed bipartite graph are respectively represented as corresponding user feature vectors, and the corresponding user feature vectors represent the attributes of the corresponding users; the multiple item nodes are respectively represented as corresponding item node feature vectors, and the corresponding item node feature vectors represent the attributes of the corresponding items; the graph topology is defined in an adjacency matrix, and the adjacency matrix specifies the edges that connect user nodes to item nodes to represent the interaction history between user nodes and item nodes; the user node embedding is a low-dimensional representation of the corresponding user feature vector, and the item node embedding is a low-dimensional representation of the corresponding item feature vector.

[0034] According to any of the above another aspects, the system includes a recommender system selection operation for determining user-specific item recommendations based on the user node embeddings and the item node embeddings. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Reference is now made, by way of example, to the accompanying drawings which show exemplary embodiments of the present application, in which:

[0036] Figure 1 A block diagram showing an example of a bipartite graph;

[0037] Figure 2 A block diagram showing an example of a graph processing system based on machine learning for processing graph structured data provided by an exemplary embodiment;

[0038] Figure 3 Shows Figure 2 A block diagram of a graph generation module of the graph processing system shown;

[0039] Figure 4 A block diagram showing a portion of a bipartite graph;

[0040] Figure 5 A block diagram showing a portion of a bipartite graph;

[0041] Figure 6 Shows Figure 2 The block diagram of the nonlinear function processing system shown in the figure;

[0042] Figure 7 The exemplary embodiment provides a method for using Figure 2 The flowchart of the process performed by the processing system shown in FIG.

[0043] Figure 8 shows a method for executing machine-readable instructions to implement Figure 2 A block diagram of an exemplary processing system for the illustrated system.

[0044] Similar reference numerals may be used in different drawings to represent similar components. DETAILED DESCRIPTION

[0045] According to an exemplary embodiment, a graph processing system is disclosed, which includes a recommender system (RS) based on a Bayesian graph convolution neural network (GCNN).

[0046] A graph is a data structure consisting of nodes and edges. Each node represents an instance or data point defined by measurement data represented as a set of node features (e.g., a multidimensional feature vector). Each edge represents a relationship connecting two nodes. A bipartite graph is a graph structure in which each node belongs to one of two different node types and only direct relationships (e.g., 1-hop neighborhoods) exist between nodes of different types. Figure 1 A sample of an observed bipartite graph 101 is briefly shown, wherein the observed bipartite graph 101 includes two types of nodes, namely user nodes u and A to u F (collectively referred to as user node set U) and project node v A to v D (collectively referred to as the project node set V). In the present invention, "u" is used to refer to one or more general user nodes, and "v" is used to refer to one or more general project nodes. Each corresponding user node u represents an instance of a user. For example, user node u i ∈ R ... j A specific instance of item j may be represented. For example, an item may be a product or service that a user can use. For example, in various scenarios, an item may be: an audio / video media item (e.g., a movie or series or video) that a user can stream or download from an online video content provider; an audio media item (e.g., a song or podcast) that a user can stream or download from an online audio content provider; an image / text media item (e.g., a news piece, a magazine article, or an advertisement) that an online content provider can provide to a user; a software application (e.g., an online application) that a user can download or access from an online software provider (e.g., an app store); different physical products that a user can order for delivery or pickup from an online retailer. The examples of possible categories of items provided above are illustrative and not exhaustive.

[0047] In an exemplary embodiment, user node u A to u F and item node v A to v F are defined by a set of corresponding node features. For example, each user node u is defined by a corresponding user node feature vector x that specifies a set of user node features. u Definition. Each user node feature numerically represents a user attribute. For example, examples of user attributes include user ID, age, gender, marital status, pet owner, etc. In general, the user node set U can be represented as the user node feature matrix X u, where each row in the matrix is ​​the feature vector x of the corresponding user node u u Each item node v is represented by a corresponding item node feature vector x that specifies a set of item node features v Definition. Each item node feature numerically represents an item attribute. For example, in the case of movie videos, examples of item attributes may include: ID, movie title, director, actor, genre, country of production, year of release, period depicted, etc. In general, the item node set V can be represented as the item node feature matrix X v , where each row in the matrix is ​​the feature vector x of the corresponding item node u v .

[0048] The edges 102 connecting the user nodes u to the corresponding item nodes v represent the relationship between the nodes and, in general, define the observed graph topology G obs . In some exemplary embodiments, the presence or absence of an edge 102 between nodes indicates the presence or absence of a relationship of a predefined type between the user represented by the user node and the item represented by the item node. For example, the presence or absence of an edge 102 between a user node u and an item node v indicates whether the user has previously taken an action expressing emotion or interest in a particular item, such as "clicking" a representation of the item or submitting a scale (e.g., 1 star to 5 stars) or a binary (e.g., "like") rating of the item. For example, an edge 102 may represent a click or rating history between a user and an item. In the illustrative embodiment described below, the edge 102 conveys binary relationship information, such that the presence of an edge indicates the presence of a relationship of a defined type (e.g., user i has previously "clicked" or rated / liked item j), and the absence of an edge indicates the absence of such a relationship. However, in further embodiments, the edge 102 may be associated with other attributes representing the strength of the relationship (e.g., the number of "clicks" by a user on a particular item, or a rating given by a user). In some embodiments, the edge 102 may indicate that the user has purchased, ordered, or otherwise consumed an item.

[0049] In an exemplary embodiment where edge 102 conveys the presence or absence of a defined relationship, the graph topology G obs It can be represented by an adjacency matrix A, which defines a binary value matrix that indicates the presence or absence of a connecting edge between each user node u and each item node v. In some examples, the adjacency matrix A corresponds to a "click" or "rating" matrix.

[0050] Therefore, the bipartite graph 101 includes information about users (e.g., a user node set U, represented by a user node feature matrix X u), information about the project (e.g., the project node set V, represented by the project node feature matrix X v representation) and information about historical interactions between users and items (e.g., graph topology G obs , represented by the adjacency matrix A).

[0051] In many practical situations, the information present in the observed bipartite graph 101 has an inherent data sparsity problem, which is due to the fact that the graph topology G obs The historical interaction data present in is typically quite limited, especially in the case of new users and items with fewer interaction records. Therefore, many user nodes and many item nodes may have very few connecting edges. Existing RS schemes may also rely too much on the observed bipartite graph 101, especially the graph topology, as the absolute ground truth. In addition, existing RS schemes may tend to recommend items similar to items represented in the user's behavior history (e.g., the user likes brand "A" cola, so brands "B" and "C" cola are recommended), resulting in a lack of diversity in recommendation results (e.g., users may benefit from snack recommendations to go with their previously favorite brand "A" cola, but the RS only recommends other types of cola).

[0052] Accordingly, as will be described in more detail below, exemplary embodiments described in some applications can solve one or more of the above problems faced by RS solutions. Specifically, in exemplary embodiments, a graph processing system is disclosed, wherein the observed graph topology G obs is not considered as ground truth, but as a sample from a family of random graph topology parameters. The graph processing system is used to generate multiple graph topologies {G1,…,G M}, the multiple graph topologies belong to the same graph topology G as the observed graph topology obs The same random graph parameter family is then used to use GCNN according to the user node U, the project node V and the generated multiple graph topologies {G1,…,G M}Generate user-item relationships (e.g., predict recommendations). Therefore, in the graph processing system currently described, relationship embeddings are generated based on multiple randomly generated graph topologies rather than directly from the observed bipartite graph topology represented by the adjacency matrix A.

[0053] in this regard, Figure 2 The block diagram of an example of a machine learning-based graph processing system 100 for processing bipartite graph structured data provided by an exemplary embodiment is shown. In the example shown, the graph processing system 100 includes a graph generation module 106 and a prediction embedding module 108. The graph generation module 106 is used to generate a set of M random bipartite graph topologies {G1, ..., G M}, where the set of M random bipartite graph topologies is the observed bipartite graph topology G obs In an exemplary embodiment, M is a hyperparameter selected based on historical experience with similar data sets. The embedding module 108 is used to apply a Bayesian approximation method to jointly learn the M bipartite graph topologies {G1, ..., G M}The user embedding and item embedding of the user node u and the item node v for each of the.

[0054] Will refer to Figure 3 The graph generation module 106 is described. In the example shown, the graph generation module 106 includes a similarity operation 302 and a graph sampling operation 304. The similarity operation 302 is used to define a user-user distance matrix DIST, which specifies the distance of each user node u relative to the other nodes u. Different algorithms can be used to calculate the user-user distance matrix DIST. Two examples of possible implementations of the similarity operation 302 are described below, namely: (1) a function based on the Jacquard coefficient; (2) a function based on a 2-hop walk to determine a probabilistic distance metric.

[0055] The Jacquard coefficient, also known as the intersection-over-union ratio, is a statistical measure used to measure the similarity and diversity of sample sets. In an exemplary embodiment, the sample set is the user's interaction history (neighborhood set). In this regard, Figure 4 A pair of user nodes (u i ,u j ) and the item nodes (v A 、v B 、v C 、v D ). Figure 4 In the example, user node u i The direct (1-hop) neighborhood of B ,v C ,v D}, user node u j The direct (1-hop) neighborhood of A ,v C ,v D}. N(u i ) and N(u j ) represent the user node u respectively i and u j Neighborhood set. For every two user nodes (u i ,u j ), Jacquard distance dist(u i ,u j) is the quotient of the size of the intersection of the neighborhood sets (e.g., the number of common item nodes, such as "common hits", which is 2 in the example shown) and the size of the union of the neighborhood sets (e.g., the total number of unique item nodes contained in the combination of the two sets, which is 4 in the example shown), mathematically expressed as (Equation 1):

[0056]

[0057] In an exemplary embodiment where the similarity operation 302 is based on the Jacquard coefficient, by performing the similarity operation 302 on the bipartite graph G contained in the observed obs Each pair of user nodes (u i ,u j ) performs the Jacquard distance calculation in equation (1) and fills the user-user distance matrix DIST.

[0058] In an alternative case where the similarity operation 302 is used to determine a probabilistic distance metric based on a 2-hop walk, an increased weight is applied to the case where two user nodes are connected to unpopular public items, as opposed to the case where two user nodes are connected to popular public items. In this regard, the heat of the item nodes is considered using a 2-hop walk when identifying a common neighborhood. Figure 5 This is shown in the example, Figure 5 An example is Figure 4 The same bipartite graph example shown, except that an additional user node u is introduced x and u y , each user node has a corresponding item node v b Accordingly, in Figure 5 In the example, user node u x and u y All are user nodes (u i ,u j ) to the public project node v B , indicating the 2-hop neighborhood of the project node v B represents popular items, with nodes connected only to the user (u i ,u j )'s project node v C In an exemplary embodiment, the public unpopular node v C The user node (u i ,u j ) is interpreted as being greater than the similarity between the popular nodes v B The user node (u i ,u j ). In such exemplary embodiments, the user-user distance matrix DIST is populated according to the node-node distance values ​​represented by the distance equation (2):

[0059]

[0060] In the distance equation (2), the neighborhood set intersection is normalized by degree D(v), which is the number of user nodes u that are directly connected to the item node v.

[0061] The graph sampling operation 304 is used to sample the observed graph topology G according to the values ​​in the user-user distance matrix DIST. obs Repeated sampling is performed to generate a graph topology G containing the observed obs A set of random graph topologies {G1,…,G M Specifically, the graph sampling operation 302 is used to create a graph topology G obs The implementation method of {G1,…,G M}, wherein the connection to the direct (i.e., 1-hop) project node v neighborhood of the user node u is replaced by a connection to the direct project node v neighborhood of a similar user node u. In this regard, the graph sampling operation 302 is used to generate a new graph topology, wherein the direct neighborhood of a user node is replaced by a neighborhood copy of other similar user node neighborhoods. The distance values ​​contained in the user-user distance matrix DIST correspond to distance probabilities, wherein the shorter the distance between two corresponding user nodes, the higher the similarity between the two user nodes, and the higher the probability that the direct neighborhood of one of the user nodes will be copied and used to replace the direct neighborhood of the other node.

[0062] in this regard, Figure 3 An example of neighborhood replication is shown graphically. Figure 3 In the example, user node u j and u m are all nodes observed in the user node set U of the bipartite graph 101, and have the graph topology G represented as obs According to the distance metric dist(u j ,u m ) is randomly determined to convert the user node u j The immediate neighbors (e.g., B ,v C ,v D} connection) is replaced by the user node u m The immediate neighbors (e.g., X ,v Y ,v Z} connection), thereby obtaining the sampled user node u' jRepeat the node sampling process for all user nodes u in the observed bipartite graph 101 to generate the graph topology G obs Random graph topology implementation G∈{G1,…,G M}. Repeat the whole process M times to provide M random graph topology implementations {G1,…,G M}.

[0063] In an exemplary embodiment, the graph sampling operation 304 effectively generates M different adjacency matrices {A1, ..., A M}, to replace the observed adjacency matrix A.

[0064] In an exemplary embodiment, the graph sampling operation 304 is used to select a random number according to the auxiliary random vector ζ∈{1,2,…,N} N Generate the graph topology G obs Every random graph topology realization G∈{G1,…,G M}, where N is the observed graph topology G obs The number of user nodes in the j'th item ζ j Represents the edges and direct adjacencies of the project node v will replace the observed graph topology G obs The user node on the edge of the j'th node in . In an exemplary embodiment, the posterior distribution of the random vector ζ is defined by the following equation (3):

[0065]

[0066]

[0067] Among them, dist(j,m) is the calculated distance specified in the similarity matrix DIST.

[0068] Accordingly, in an exemplary embodiment, the graph sampling operation 304 generates a random graph topology realization G∈{G1, . . . , G M}:

[0069]

[0070] Repeat the generating function of equation (4) M times to obtain M different graph topologies {G1,…,G M}, where each graph topology belongs to the observed graph topology G obs The same family of random graph topology parameters.

[0071] refer to Figure 6 In an exemplary embodiment, the embedding module 108 is used to embed the M different graph topologies {G1, ..., G M} each applies a Bayesian approximation or a nonlinear function 602 to jointly learn a user node embedding e for each of the user nodes and item nodes for each of the M graph topologies, respectively. u and project nodes embedded in e v Then, through the average calculation operation 604, the average of the user embedding and item embedding generated for each user node and item node in the M graphs is calculated for each node to provide a final embedding for each node.

[0072] In an exemplary embodiment, the nonlinear function 602 is implemented using one or more GCNNs 603. In this regard, the GCNN 603 can take the form of any bipartite graph convolutional neural network suitable for implementing RS and is used to process the bipartite graph according to a set of learnable parameters (e.g., weights W). In an exemplary embodiment, the GCNN 603 is initially pre-trained using a traditional GCCN training method to learn a set of initial weights W init and a set of initial user node embeddings and item node embeddings E init ={e u(1) 0 ,…,e u(N) 0} and {e v(1) 0 ,…,e v(F) 0}, where N is the number of user nodes u, and F is the number of project nodes v.

[0073] Then, using the M different graph topologies {G1,…,G M} Retrain the GCNN 603. For the M different graph topologies {G1,...,G M}, the GCNN 603 is trained separately in combination with the user node set U and the project node set V to learn multiple sets of corresponding GCNN weights {W1,…,W M} and multiple sets of corresponding user node embeddings and item node embeddings {E1,…,E M}, where the specific graph topology G j The item node embedding and user node embedding are E j =(user node embedding {e u(j ,1),…,e u(j,N)}, the project node is embedded in {e v(j ,1),…,e v(j,F) In order to target the M different graph topologies {G1,…,G M}, for each individual graph topology {G1,…,G M}, using the observed graph topology Gobs A set of initial weights W generated init and a set of initial user node embeddings and item node embeddings E init Reinitialize the GCNN 603. In some examples, the M different graph topologies {G1, ..., G M}, the software implementation of the GCNN 603 is repeated to train each of the M different graph topologies {G1, ..., G M} multiple software implementations of multiple parallel training of the GCNN 603.

[0074] Accordingly, the nonlinear function 602 outputs M sets of embeddings {E1, ..., E M}, each group is embedded in E i A low-dimensional representation of each of the user node u and the item node v in a corresponding graph topology G, the graph topology G itself being the observed graph topology G obs Then, through the average value calculation operation 604, all the graph topologies {G1,…,G M} to generate a set of final embeddings E final = User node embedding {e u * (1) ,…,e u * (N)} and project nodes embedded in {e v * (1) ,…,e v * (F)}.

[0075] In at least some applications, using the graph topology G observed from obs A set of randomly generated graph topologies {G1,…,G M} to retrain the previously observed graph topology G obs Trained GCNNs can improve the embedding quality by introducing uncertainty in the learning process.

[0076] In summary, in the above exemplary embodiment, the graph generation module 106 implements a neighborhood replication graph generation framework, which generates a neighborhood replication graph from the observed bipartite graph 101 (including the user node set U, the project node set V and the observed graph topology G obs) to learn the generation operations (e.g., the similarity operation 302 and the graph sampling operation 304). Then, the generated set of graph topologies {G1,…,G M} is used together with the user node set U and the item node set V to learn a nonlinear function 602, thereby capturing the user's hidden preferences and introducing diversity. In an exemplary embodiment, the graph sampling operation 304 applies probabilistic sampling to generate a random graph topology {G1, ..., G M In an exemplary embodiment, the user node set U, the item node set V, and the graph topology G of the observed bipartite graph 101 are used. obs To pre-train the GCNN 603, and then for the generated random graph topology {G1,...,G M} to retrain the nonlinear function 602 and the GCNN 603 to generate multiple sets of corresponding embeddings {E1, ..., E M}, and then the average is calculated on a per-node basis to provide the final embedding E final .

[0077] Then, the set of final embeddings E can be processed using the known RS method. final In this regard, in an exemplary embodiment, the recommender system selection operation 606 may be combined with the graph processing system 100 to implement RS. As described above, the final embedding E final The final user node embedding e of each of the N user nodes u * and the final item node embedding e of each of the F item nodes v * In an exemplary embodiment, the recommender system selects operation 606 for determining user-specific recommendations as follows. For each user-item pair, the user node embedding e is calculated. u * and the item node is embedded in e v * The corresponding pairwise dot products of u * ·e v * ). Therefore, in the case of F project nodes v, for each user node u i , F scalar value dot products will be calculated. Each scalar value represents the dot product with the user node u iThe probability prediction that the associated user i will click on the corresponding item j for which the scalar value has been calculated. In the case where RS is used to recommend up to K items, K items that were not previously connected to the user and have the highest scalar values ​​calculated for the user will be selected for recommendation to the user.

[0078] Thus, in some examples, user-specific item recommendations may be used to generate targeted messages that are communicated to specific users. For example, the targeted messages may be generated on an automated computer-based RS operated by a platform provider. An electronic device associated with the user may access or receive the targeted messages via a communications network and present a representation of the targeted messages to the user via a user interface of the electronic device.

[0079] Figure 7 A flow chart of actions performed by a graph processing system 100 provided by an exemplary embodiment is shown. The graph processing system 100 is used to process an observed bipartite graph 101, the bipartite graph 101 comprising user nodes u, item nodes v, and edges 102, the edges 102 connecting at least some of the user nodes u to some of the item nodes v, such that at least some of the user nodes u have a node neighborhood N(u) comprising one or more item nodes v.

[0080] The graph generation module 106 is used to derive multiple random graph topologies {G1, ..., G M}, the plurality of random graph topologies are the observed bipartite graph topologies G obs The embedding module 108 is used to use the user node set U, the project node set V of the observed bipartite graph 101 and the multiple random graph topologies {G1, ..., G M} to learn the nonlinear function 602 to generate user node embeddings {e* u(1) ,…,e* u(N)} and project nodes embedded in {e* v(1) ,…,e* v(F)}(action 706).

[0081] In an exemplary embodiment, the graph generation module 106 is used to replace the node neighborhoods of at least some user nodes with the node neighborhoods of other user nodes based on probabilistic sampling based on the calculated similarities between pairs of user nodes. In this regard, the similarity operation 302 of the graph generation module 106 can be used to define a user-user distance matrix DIST, which specifies the distance matrix DIST of each pair of user nodes (u i ,u j ) (action 704A). The graph sampling operation 304 is used to replace the node neighborhoods of at least some user nodes with the node neighborhoods of other user nodes by sampling the other user nodes according to probability values ​​corresponding to the distances specified in the user-user distance matrix DIST (action 704B); wherein, compared with a larger distance, a smaller distance indicates a greater similarity between the user nodes and corresponds to a larger probability value.

[0082] In some examples, each pair of user nodes (u i ,u j ) is based on the pair of user nodes (u i ,u j ) in the node neighborhood of the union of multiple project nodes v and the pair of user nodes (u i ,u j ) in the intersection of their node neighborhoods. In some examples, each pair of user nodes (u i ,u j ) is also based on the pair of user nodes (u i ,u j )’s heat of any project node v in the intersection of the node neighborhoods.

[0083] In an exemplary embodiment, the nonlinear function 602 is learned by: for the random graph topology {G1, . . . , G M Each of} generates a corresponding set of user node embeddings and item node embeddings E j ={e u(j ,1),…,e u(j,N)} and {e v(j ,1),…,e v(j,F)}; Determine a set of final user node embeddings and project node embeddings E according to the generated multiple sets of user node embeddings and project node embeddings final ={e* u(1) ,…,e* u(N)} and {e* v(1) ,…,e* v(F)}(action 706A).

[0084] In an exemplary embodiment, the embedding module 108 includes an average calculation operation 604 for determining the set of end-user node embeddings and item node embeddings E by: final :Compute the random graph topology {G1,…,G M}; Calculate the average value of all the user node embeddings generated by each user node in the random graph topology {G1,…,G M} generates the average value of all the item node embeddings generated by each item node in the said project node (action 706B).

[0085] In an exemplary embodiment, the nonlinear function 602 is learned by the GCCN 603, and the embedding module 108 is used to first pre-train the GCCN 603 to generate a set of initial user node embeddings and item node embeddings E according to the observed bipartite graph 101. init (Action 702). The embedding module 108 is used to learn the nonlinear function 602 by retraining the pre-trained GCCN 603 respectively to obtain the random graph topology {G1, ..., G M} each generates an embedding of the user node u and an embedding of the item node v.

[0086] In some examples, user-specific item recommendations may be determined based on the user node embeddings and the item node embeddings.

[0087] In the above example, the similarity operation 302 and the graph sampling operation 304 are performed with a focus on user nodes, so that the graph generation module 106 generates a graph topology in which the neighborhood of the user node is replaced with the neighborhood of other user nodes. However, in an alternative embodiment, the graph generation module 106 can be used to perform the similarity operation 302 and the graph sampling operation 304 from the perspective of project nodes rather than from the perspective of user nodes. In such examples, the neighborhood of the project node is replaced with the neighborhood of the project node similar to the derived random graph implementation. In such alternative embodiments, the process for determining the similarity between the project node and the sampled project node neighborhood to generate a random graph topology can be the same as the above-mentioned process regarding the project node, but the roles of the project node and the user node will be reversed.

[0088] Processing Unit

[0089] In an exemplary embodiment, the graph processing system 100 is a computer implemented using one or more computing devices. Figure 8A block diagram of an exemplary processing unit 170 is shown, which may be used in a computing device to execute machine-executable instructions to implement system 100. Other processing units suitable for implementing the embodiments described herein may be used, which may include components different from those discussed below. Although Figure 8 A single instance of each component is shown, but there may be multiple instances of each component in the processing unit 170.

[0090] The processing unit 170 may include one or more processing devices 172, such as a processor, a microprocessor, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a dedicated logic circuit or a combination thereof. In addition, the processing unit 170 may also include one or more input / output (I / O) interfaces 174, which may be connected to one or more appropriate input devices 184 and / or output devices 186. The processing unit 170 may include one or more network interfaces 176 for wired or wireless communication with a network.

[0091] In addition, the processing unit 170 may also include one or more storage units 178, which may include a large capacity storage unit, such as a solid state drive, a hard disk drive, a magnetic disk drive, and / or an optical disk drive. The processing unit 170 may include one or more memories 180, which may include volatile or non-volatile memory (e.g., flash memory, random access memory (RAM), and / or read-only memory (ROM)). The memory 180 may store instructions for execution by the processing device 172, such as executing the examples described in the present invention. The one or more memories 180 may include other software instructions, such as for implementing an operating system and other applications / functions.

[0092] A bus 182 may be present to provide communication between components of the processing unit 170, including the processing device 172, the I / O interface 174, the network interface 176, the storage unit 178, and / or the memory 180. The bus 182 may be any suitable bus architecture including, for example, a memory bus, a peripheral bus, or a video bus.

[0093] Although the present invention uses steps in a certain order to describe methods and processes, one or more steps of the methods and processes may be omitted or modified as appropriate. One or more steps may be performed in an order other than the order described as appropriate.

[0094] Although the present invention is at least partially described in terms of methods, it will be understood by those skilled in the art that the present invention also relates to various components for performing at least some aspects and features of the described methods, whether hardware components, software, or any combination of the two. Therefore, the technical solution of the present invention can be embodied in the form of a software product. Suitable software products can be stored in a pre-recorded storage device or other similar non-volatile or non-transient computer-readable medium, including DVDs, CD-ROMs, USB flash drives, mobile hard disks, or other storage media. The software product includes instructions tangibly stored thereon, which enable a processing device (e.g., a personal computer, a server, or a network device) to execute the method examples disclosed herein.

[0095] The present invention may be embodied in other specific forms without departing from the subject matter of the claims. The exemplary embodiments described are considered to be illustrative and non-restrictive in all respects. Selected features in one or more of the above embodiments may be combined to create alternative embodiments not explicitly described, and features suitable for such combinations are understood to fall within the scope of the present invention.

[0096] All values ​​and subranges within the disclosed range are also disclosed. In addition, although the disclosed and illustrated systems, devices and processes herein may include a specific number of elements / assemblies, these systems, devices and assemblies may be modified to include more or less such elements / assemblies. For example, although any disclosed element / assembly may be a single quantity, the embodiments disclosed herein may be modified to include a plurality of such elements / assemblies. The subject matter described herein is intended to encompass and include all suitable changes in technology.

Claims

1. A computer-implemented method for processing an observed bipartite graph, characterized in that The method is applied to a recommendation system based on a Bayesian graph convolutional network, the observed bipartite graph comprising a plurality of user nodes, a plurality of item nodes, and an observed graph topology, the observed graph topology defining edges connecting at least some of the user nodes to some of the item nodes such that at least some of the nodes have node neighborhoods comprising edge connections to one or more other nodes, the method comprising: deriving a plurality of random graph topologies by replacing the node neighborhoods of at least some nodes with the node neighborhoods of other nodes, the plurality of random graph topologies being implementations of the observed graph topology; The plurality of user nodes, the plurality of item nodes, and the plurality of random graph topologies are used to learn a nonlinear function to generate user node embeddings and item node embeddings for the plurality of user nodes and the plurality of item nodes, respectively, wherein the user node is used to represent a user associated with a specific registered user account or a unique user identifier, and the item node is used to represent a specific instance of a product or service used by the user; the user node is defined by a set of user node features, each of the user node features represents a user attribute, and the products or services used by the user include media items, software applications, and physical products; the item node is defined by a set of item node features, each of the item node features represents an item attribute; and the edges connecting at least some of the user nodes to some of the item nodes indicate that the user has taken an action expressing emotion or interest in a specific item.

2. The method according to claim 1, characterized in that: The node neighborhoods of at least some nodes are replaced with the node neighborhoods of other nodes according to probabilistic sampling based on calculated similarities between pairs of nodes.

3. The method according to claim 2, characterized in that The at least some nodes and the other nodes are respectively corresponding user nodes, and deriving the multiple random graph topologies includes: defining a user-user distance matrix, the user-user distance matrix specifies the distance of each node pair of the user nodes; replacing the node neighborhoods of at least some nodes with the node neighborhoods of other nodes includes: sampling the other nodes according to probability values ​​corresponding to the distances specified in the user-user distance matrix; wherein, compared with a larger distance, a smaller distance indicates a greater similarity between user nodes and corresponds to a larger probability value.

4. The method according to claim 3, characterized in that The specified distance of each pair of user nodes is based on a plurality of item nodes in a union of node neighborhoods of each pair of user nodes and a plurality of item nodes in an intersection of node neighborhoods of each pair of user nodes.

5. The method according to claim 4, characterized in that The specified distance of each pair of user nodes is also based on the heat of any item node in the intersection of the node neighborhoods of the each pair of user nodes.

6. The method according to any one of claims 1 to 5, characterized in that Learning the nonlinear function includes: using the user nodes and the item nodes to generate a corresponding set of user node embeddings and item node embeddings for each of the random graph topologies; and determining a set of final user node embeddings and item node embeddings based on the generated multiple sets of user node embeddings and item node embeddings.

7. The method according to claim 6, characterized in that Determining the set of final user node embeddings and item node embeddings includes: calculating the average of the user node embeddings generated for each user node in all the random graph topologies; calculating the average of the item node embeddings generated for each item node in all the random graph topologies.

8. The method according to claim 6, characterized in that Also includes: A graph convolutional neural network (GCCN) is pre-trained to generate a set of initial user node embeddings and item node embeddings based on the observed bipartite graph, wherein learning the non-linear function comprises: for each of the random graph topologies, retraining the pre-trained GCCN respectively to generate the user node embeddings and the item node embeddings for the random graph topology.

9. The method according to any one of claims 1 to 5, characterized in that The plurality of user nodes of the observed bipartite graph are respectively represented as corresponding user feature vectors, the corresponding user feature vectors representing attributes of the corresponding users; the plurality of project nodes are respectively represented as corresponding project node feature vectors, the corresponding project node feature vectors representing attributes of the corresponding projects, and the observed graph topology is defined in an adjacency matrix; The user node embedding is a low-dimensional representation of the corresponding user feature vector, and the item node embedding is a low-dimensional representation of the corresponding item feature vector.

10. The method according to claim 9, characterized in that Also includes: Based on the user node embeddings and the item node embeddings, user-specific item recommendations are determined.

11. A graph processing system for processing an observed bipartite graph, characterized in that The graph processing system comprises a Bayesian graph convolutional network based recommendation system, the observed bipartite graph comprises a plurality of user nodes, a plurality of item nodes, and a graph topology, the graph topology defining edges connecting at least some of the user nodes to some of the item nodes such that at least some of the nodes have node neighborhoods that include connections to one or more other nodes, the graph processing system comprising: a graph generation module for deriving a plurality of random graph topologies by replacing the node neighborhoods of at least some nodes with the node neighborhoods of other nodes, the plurality of random graph topologies being implementations of the observed graph topology; An embedding module is used to learn a nonlinear function using the multiple user nodes, the multiple item nodes and the random graph topology to generate user node embeddings and item node embeddings for the multiple user nodes and the multiple item nodes, respectively, wherein the user node is used to represent a user associated with a specific registered user account or a unique user identifier, and the item node is used to represent a specific instance of a product or service used by the user; the user node is defined by a set of user node features, each of which represents a user attribute, and the products or services used by the user include media items, software applications, and physical products; the item node is defined by a set of item node features, each of which represents an item attribute; the edges connecting at least some of the user nodes to some of the item nodes indicate that the user has taken an action expressing emotion or interest in a specific item.

12. The system according to claim 11, characterized in that The graph generation module is used to replace the node neighborhoods of at least some nodes with the node neighborhoods of other nodes according to probability sampling based on calculated similarities between node pairs.

13. The system according to claim 12, characterized in that The at least some nodes and the other nodes are user nodes, and the graph generation module comprises: A similarity operation for defining a user-user distance matrix, the user-user distance matrix specifying a distance between each node pair of the user nodes; A graph sampling operation is used to replace the node neighborhoods of at least some user nodes with the node neighborhoods of other user nodes in the following manner: sampling the other user nodes according to probability values ​​corresponding to distances specified in the user-user distance matrix; wherein, compared with larger distances, smaller distances indicate greater similarity between user nodes and correspond to larger probability values.

14. The system according to claim 13, characterized in that The specified distance of each pair of user nodes is based on a plurality of item nodes in a union of node neighborhoods of each pair of user nodes and a plurality of item nodes in an intersection of node neighborhoods of each pair of user nodes.

15. The system according to claim 14, characterized in that The specified distance of each pair of user nodes is also based on the heat of any item node in the intersection of the node neighborhoods of the each pair of user nodes.

16. The system according to any one of claims 11 to 15, characterized in that The embedding module is used to learn the nonlinear function in the following manner: for the nonlinear function, generating a corresponding set of user node embeddings and item node embeddings for each of the random graph topologies; and determining a set of final user node embeddings and item node embeddings based on the generated multiple sets of user node embeddings and item node embeddings.

17. The system according to claim 16, characterized in that The embedding module includes an average calculation operation for determining the set of end-user node embeddings and item node embeddings by: calculating an average of the user node embeddings generated for each user node in all of the random graph topologies; An average of the item node embeddings generated for each item node in all of the random graph topologies is calculated.

18. The system according to claim 16, characterized in that The nonlinear function is implemented using a graph convolutional neural network (GCCN), and the embedding module is used to: pre-train the GCCN to generate a set of initial user node embeddings and item node embeddings based on the user nodes, the item nodes, and the observed graph topology of the observed bipartite graph; The non-linear function is learned by retraining the pre-trained GCCN separately to generate the user node embedding and the item node embedding for each of the random graph topologies.

19. The system according to any one of claims 11 to 15, characterized in that The plurality of user nodes of the observed bipartite graph are respectively represented as corresponding user feature vectors, the corresponding user feature vectors representing attributes of the corresponding users; the plurality of item nodes are respectively represented as corresponding item node feature vectors, the corresponding item node feature vectors representing attributes of the corresponding items; the graph topology is defined in an adjacency matrix, the adjacency matrix specifies the edges connecting user nodes to item nodes to represent interaction histories between user nodes and item nodes; The user node embedding is a low-dimensional representation of the corresponding user feature vector, and the item node embedding is a low-dimensional representation of the corresponding item feature vector.

20. The system according to claim 19, characterized in that Also included is a recommender system selection operation for determining user-specific item recommendations based on the user node embedding and the item node embedding.

21. A computer readable medium, characterized in that The computer-readable medium comprises instructions which, when executed by a processor of a processing system, cause the processing system to perform a method according to any one of claims 1 to 10 .

22. A computer program product, characterized in that The computer program product comprises instructions which, when executed by a processor of a processing system, cause the processing system to perform a method according to any one of claims 1 to 10 .

Images