An online interest group recommendation method based on self-attention and contrastive learning

Abstract

The application discloses an online interest group recommendation method based on self-attention and contrast learning, and recommends online interest groups to users through an interest group recommendation model SCL4GR, wherein the interest group recommendation model is composed of a node embedding module, a sequence encoding module, a recommendation generation module and a contrast learning module, and the recommendation method comprises the following steps: firstly, three social networks are constructed, and then a graph convolution network GCN is used to obtain representation vectors of users and online interest groups; then the representation vectors are embedded into a Transformer model to capture the mode of user interest change; finally, the dynamic interest is spliced with the user representation vector by using the recommendation generation module to obtain the current preference of the user, the preference is input into a multi-layer fully connected neural network with a softmax function, the probability distribution of all candidate interest groups is obtained, and then the online interest groups are recommended to the user. Compared with traditional methods such as a recurrent neural network and a Markov chain, the adaptability and execution efficiency of the model are significantly improved.

Description

Technical Field

[0001] This invention relates to the field of online interest group recommendation technology, and more specifically to an online interest group recommendation method based on self-attention and contrastive learning. Background Technology

[0002] Event-based social networks (EBSNs) have become increasingly popular in recent years, such as Meetup.com in the US and Douban in China. They provide convenience for users who interact simultaneously in online interest groups and offline events. As of February 2023, Meetup, one of the largest EBSNs, had over 50 million users and generated over 10,000 events weekly in more than 190 countries through its platform.

[0003] Since Liu et al. [1] first proposed the concept of EBSN, people have conducted extensive research on different types of recommendation problems in EBSN [2]. Among them, event recommendation and interest group recommendation are two key areas. The purpose of event recommendation is to try to recommend interesting events to individual users, while interest group recommendation focuses on how to recommend events to user groups with common interests. In addition to the above recommendations, there are other types of recommendation work, such as venue recommendation [3], event-partner pair recommendation [4], bilateral recommendation [5], etc. Although many achievements have been made in the field of EBSN recommendation, there is still a neglected recommendation problem, namely online interest group recommendation. Unlike traditional interest group recommendation, online interest group recommendation aims to recommend online interest groups to individual users.

[0004] On the EBSN platform, if a user wants to participate in an event, they must first join the online interest group that posted the event. Over time, a user's interests may change, requiring them to join new interest groups. Joining new interest groups satisfies a user's psychological need to make new friends, explore new things, and pursue their passions. However, with hundreds of thousands of online interest groups on the platform, finding a suitable one is not easy. Therefore, helping users solve this problem is of great practical significance in improving user satisfaction and loyalty to the platform.

[0005] In general, solving the online interest group recommendation problem faces three challenges: 1) Dynamic interests. Users' interest in joining interest groups may change dynamically over time, requiring an effective method to capture the patterns of change; 2) Sparse supervision signals. Most models implement recommendation tasks under a supervised learning paradigm. In EBSN, there are typically hundreds of thousands of online interest groups, but most users only join a few, resulting in extremely sparse interaction data. The lack of sufficient training data hinders the model from achieving its full performance; 3) Heterogeneous social networks. Unlike traditional social networks (such as Facebook and Twitter), EBSN users simultaneously possess online and offline social networks. Online networks are formed by users joining online interest groups, while offline networks are formed by their participation in offline events. Social activities in these two networks interact and promote each other. Modeling the cooperative relationships between these networks is crucial to improving the performance of online interest group recommendations.

[0006] To address the aforementioned challenges, this paper proposes a novel online interest group recommendation model (named SCL4GR) based on self-attention and contrastive learning. First, considering that historical interest group sequences contain dynamic changes in user interests, inspired by the great success of Transformer sequence models in machine translation [6], this paper utilizes the self-attention mechanism of Transformer to capture the changing patterns of user interests. Compared with traditional sequence models such as recurrent neural networks and Markov chains, the Transformer-based model has significantly improved in terms of model adaptability and execution efficiency. Second, to address the challenge of sparsity of supervision signals, inspired by the successful application of self-supervised learning (SSL) in computer vision [7] and natural language processing [8], this paper proposes a learning paradigm based on contrastive self-supervised learning. In this paradigm, the classic supervision task is supplemented by an auxiliary self-supervised task, which enhances the learning of user / group / event representations through self-discrimination. Finally, to model the cooperative associations of different social networks, this paper generates different social views of an instance (e.g., user / interest group / event) and captures cooperative associations through contrastive self-supervised learning. Specifically, this paper first constructs three social networks: an online network, an offline network, and an integrated network (i.e., a combination of the first two networks). This paper views these three networks as different perspectives on users' social relationships. Then, this paper uses graph convolutional networks (GCNs)[9] on each of the three networks to obtain view-aware representations of instances in each network. Finally, this paper brings the views of the same instance closer together in the embedding space to encourage the extraction of unique information contained in each view into another view; at the same time, this paper separates the perspectives of different instances, thereby enhancing the discriminative power of the representation.

[0007] References: Summary of the Invention

[0008] The purpose of this invention is to address the shortcomings of existing technologies by proposing an online interest group recommendation method based on self-attention and contrastive learning. First, three social networks are constructed: an online network, an offline network, and an integrated network (a combination of the first two networks). These three networks are considered as different perspectives on the user's social relationships. Then, graph convolutional networks (GCNs) [9] are used on the three networks respectively to obtain view-aware representations of instances in each network. Finally, the views of the same instance are brought closer together in the embedding space to encourage the extraction of unique information contained in each view into another view, thus separating the perspectives of different instances and enhancing the discriminative power of the representation.

[0009] To achieve the above objectives, the present invention adopts the following technical solution:

[0010] An online interest group recommendation method based on self-attention and contrastive learning recommends online interest groups to users through an interest group recommendation model (named SCL4GR). The interest group recommendation model consists of a node embedding module, a sequence encoding module, a recommendation generation module, and a contrastive learning module.

[0011] The node embedding module uses a graph convolutional network (GCN) to convert each node of the online network into a vector, encoding its features and structural information, to obtain the representation vectors of users and online interest groups.

[0012] The sequence encoding module is used to embed the transformed online network nodes into the Transformer model to capture the patterns of changes in user interests and obtain the dynamic interests hidden in the user behavior sequence.

[0013] The recommendation generation module is used to concatenate dynamic interests with user representation vectors to obtain the user's current preferences, and input the preferences into a multi-layer fully connected neural network with a softmax function to obtain the probability distribution of all candidate interest groups.

[0014] The contrastive learning module is used to enhance the supervision task by using different social networks as different views as nodes and through contrastive learning tasks.

[0015] The method for recommending online interest groups to users using an interest group recommendation model is as follows:

[0016] Step 1: Construct three social networks—online network, offline network, and integrated network—based on different perspectives on users' social relationships. Obtain all node information in the user's online network and input it into the node embedding module. The node embedding module converts each node in the online network into a vector and encodes its feature and structural information to obtain the representation vectors of users and online interest groups.

[0017] Step 2: Embed the representation vectors of users and online interest groups obtained after node transformation into the sequence encoding module to capture the patterns of user interest changes and obtain the dynamic interests hidden in the user behavior sequence.

[0018] Step 3: Use the recommendation generation module to concatenate the dynamic interest with the user representation vector to obtain the user's current preference. Then, input the preference into a multi-layer fully connected neural network with a softmax function to obtain the probability distribution of all candidate interest groups, and then recommend the corresponding online interest groups to the user.

[0019] Specifically, in step 1, the node embedding module converts each node of the online network into a vector and encodes its feature and structural information to obtain representations of users and online interest groups. The specific process is as follows:

[0020] Let the online network be represented as G. on = <U,G,A on >, where U represents the user set, U = {u1, u2, ..., u} n}; G represents an online interest set, G = {g1, g2, ..., g} m}; E represents the event set, E = {e1, e2, ..., e k};A on Let edge set represent an edge set, where if a user participates in an online interest group, there is an edge connecting the user and the interest group.

[0021] Let the embedding vector of user U be The embedding vector of interest group G is Where d is the vector dimension, and the superscript (0) indicates the initial vector;

[0022] Since the interest groups a user joins can reflect their interests and hobbies, and the users who join an interest group can also be used as features of that interest group to measure the similarity between two interest groups, this characteristic can be used to propagate information between connected users and interest groups. Through a graph convolutional network (GCN), for a user node u, its surrounding neighbors are aggregated using an aggregation function to obtain the user's representation vector after the first layer of the GCN:

[0023]

[0024] In the above formula, Let N(u) represent the user's representation vector after the first layer of GCN, where N(u) is the set of user U's first-order neighbors and the user itself, and d u Increment the degree of user node u by 1, d g It is the degree of node g. This is the weight matrix used to extract information, and d′ is the transformed dimension. It is the bias, and σ(·) is the activation function;

[0025] Similarly, through the graph convolutional network GCN, for a node g of interest group, its surrounding neighbors are aggregated using an aggregation function to obtain the representation vector of interest group G after the first layer of GCN;

[0026] Since higher-order association information has a significant impact on evaluating the correlation between users and interest groups, it is often necessary to stack multiple GCN layers to obtain the final representation vectors of user u and interest group g.

[0027] Assuming l GCN layers are stacked, a user or interest group can receive information propagated from l-hop neighbors; in the l-hop step, the iterative calculation formula for the representation of user u is:

[0028]

[0029] in, It is a trainable weight matrix, d l ,d l-1 These are the transformation dimensions of the l-th and (l-1)-th layers, respectively. It is the interest group representation output by the (l-1)th layer;

[0030] Similarly, the representation vector of the interest group g in the l-th layer GCN is calculated.

[0031] By stacking multiple GCN layers, and using the output of the last GCN layer as the final representation vector for user u and interest group g, respectively denoted as...

[0032] Specifically, step 2 involves inputting the representation vector of the online interest group obtained after node transformation into the sequence encoding module to capture the pattern of user interest changes and obtain the dynamic interests hidden in the user behavior sequence. The specific process is as follows:

[0033] Let S u =[g u,1 ,...,g u,i ,...,g u,t[] represents the online interactive interest group sequence of user u. To utilize the order information of the input sequence, the position vector is added to the interest group representation vector through a position encoding method. Then, the interest group representation vector is calculated for each position simultaneously in a Transformer layer using a Transformer encoding model. Finally, the interest group representation vectors of all positions are stacked into a matrix. Used to calculate the attention weights of users to interest groups, thereby obtaining the dynamic interests hidden in the user behavior sequence;

[0034] Stacked matrix of interest group representation vectors for each layer The calculation formula is as follows:

[0035] B (l-1) =LN(G (l-1) +Dropout(MH(G (l-1) (3)

[0036] Trm(G (l-1) )=LN(B (l-1) +Dropout(PFFN(B (l-1) (4)

[0037]

[0038] Where d is the transformation dimension, l∈{1...L} represents the l-th layer, LN(·), Dropout(·), and MH(·) represent layer normalization, dropout, and multi-head attention operations, respectively, and PFFN(·) represents a point-to-feedforward network.

[0039] Specifically, in step 3, the recommendation generation module concatenates dynamic interests with user representation vectors to obtain the user's current preferences. These preferences are then input into a multi-layer fully connected neural network with a softmax function to obtain the probability distribution of all candidate interest groups. Online interest groups are then recommended to the user. The specific process is as follows:

[0040] To learn more complex transition patterns, multiple Transformer layers are typically stacked. Let l Transformer layers produce the final output matrix of all interest groups in the user behavior sequence.

[0041] Suppose that the interest group g in step t is... t The mask represents the interest group obtained in step t. As a masked interest group g t The predicted value of the characterization; taking into account Only the dynamic interests and preferences of user u were encoded, without considering the user's own characteristic input, such as user ID, age, and gender. Therefore, further... and user representation The data is concatenated to obtain the final representation of the user's preferences at time step t.

[0042] User preference representations are input into a K-layer fully connected network with ReLU activation to generate the probability distribution of user u’s interest in each candidate interest group.

[0043]

[0044]

[0045] in, These are parameters for the science department, and d is the dimension of the embedding vector;

[0046] Based on the probability distribution of user u’s interest in each candidate interest group, the corresponding online interest groups are recommended to the user.

[0047] Furthermore, step 1 also includes deriving self-supervised signals from the original user behavior sequences using contrastive learning. This is achieved by using online networks, offline networks, and ensemble networks as three different views of user social relationships, each capturing unique information about these relationships. Identical instances across different views are treated as positive sample pairs, while different instances across different views are treated as negative sample pairs. Then, InfoNCE loss is used to maximize the consistency between positive sample pairs and minimize the consistency between negative sample pairs. All views collaborate to maximize the model's capabilities. The specific process is as follows:

[0048] Considering the contrast between the online and offline network views, since only user nodes appear in both networks simultaneously, only the contrast loss L between user embeddings is calculated. on-off :

[0049]

[0050] in, Used to calculate user positive sample pairs Losses:

[0051]

[0052]

[0053] Where sim(·) is the similarity function, used to measure the similarity between two vectors. It is the online network representation vector of user u. It is the negative sample representation vector in the offline network view;

[0054] Considering the contrast between the online network view and the integrated network view, we first calculate the loss between positive user pairs and positive interest group pairs separately, and then fuse the two losses to obtain the contrast loss L between the two views. on-int :

[0055]

[0056] Among them, the hyperparameter λ1 is used to control the strength of the contrast effect of the interest group representation vectors;

[0057] Similarly, considering the contrast between the offline network view and the integrated network view, the contrast loss between positive user pairs and positive event pairs is calculated separately, and the final fused loss L is obtained. off-int :

[0058]

[0059] Where λ2 is a hyperparameter;

[0060] By fusing the contrast losses from the different views described above, we obtain the objective function for the self-supervised learning task:

[0061] L c =L on-off +β1L on-int +β2L off-int (14)

[0062] Where β1 and β2 are hyperparameters.

[0063] Furthermore, since a bidirectional Transformer model is used to predict the next interest group a user wants to join, the Cloze test task is applied to sequence recommendation based on the user's interaction sequence. The specific process is as follows:

[0064] For each input sequence, several interest groups "[mask]" in the input sequence are randomly masked, and then the original IDs of the masked interest groups are predicted based only on their left and right contexts. The latent vectors corresponding to the "[mask]" tags are input into a softmax function acting on the set of interest groups to obtain the preference probabilities of the masked interest groups. The final objective function is defined as follows:

[0065]

[0066] Among them, S u ′ is the sequence of interest groups S u The masked version; G u m Represents the masked set of interest groups; It is the real interest group that is masked; P(·) is the probability defined by formula (8);

[0067] To improve recommendation performance, step 3 also employs a multi-task training strategy, combining classic recommendation tasks and self-supervised learning tasks for training:

[0068] L = L s +ρL c (16)

[0069] Where ρ is a hyperparameter that controls the intensity of the self-supervised learning task.

[0070] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0071] 1. This invention proposes a novel online interest group recommendation model, SCL4GR, which uses Transformer and GCN techniques to capture users' dynamic interest patterns and high-order social interactions between users. Compared with traditional sequence models such as recurrent neural networks and Markov chains, the Transformer-based model has significant improvements in model adaptability and execution efficiency.

[0072] 2. This invention proposes an online interest group recommendation method based on self-attention and contrastive learning. Based on the contrastive self-supervised learning paradigm of the EBSN heterogeneous network, the self-discrimination task is used as a self-supervised task to provide auxiliary supervision signals for representation learning. At the same time, contrastive learning is used to model the cooperative associations between different social networks in EBSN, which promotes mutual enhancement between different social views. Attached Figure Description

[0073] Figure 1 This is a diagram illustrating the overall architecture of the online interest group recommendation model SCL4GR of this invention. Detailed Implementation

[0074] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0075] This invention provides an online interest group recommendation method based on self-attention and contrastive learning, through methods such as... Figure 1 The interest group recommendation model SCL4GR shown recommends online interest groups to users. The interest group recommendation model consists of a node embedding module, a sequence encoding module, a recommendation generation module, and a contrastive learning module.

[0076] The node embedding module uses a graph convolutional network (GCN) to convert each node of the online network into a vector, encoding its features and structural information, to obtain the representation vectors of users and online interest groups.

[0077] The sequence encoding module is used to embed the transformed online network nodes into the Transformer model to capture the patterns of changes in user interests and obtain the dynamic interests hidden in the user behavior sequence.

[0078] The recommendation generation module is used to concatenate dynamic interests with user representation vectors to obtain the user's current preferences, and input the preferences into a multi-layer fully connected neural network with a softmax function to obtain the probability distribution of all candidate interest groups.

[0079] The contrastive learning module is used to enhance the supervision task by using different social networks as different views as nodes and through contrastive learning tasks.

[0080] The method for recommending online interest groups to users using an interest group recommendation model is as follows:

[0081] Step 1: Construct three social networks—online network, offline network, and integrated network—based on different perspectives on users' social relationships. Obtain all node information in the user's online network and input it into the node embedding module. The node embedding module converts each node in the online network into a vector and encodes its feature and structural information to obtain the representation vectors of users and online interest groups.

[0082] Step 2: Embed the representation vectors of users and online interest groups obtained after node transformation into the sequence encoding module to capture the patterns of user interest changes and obtain the dynamic interests hidden in the user behavior sequence.

[0083] Step 3: Use the recommendation generation module to concatenate the dynamic interest with the user representation vector to obtain the user's current preference. Then, input the preference into a multi-layer fully connected neural network with a softmax function to obtain the probability distribution of all candidate interest groups, and then recommend the corresponding online interest groups to the user.

[0084] Specifically, in step 1, the node embedding module converts each node of the online network into a vector and encodes its feature and structural information to obtain representations of users and online interest groups. The specific process is as follows:

[0085] Let the online network be represented as G. on = <U,G,A on >, where U represents the user set, U = {u1, u2, ..., u} n}; G represents an online interest set, G = {g1, g2, ..., g} m}; E represents the event set, E = {e1, e2, ..., e k};A on Let edge set represent an edge set, where if a user participates in an online interest group, there is an edge connecting the user and the interest group.

[0086] Let the embedding vector of user U be The embedding vector of interest group G is Where d is the vector dimension, and the superscript (0) indicates the initial vector;

[0087] Since the interest groups a user joins can reflect their interests and hobbies, and the users who join an interest group can also be used as features of that interest group to measure the similarity between two interest groups, this characteristic can be used to propagate information between connected users and interest groups. Through a graph convolutional network (GCN), for a user node u, its surrounding neighbors are aggregated using an aggregation function to obtain the user's representation vector after the first layer of the GCN:

[0088]

[0089] In the above formula, Let N(u) represent the user's representation vector after the first layer of GCN, where N(u) is the set of user U's first-order neighbors and the user itself, and d u Increment the degree of user node u by 1, d g It is the degree of node g. This is the weight matrix used to extract information, and d′ is the transformed dimension. It is the bias, and σ(·) is the activation function;

[0090] Similarly, through the graph convolutional network GCN, for a node g of interest group, its surrounding neighbors are aggregated using an aggregation function to obtain the representation vector of interest group G after the first layer of GCN;

[0091] Since higher-order association information has a significant impact on evaluating the correlation between users and interest groups, it is often necessary to stack multiple GCN layers to obtain the final representation vectors of user u and interest group g.

[0092] Assuming l GCN layers are stacked, a user or interest group can receive information propagated from l-hop neighbors; in the l-hop step, the iterative calculation formula for the representation of user u is:

[0093]

[0094] in, It is a trainable weight matrix, d l ,d l-1 These are the transformation dimensions of the l-th and (l-1)-th layers, respectively. It is the interest group representation output by the (l-1)th layer;

[0095] Similarly, the representation vector of the interest group g in the l-th layer GCN is calculated.

[0096] By stacking multiple GCN layers, and using the output of the last GCN layer as the final representation vector for user u and interest group g, respectively denoted as...

[0097] Specifically, step 2 involves inputting the representation vector of the online interest group obtained after node transformation into the sequence encoding module to capture the pattern of user interest changes and obtain the dynamic interests hidden in the user behavior sequence. The specific process is as follows:

[0098] Let S u =[g u,1 ,...,g u,i ,...,g u,t [] represents the online interactive interest group sequence of user u. To utilize the order information of the input sequence, the position vector is added to the interest group representation vector through a position encoding method. Then, the interest group representation vector is calculated for each position simultaneously in a Transformer layer using a Transformer encoding model. Finally, the interest group representation vectors of all positions are stacked into a matrix. Used to calculate the attention weights of users to interest groups, thereby obtaining the dynamic interests hidden in the user behavior sequence;

[0099] Stacked matrix of interest group representation vectors for each layer The calculation formula is as follows:

[0100] B (l-1) =LN(G (l-1) +Dropout(MH(G (l-1) (3)

[0101] Trm(G (l-1) )=LN(B (l-1) +Dropout(PFFN(B (l-1) (4)

[0102]

[0103] Where d is the transformation dimension, l∈{1...L} represents the l-th layer, LN(·), Dropout(·), and MH(·) represent layer normalization, dropout, and multi-head attention operations, respectively, and PFFN(·) represents a point-to-feedforward network.

[0104] Specifically, in step 3, the recommendation generation module concatenates dynamic interests with user representation vectors to obtain the user's current preferences. These preferences are then input into a multi-layer fully connected neural network with a softmax function to obtain the probability distribution of all candidate interest groups. Online interest groups are then recommended to the user. The specific process is as follows:

[0105] To learn more complex transition patterns, multiple Transformer layers are typically stacked. Let l Transformer layers produce the final output matrix of all interest groups in the user behavior sequence.

[0106] Suppose that the interest group g in step t is... t The mask represents the interest group obtained in step t. As a masked interest group g t The predicted value of the characterization; considering h u L ,t Only the dynamic interests and preferences of user u were encoded, without considering the user's own characteristic input, such as user ID, age, and gender. Therefore, further... and user representation The data is concatenated to obtain the final representation of the user's preferences at time step t.

[0107] User preference representations are input into a K-layer fully connected network with ReLU activation to generate the probability distribution of user u’s interest in each candidate interest group.

[0108]

[0109]

[0110] in, These are parameters for the science department, and d is the dimension of the embedding vector;

[0111] Based on the probability distribution of user u’s interest in each candidate interest group, the corresponding online interest groups are recommended to the user.

[0112] Furthermore, step 1 also includes deriving self-supervised signals from the original user behavior sequences using contrastive learning. This is achieved by using online networks, offline networks, and ensemble networks as three different views of user social relationships, each capturing unique information about these relationships. Identical instances across different views are treated as positive sample pairs, while different instances across different views are treated as negative sample pairs. Then, InfoNCE loss is used to maximize the consistency between positive sample pairs and minimize the consistency between negative sample pairs. All views collaborate to maximize the model's capabilities. The specific process is as follows:

[0113] Considering the contrast between the online and offline network views, since only user nodes appear in both networks simultaneously, only the contrast loss L between user embeddings is calculated. on-off :

[0114]

[0115] in, Used to calculate user positive sample pairs Losses:

[0116]

[0117]

[0118] Where sim(·) is the similarity function, used to measure the similarity between two vectors. It is the online network representation vector of user u. It is the negative sample representation vector in the offline network view;

[0119] Considering the contrast between the online network view and the integrated network view, we first calculate the loss between positive user pairs and positive interest group pairs separately, and then fuse the two losses to obtain the contrast loss L between the two views. on-int :

[0120]

[0121] Among them, the hyperparameter λ1 is used to control the strength of the contrast effect of the interest group representation vectors;

[0122] Similarly, considering the contrast between the offline network view and the integrated network view, the contrast loss between positive user pairs and positive event pairs is calculated separately, and the final fused loss L is obtained. off-int :

[0123]

[0124] Where λ2 is a hyperparameter;

[0125] By fusing the contrast losses from the different views described above, we obtain the objective function for the self-supervised learning task:

[0126] L c =L on-off +β1L on-int +β2L off-int (14)

[0127] Where β1 and β2 are hyperparameters.

[0128] Furthermore, since a bidirectional Transformer model is used to predict the next interest group a user wants to join, the Cloze test task is applied to sequence recommendation based on the user's interaction sequence. The specific process is as follows:

[0129] For each input sequence, several interest groups "[mask]" in the input sequence are randomly masked, and then the original IDs of the masked interest groups are predicted based only on their left and right contexts. The latent vectors corresponding to the "[mask]" tags are input into a softmax function acting on the set of interest groups to obtain the preference probabilities of the masked interest groups. The final objective function is defined as follows:

[0130]

[0131] Among them, S u ′ is the sequence of interest groups S u The masked version; G u m Represents the masked set of interest groups; It is the real interest group that is masked; P(·) is the probability defined by formula (8);

[0132] To improve recommendation performance, step 3 also employs a multi-task training strategy, combining classic recommendation tasks and self-supervised learning tasks for training:

[0133] L = L s +ρL c (16)

[0134] Where ρ is a hyperparameter that controls the intensity of the self-supervised learning task.

[0135] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An online interest group recommendation method based on self-attention and contrastive learning, characterized in that, The online interest group is recommended to a user through an interest group recommendation model, and the interest group recommendation model is composed of a node embedding module, a sequence encoding module, a recommendation generation module and a contrast learning module; The node embedding module is used for converting each node of an online network into a vector through a graph convolution network (GCN) to encode feature and structure information of the node, and obtaining a representation vector of a user and an online interest group; The sequence encoding module is used for embedding the converted online network node into a Transformer model to capture a pattern of interest change of the user and obtain dynamic interest hidden in a user behavior sequence; The recommendation generation module is used for splicing the dynamic interest and the representation vector of the user to obtain a current preference of the user, and inputting the preference into a multi-layer fully connected neural network with a softmax function to obtain a probability distribution of all candidate interest groups; The contrast learning module is used for strengthening a supervised task through a contrast learning task by taking different views of nodes of different social networks; The online interest group is recommended to a user through an interest group recommendation model, and the interest group recommendation model is composed of a node embedding module, a sequence encoding module, a recommendation generation module and a contrast learning module; Step 1: three social networks of an online network, an offline network and an integrated network are constructed according to different views of social relations of a user, all node information of the user in the online network is obtained, and the node information is input into the node embedding module, each node of the online network is converted into a vector by the node embedding module, and feature and structure information of the node are encoded to obtain a representation vector of the user and an online interest group, and the representation vector includes the following contents: Let online network be represented as G on = <U, G, A on >, where U represents a user set, U = {u1, u2,..., u n}; G represents an online interest set, G = {g1, g2,..., g m}; E represents an event set, E = {e1, e2,..., e k}; and A on represents an edge set, that is, if a user participates in an online interest group, there is an edge connected between the user and the interest group; Let the embedding vector of user U be Let the embedding vector of interest group G be where d is the vector dimension, and the superscript (0) denotes the initial vector. Since the interest groups participated by the user can reflect the interest and hobby of the user, and the users participating in the interest groups can be used as features of the interest groups to measure the similarity between two interest groups, information propagation is performed between connected users and interest groups by using this feature, and a representation vector of a user node u after a first layer GCN is obtained by aggregating neighbors of the user node u using an aggregation function through a graph convolution network (GCN): In the above formula, denotes the representation vector of the user behind the first layer GCN, N(u) is the set of first-order neighbors of user U and the user itself, d u is the degree of user node u plus 1, d g is the degree of node g, is the weight matrix for extracting information, d' is the dimension after transformation, is the bias, and σ(·) is the activation function; Similarly, a representation vector of an interest group node g after a first layer GCN is obtained by aggregating neighbors of the interest group node g using an aggregation function through a graph convolution network (GCN); Since high-order correlation information has an important influence on evaluating the correlation between the user and the interest group, it is necessary to stack multiple GCN layers to obtain a final representation vector of the user u and the interest group g; Suppose that l GCN layers are stacked, so that a user or an interest group can receive information propagated from l-hop neighbors; and an iterative calculation formula of the representation of the user u at the lth hop step is as follows: wherein, is a trainable weight matrix, d l ,d l-1 are the transformation dimensions of the lth and (l−1)th layers, respectively, is the interest group representation of the (l−1)th layer output; Similarly, the representation vector of the interest group g of the lth GCN layer is calculated By stacking multiple GCN layers, and taking the output of the last GCN layer as the final representation vector of user u and interest group g, respectively denoted as Step 2: the representation vector of the user and the online interest group obtained after the node conversion is embedded into the sequence encoding module to capture a pattern of interest change of the user and obtain dynamic interest hidden in a user behavior sequence, and the representation vector includes the following contents: Let S u = [g u,1 ,...,g u,i ,...,g u,t ] represent the online interaction interest group sequence of user u, in order to utilize the order information of the input sequence, the position vector is added to the interest group representation vector by the position encoding method; then the interest group representation vector is calculated for each position in a Transformer layer by the Transformer encoding model, and the interest group representation vectors of all positions are stacked into a matrix for calculating the attention weight of the user to the interest group, obtaining the dynamic interest hidden in the user behavior sequence; interest group representation vector stack matrix for each layer The calculation formula is as follows: B (l-1) = LN(G (l-1) + Dropout(MH(G (l-1) ))) (3) Trm(G (l-1) ) = LN(B (l-1) + Dropout(PFFN(B (l-1) )) (4) Wherein, d is a transformation dimension, l represents the lth layer, LN(·), Dropout(·) and MH(·) represent layer normalization, dropout and multi-head attention operation respectively, and PFFN(·) represents a point forward feed network. Step 3, the dynamic interest is spliced with the user representation vector by the recommendation generation module to obtain the current preference of the user, and the preference is input into a multi-layer fully connected neural network with a softmax function to obtain the probability distribution of all candidate interest groups, and then the corresponding online interest group is recommended to the user.

2. The online interest group recommendation method based on self-attention and contrastive learning according to claim 1, characterized in that, In step 3, the dynamic interest is spliced with the user representation vector by the recommendation generation module to obtain the current preference of the user, and the preference is input into a multi-layer fully connected neural network with a softmax function to obtain the probability distribution of all candidate interest groups, and then the corresponding online interest group is recommended to the user, the specific process is as follows: In order to learn more complex transition patterns, multiple Transformer layers are usually stacked, and the last output matrix of all interest groups of the user behavior sequence after l layers of Transformer is Assuming the interest group g t masked, the interest group g at step t is characterized by t the predicted value; considering only the dynamic interest preference of user u, without considering the feature input of the user itself, such as the ID, age, gender of the user, the user representation is further concatenated with and the user representation to obtain the final representation of the user's preference at time step t The user preference representation is input into a K-layer fully connected network with ReLU activation function to generate a probability distribution of the user u's interest in each candidate interest group wherein, is a parameter of the scientific department, d is the dimension of the embedding vector; According to the generated probability distribution of the user u's interest in each candidate interest group, the corresponding online interest group is recommended to the user.

3. The online interest group recommendation method based on self-attention and contrastive learning according to claim 2, characterized in that, In step 1, the self-supervised signal is derived from the original user behavior sequence by contrastive learning, and by taking the online network, offline network and integrated network as three different views of user social relationship, each view captures unique information of social relationship, the same instances of different views are taken as positive sample pairs, and different instances of different views are taken as negative sample pairs, then InfoNCE loss is used to maximize the consistency between positive sample pairs and minimize the consistency between negative sample pairs, all views cooperate with each other to maximize the model capability, the specific process is as follows: Considering the contrast between the online network view and the offline network view, since only the user nodes appear in both networks, only the contrast loss L between the user embeddings is calculated on-off : wherein, for computing the loss of a user positive sample pair ​ where sim(·) is a similarity function that measures the similarity between two vectors, is the online network representation vector of user u, is the negative sample representation vector in the offline network view; Considering the contrast between online network view and integrated network view, the loss between positive user pairs and positive interest group pairs is calculated respectively, and then the two kinds of losses are fused to obtain the contrast loss L between the two views on-int : Wherein, the hyperparameter λ1 is used to control the strength of the contrast effect of the interest group representation vector; Similarly, considering the contrast between the offline network view and the integrated network view, the contrast loss between positive user pairs and positive event pairs is calculated respectively, and the final fused loss L is obtained off-int : Wherein, λ2 is a hyperparameter; The contrast loss under the above different views is fused to obtain the objective function of the self-supervised learning task: L c = L on-off + β1L on-int + β2L off-int . (14) Wherein, β1, β2 are hyperparameters.

4. The online interest group recommendation method based on self-attention and contrastive learning according to claim 3, characterized in that, Since a bidirectional Transformer model is used to predict the next interest group that the user wants to join, based on the user's interaction sequence, the Cloze task is applied to sequence recommendation, the specific process is as follows: For each input sequence, a number of interest groups "[mask]" in the input sequence are randomly masked, and then the original ID of the masked interest group is predicted based only on its left and right context; The corresponding latent vector marked by "[mask]" is input into a softmax function acting on the interest group set to obtain the preference probability of the masked interest group, and the final objective function is defined as follows: where S u is a masked version of the interest group sequence S u ; represents the set of masked interest groups; is the true interest group of the masked interest group; P(·) is the probability defined in equation (8); To improve the recommendation performance, a multi-task training strategy is adopted in step 3 to jointly train the classic recommendation task and the self-supervised learning task: L = L s + pL c (16) Wherein, ρ is a hyperparameter that controls the strength of the self-supervised learning task.

Images