An information recommendation method in a dangerous scene based on multi-behavior multi-view

Abstract

This invention discloses an information recommendation method for crisis scenarios based on multiple behaviors and multiple views. The specific steps of the method are as follows: (1) Preprocess the data according to the characteristics of the crisis scenario and obtain a recommendation candidate pool; (2) Use user vectors, item vectors, and item text content as input to the model; (3) Obtain the initial embedding of the user and the initial embedding of the item through the embedding layer; (4) Use the multiple views to learn the representation of the user's multiple behaviors through the candidate generation layer; (5) Update the model parameters according to the results and loss, and continuously train until the convergence stopping condition is reached; (6) Obtain the TOP N recommendation list through the ranking layer; (7) Save the model, load the model into the new data, and obtain the recommendation list. This invention improves the accuracy of the information recommendation model in crisis scenarios by fully combining multiple behaviors, multiple views, etc. with the characteristics of crisis scenarios, thereby improving the recommendation task effect in this special scenario.

Description

Technical Field

[0001] This invention relates to the field of information recommendation technology, and more specifically, to an information recommendation method for crisis scenarios based on multiple behaviors and multiple views. Background Technology

[0002] Personalized recommendation systems have become a widely used service in today's society. In many fields such as e-commerce, entertainment, news, recruitment platforms, and social networks, recommendation systems have become fully integrated into our lives. With the development of deep learning technology, recommendation algorithms have made significant progress in sequence recommendation, graph neural network recommendation, and multi-behavior recommendation, significantly improving recommendation performance compared to traditional methods such as collaborative filtering and matrix factorization.

[0003] However, crisis scenarios are characterized by timeliness and urgency. Existing recommendation algorithms do not model these characteristics. Furthermore, when making recommendations based on multiple user behaviors (such as clicks, favorites, comments, etc.), the core issue is to jointly consider these behaviors to improve the performance of the target behavior. Information features are particularly important in crisis scenarios, but current algorithms cannot utilize multi-behavior differentiation for attribute feature development, thus failing to fully extract the various accompanying information. Summary of the Invention

[0004] This invention proposes an information recommendation method for crisis scenarios based on multiple behaviors and multiple views. It models the timeliness and urgency of crisis scenarios, enabling the model to adapt to the situation. Furthermore, it utilizes user behaviors on information to construct sequence-based and graph-based representation learning methods, incorporating information features to allow the model to learn individual and global preferences, thereby improving recommendation accuracy from a multi-dimensional perspective. Finally, an algorithm is used to re-rank the recommendation results, further enhancing the recommendation effect.

[0005] The technical solution of the present invention is described in detail below.

[0006] This invention provides an information recommendation method for crisis scenarios based on multiple behaviors and multiple views. Based on an information recommendation model, it fully integrates the characteristics of multiple behaviors, multiple views, and crisis scenarios, learning semantic information from various behaviors and preference features from multiple views to improve the accuracy of information recommendation in crisis scenarios from a multi-dimensional perspective. The information recommendation model includes an embedding layer, a candidate generation layer, and a ranking layer. The embedding layer is used to input the initial user embedding and initial item embedding into the entire model. The candidate generation layer provides the TOP N recommendation results, which include a graph view and a sequence view. The ranking layer rearranges the TOP N results from the upper layer to obtain the final recommendation list.

[0007] The specific steps are as follows:

[0008] Step 1: Considering the urgency of the crisis scenario, preprocess the raw data, calculate the average time it takes for a message to be interacted with, and directly delete messages that have not been interacted with beyond this time from the recommendation candidate pool.

[0009] Step 2: Initialize the embedding of sequences and graph views, with the user initially embedding e. u Generated from user ID, the initial item embeddings in the sequence view are concatenated from item feature encodings and item text pre-trained vectors, while the initial item embeddings in the graph view are directly derived from the item text pre-trained vectors; let x v D represents the result of feature encoding of item v, excluding its content. v This represents the representation vector obtained after BERT pre-training of the content of item v. The initial embedding of item v in the sequence view is then... Initial embedding of the diagram view item v It can be represented as;

[0010]

[0011]

[0012]

[0013] Step 3: Input the initial user embedding and initial item embedding into both sequence and graph views for representation learning. In the sequence view, different sequences of different behaviors at the individual user level are encoded using a sequence encoder (SeqEncoder) to obtain a single representation u of the user under behavior b. s,b Then, the individual user representations are merged to obtain the user representation u in the integrated sequence view. s , where α u,b This represents the weight of user u under behavior b. An undirected user-item-attribute graph based on multiple behaviors is constructed in the graph view. User embedding propagation, item embedding propagation, and multi-behavior-based item-item propagation are used for learning to obtain the user representation u in the sequence view. g and project representation v g Furthermore, considering the time-sensitive nature of crisis scenarios, a time decay function is incorporated into the initial embedding of items in the sequence and during graph propagation to determine time weights. Item representation in the sequence view. Initial embedding of sequence view item v Scalar multiplication with a time decay function. In a graph view, edge weights are set to the time decay function during propagation.

[0014]

[0015]

[0016]

[0017] Step 4: Concatenate the results of Step 3 to obtain the fused user representation u and item representation v, where || represents the vector concatenation operation;

[0018] u=MLP U (u s ||u g )

[0019]

[0020] Step 5: Calculate user-based scores and item-based scores using the fused user representation and items respectively, fuse the two scores to obtain the final score, and use the final score to construct an auxiliary loss;

[0021] Step 6: Construct three contrastive learning tasks: multi-behavior contrastive learning, multi-view contrastive learning, and behavior discrimination contrastive learning, and construct different auxiliary losses; among them: the goal of multi-behavior contrastive learning is to learn the differences between different user behaviors; the goal of behavior discrimination contrastive learning is to minimize the differences between different single behavior representations of the same user and maximize the differences between different users; the goal of multi-view contrastive learning is to make the sequence view and graph view user representations of the same user more similar to those of other users.

[0022] Step 7: Integrate the auxiliary losses from Steps 5 and 6 into a total loss for learning;

[0023] Step 8: Based on the loss from Step 7, switch hyperparameters and repeat the training until convergence.

[0024] Step 9: Input the first M results given in Step 8 into the sorting layer, use the sorting algorithm to re-sort them, obtain the re-sorted recommendation list, and save the model;

[0025] Step 10: For new data, process it using the method in Step 1, load the model from Step 9, and obtain the TOPN recommendation list.

[0026] In this invention, step 5, the calculation method based on the user's score y1, is as follows:

[0027] y1(u,i)=u T ·v

[0028] The score y2 is calculated based on the project score as follows, where M b It is a trainable matrix based on behavior b that measures the correlation between two items, m. i,b This represents the vector obtained by aggregating item v (based on multi-behavior item-item propagation learning) under behavior b in a graph view:

[0029]

[0030] Combining the two through the hyperparameter λ, the final score is as follows:

[0031] y(u,i)=λ·y1(u,i)+(1-λ)·y2(u,i)

[0032] An auxiliary loss is constructed based on the final score, where O = {(u,i,j)|(u,i)∈R} + ,(u,j)∈R -} represents the training set, R + R represents the positive set of target behaviors in the training set. - This represents the negative set sampled from the training set. β and θ represent the regularization weights and trainable parameters of the model, respectively. σ is the sigmoid function. L2 regularization is used to prevent overfitting.

[0033]

[0034] In step 6, consider a system containing N users {u1, u2, ..., u...} N Small batches, for each user u i Representation of two single actions, b1 and b2, for randomly selecting actions. and As the opposite of comparative learning, and As negative pairs, projection maps all single user behavior representations to the same semantic space, resulting in... Therefore, L SeqCL and L GraphCL Let the auxiliary losses based on multi-view contrastive learning in the sequence and graph views be represented respectively, and defined as:

[0035]

[0036]

[0037]

[0038] Will and Consider it the opposite of comparative learning, and Consider them as negative pairs. Similarly, by projecting... Therefore, the loss of multi-view contrast learning is denoted as:

[0039]

[0040] Let the target behavior be v. i The auxiliary behavior item is v j Other random negative terms within the batch are v k The user indicates that u should first approach v. i Then v j Finally, with the random negative term v k Phase difference. Similarly, using MLP p4 Projecting, we get u p4 v i,p4 v j,p4 and v k,p4 Therefore, the loss from behavioral differentiation and contrast learning is denoted as:

[0041]

[0042] In step 7, the total loss is , where λ represents the parameter for adjusting the weights:

[0043] L=λ o L o +λ1L SeqCL +λ2L GraphCL +λ3L ViewCL +λ4L DisCL .

[0044] In this invention, in step 9, the sorting algorithm adopts the Determinantal Point Process (DPP)[1].

[0045] Compared with the prior art, the beneficial effects of the present invention are as follows: the present invention improves the accuracy of information recommendation models in crisis scenarios by fully combining multiple behaviors, multiple views and other characteristics with the crisis scenario, thereby improving the recommendation task effect in this special scenario. Attached Figure Description

[0046] Figure 1 This is a simplified flowchart of the information recommendation algorithm for crisis scenarios based on multiple behaviors and multiple views, as described in this invention.

[0047] Figure 2 This is a flowchart of the information recommendation algorithm for crisis scenarios based on multiple behaviors and multiple views, as presented in this invention.

[0048] Figure 3 This is a diagram of the information recommendation model architecture for crisis scenarios in this invention.

[0049] Figure 4 This is a schematic diagram of the auxiliary loss of the present invention. Detailed Implementation

[0050] The method of the present invention will be further described below with reference to the accompanying drawings and the description of the invention.

[0051] Figure 1 , Figure 2 This is a flowchart of the algorithm of the present invention.

[0052] Figure 3 The following is an architecture diagram of the information recommendation model in the crisis scenario of the present invention. The model consists of three parts: an embedding layer, a candidate generation layer, and a ranking layer. The embedding layer is used to input the initial user embedding and the initial item embedding into the entire model. The candidate generation layer consists of two parts: a graph and a sequence view. Its function is to provide the TOP N recommendation results. The sequence view part can use Bert4rec[2] to encode the historical sequences of different user behaviors, and then fuse the single behavior representation of the user in the sequence view to obtain the user representation in the integrated sequence view. The item representation in the sequence is obtained by multiplying the item embeddings by the time decay function. The graph view learns the user representation and item representation in the graph view by establishing an undirected graph of "user-item-attribute". Finally, the user representation and item representation learned by the two views are fused, and the fusion result is used to score and calculate the ranking. The ranking layer directly uses the DPP algorithm to rearrange the TOP N results of the upper layer to obtain the final recommendation list.

[0053] Figure 4 This is a schematic diagram of the auxiliary loss of the present invention.

[0054] Here, we first state the problem. Assume the system has B = {b1, b2, b3, ..., b}. B The behavior type of user u i In other words, its historical behavior sequence is represented as in This represents the user's historical sequence under behavior b. Furthermore, let the user set and item set be U and V respectively, and the item attribute set be A. The problem solved by this invention can then be defined as: in a crisis scenario, given U, V, and A, and the user's historical behavior sequence S, output the estimated probability of the user interacting with an item under the target behavior.

[0055] This invention provides an information recommendation method for crisis scenarios based on multiple behaviors and multiple views, comprising the following steps:

[0056] Step 1: Considering the urgency of the crisis scenario, preprocess the raw data, calculate the average time it takes for a message to be interacted with, and directly delete messages that have not been interacted with beyond this time from the recommendation candidate pool.

[0057] Step 2: Initialize the embeddings for sequence and graph views. The initial user embedding is generated from the user ID. The initial item embedding in the sequence view is formed by concatenating the item feature code and the item text pre-trained vector. The initial item embedding in the graph view is directly formed by the item text pre-trained vector.

[0058]

[0059]

[0060]

[0061] Step 3: Input the initial user embedding and the initial item embedding into two views, sequence and graph, for representation learning. In the sequence view, different sequences are built for different behaviors and learned. In the graph view, an undirected graph based on multiple behaviors, user-item-attribute, is built for learning. Considering the timeliness of crisis scenarios, a time decay function is introduced for each item to determine the time weight.

[0062] Step 4: Combine the results from Step 3 to obtain the merged user representation and project representation;

[0063] u=MLP U (u s ||u g )

[0064]

[0065] Step 5: Calculate user-based scores and item-based scores using the fused user representation and items respectively, fuse the two scores to obtain the final score, and use the final score to construct an auxiliary loss;

[0066] y1(u,i)=u T ·v

[0067]

[0068] y(u,i)=λ·y1(u,i)+(1-λ)·y2(u,i)

[0069]

[0070] Step 6: Construct three contrastive learning tasks: multi-action contrastive learning, multi-view contrastive learning, and action discrimination contrastive learning, and construct different auxiliary losses;

[0071]

[0072]

[0073]

[0074]

[0075] Step 7: Integrate the auxiliary losses from Steps 5 and 6 into a total loss for learning;

[0076] L=λ o L o +λ1L SeqCL +λ2L GraphCL +λ3L ViewCL +λ4L DisCL .

[0077] Step 8: Based on the loss from Step 7, switch hyperparameters and repeat the training until convergence.

[0078] Step 9: Input the first M results given in Step 8 into the ranking layer, use the DPP algorithm to re-rank them, obtain the re-ranked recommendation list, and save the model;

[0079] Step 10: For new data, process it using the method in Step 1, load the model from Step 9, and obtain the TOPN recommendation list.

[0080] Specifically, to ensure the urgency of crisis scenarios (i.e., projects that have not been interacted with for a long time have lower recommendation value), after calculating the average interaction time of projects in step 1, projects that have not been interacted for more than that time are directly removed from the recommendation pool, and the final recommendation candidate pool is constructed.

[0081] Next is the model's embedding layer. The initial user embedding is generated by encoding the user ID. The initial item embedding is different, with two types: sequence and graph. In the sequence view, the initial item embedding is formed by concatenating the item feature encoding with the item text pre-trained vector (the text pre-training model can be flexibly selected, such as the BERT model). In the graph view, since the item features will be input into the graph, the item text pre-trained vector is directly used as the initial item embedding.

[0082] Next is the candidate generation layer of the model. For the sequence view and graph view in step 3, representation learning is performed based on multiple behaviors, which can simultaneously obtain the user's preferences from different local and global perspectives. For the sequence view, an encoder is used to encode different behavior sequences separately (the encoder can be flexibly selected, such as Bert4Rec, etc.), and then fused based on the user's weight under that behavior. For the graph view, user embedding propagation, item embedding propagation, and item-to-item propagation based on multiple behaviors are performed. At the same time, considering the timeliness of crisis scenarios (i.e., information created a long time ago has little value for recommendation), a time decay function is added to the initial item embedding in the sequence and the graph propagation process.

[0083] Since different views can express different user preferences, step 4 obtains a merged user representation and item representation through the splicing operation, which can learn a better representation from multiple dimensions;

[0084] Step 5 uses the results from Step 4 to calculate the score, which includes user-based scores and item-based scores, and combines the two to obtain the final score;

[0085] To better learn the commonalities and differences between different views and behaviors, step 6 constructs three contrastive learning tasks, including multi-behavior contrastive learning (the goal is to minimize the differences between different single behavior representations of the same user and maximize the differences between different users), multi-view contrastive learning (the user representations of the sequence view and graph view of the same user should be closer than those of other users), and behavior differentiation contrastive learning (the differences between different behaviors of users are also very important), and constructs an auxiliary loss for contrastive learning.

[0086] Steps 7 and 8 fuse various auxiliary losses by assigning different weights and then train the system.

[0087] After training converges, the top M results are fed into the last layer of the model (i.e., the ranking layer) to be re-sorted, and the TOP N results are used as the final recommendation list. At the same time, the model is saved.

[0088] For new data, after processing it using the method in step 1, load the model to obtain the recommendation results.

[0089] Example 1

[0090] This invention constructs a crisis-related dataset by crawling microblogs related to crises. This dataset comprises three parts: user information (such as user ID, user's microblog nickname, user gender, etc.), microblog information (such as microblog ID, microblog content, microblog posting time, etc.), and user-microblog interaction information (user ID, microblog ID, interaction type). Table 1 shows the performance of this invention on this dataset.

[0091] Table 1

[0092]

[0093]

[0094] Since there is currently no research on information recommendation in crisis scenarios, this invention uses four classic recommendation algorithms and two other deep learning recommendation models as comparison models.

[0095] Popularity-based recommendations: also known as trending recommendations, the most common approach is to recommend highly popular content from the list to users.

[0096] • kNN: The k-nearest neighbor method based on items. Its core idea is to recommend items similar to those that the target user already likes. The similarity between items can be calculated using methods such as the Pearson correlation coefficient.

[0097] • Matrix factorization (MF): The user rating matrix is ​​decomposed into the product of two low-dimensional matrices, thus obtaining the latent vectors of users and items. By calculating the inner product of the user latent vector and the item latent vector, the predicted rating value of the user for the item is obtained.

[0098] FM (Factorization Machine) aims to solve the feature combination problem under sparse data. By crossing features, it increases the nonlinearity and generalization ability of the model, improves the model capacity, and can capture more additional information.

[0099] ·ML: A universal recommendation framework proposed by Google[3]. The ML model adopts a simple architecture, which feeds a bunch of fully connected ReLU layers into a logic layer, and then the logic layer feeds a linear combination of activations from the last hidden layer into a sigmoid unit to output a score in the [0,1] interval.

[0100] ·MMCLR: A multi-behavior recommendation model [4]. To alleviate the sparsity problem of target behaviors, a contrastive learning task is constructed through different views to capture the relationship between multiple behaviors.

[0101] As shown in Table 1, the present invention significantly outperforms all comparative models in all metrics, and achieves an improvement of 3.3% to 31.3% over the best comparative model. This verifies the effectiveness of the present invention.

[0102] Compared to the six contrastive models, this invention performs best, which can be explained in several ways. First, the contrastive models are all methods that cannot distinguish behaviors, which may lead to information loss. This invention is a behavior-discriminating model that can extract user behavior weights and item relevance from multi-behavior data. This also confirms the benefits of exploring multi-behavior models for modeling collaborative effects in recommendation. Second, this invention jointly models multiple views, capturing a variety of information to understand different aspects of user preferences, and the contrastive learning task helps capture the commonalities and differences between different behaviors and views. Finally, this invention benefits from textual information and attribute awareness. The results of text pre-training can introduce auxiliary information into the model, demonstrating the effectiveness of text in the semantic space for modeling potential factors of users and items. Attribute information can provide additional rich information for modeling user preferences and item features. In addition, this invention can adapt to the timeliness and urgency of crisis scenarios, which is also one of the reasons why this invention performs well on this crisis-related dataset.

[0103] References

[0104] [1]Chen L,Zhang G,Zhou E.Fast greedy map inference for determinantalpoint process to improve recommendation diversity[J].Advances in NeuralInformation Processing Systems,2018,31.

[0105] [2]Sun F,Liu J,Wu J,et al.BERT4Rec:Sequential recommendation withbidirectional encoder representations from transformer[C] / / Proceedings of the28th ACM international conference on information and knowledgemanagement.2019:1441-1450.

[0106] [3]Chen S J,Qin Z,Wilson Z,et al.Improving recommendation quality ingoogle drive[C] / / Proceedings of the 26th ACM SIGKDD international conferenceon knowledge discovery&data mining.2020:2900-2908.

[0107] [4]Wu Y,Xie R,Zhu Y,et al.Multi-view multi-behavior contrastivelearning in recommendation[C] / / International Conference on Database Systemsfor Advanced Applications.Cham:Springer International Publishing,2022:166-182。

Claims

1. An information recommendation method for crisis scenarios based on multiple behaviors and multiple views, characterized in that, Based on an information recommendation model, it fully integrates the characteristics of multiple behaviors, multiple views, and crisis scenarios, learns the semantic information in multiple behaviors and the preference features in multiple views, and improves the accuracy of information recommendation in crisis scenarios from multiple dimensions; the information recommendation model includes an embedding layer, a candidate generation layer, and a ranking layer; The embedding layer is used to input the initial user embedding and the initial item embedding for the entire model; The candidate generation layer provides the top N recommendation results, which consists of two parts: a graph view and a sequence view. The ranking layer rearranges the top N results from the upper layer to obtain the final recommendation list. The specific steps are as follows: Step 1: Preprocess the raw data Calculate the average time it takes for a message to be interacted with, and directly remove messages that have not been interacted with beyond that time from the recommendation candidate pool; Step 2: Initialize the embedding of sequence views and graph views. User initial embedding Generated by user ID, the initial item embeddings in the sequence view are concatenated from the item feature encoding and the item text pre-trained vector, while the initial item embeddings in the graph view directly use the item text pre-trained vector. Step 3: Input the initial user embedding and initial item embedding into both sequence and graph views for representation learning. In the sequence view, different sequences of different behaviors at the individual user level are encoded to obtain the user's behavior. The individual user representations are then merged to obtain the integrated user representations in the sequence view. In the graph view, an undirected user-item-attribute graph based on multiple behaviors is constructed. User embedding propagation, item embedding propagation, and multi-behavior-based item-item propagation are used for learning to obtain the user representation in the sequence view. and project representation Meanwhile, considering the time-sensitive nature of crisis scenarios, a time decay function is added to the initial embedding of items in the sequence and the graph propagation process to determine time weights, and the items in the sequence view are represented as follows: The initial embedding of the sequence view items is multiplied by the time decay function; the graph view sets the edge weights to the time decay function during propagation. Step 4: Concatenate the results of Step 3 to obtain the merged user representation u and item representation v. This represents a vector concatenation operation; ； Step 5: Calculate user-based scores and item-based scores using the fused user representation and items respectively, fuse the two scores to obtain the final score, and use the final score to construct an auxiliary loss; Step 6: Construct three contrastive learning tasks: multi-action contrastive learning, multi-view contrastive learning, and action discrimination contrastive learning, and construct different auxiliary losses; The goal of multi-behavior contrastive learning is to learn the differences between different user behaviors. The goal of behavior differentiation contrastive learning is to minimize the differences between different single behavior representations of the same user and maximize the differences between different users. The goal of multi-view contrastive learning is to make the sequence view and graph view user representations of the same user more similar to those of other users. Step 7: Integrate the auxiliary losses from Steps 5 and 6 into a total loss for learning; Step 8: Based on the loss from Step 7, switch hyperparameters and repeat the training until convergence. Step 9: Input the first M results given in Step 8 into the sorting layer, use the sorting algorithm to re-sort them, obtain the re-sorted recommendation list, and save the model; Step 10: For new data, process it using the method in Step 1, load the model from Step 9, and obtain the TOP N recommendation list.

2. The information recommendation method according to claim 1, characterized in that, In step 5, based on the user's score The calculation method is as follows: ； Based on project score The calculation method is as follows: ； in, Based on behavior A trainable matrix that measures the correlation between two projects. This represents the projects learned through multi-behavior project-project propagation in the graph view. In behavior The vector obtained by the lower aggregation: Both are connected via hyperparameters The final scores are as follows: ； An auxiliary loss is constructed based on the final score, where, Represents the training set, This represents the positive set of target behaviors in the training set. This represents the negative set sampled from the training set. and These represent the regularization weights and trainable parameters of the model, respectively. yes function, Regularization is used to prevent overfitting: ； In step 6, consider a containing Small batches for individual users, for each user Random selection behavior and The two single actions represent, for ;Will and As the opposite of comparative learning, and As negative pairs, projection maps all single user behavior representations to the same semantic space, resulting in... ,therefore, and Let the auxiliary losses based on multi-view contrastive learning in the sequence view and graph view be defined as follows: ； ； ； Will and Consider it the opposite of comparative learning, and Treating them as negative pairs, similarly, by projecting... , Therefore, the loss of multi-view contrastive learning is denoted as: ； Let the target behavior be Auxiliary behavior items are Other random negative items within the batch are Then the user indicates We should first get close to Then Finally, with the random negative term Phase difference; similarly, utilizing Projecting, we obtain , , and Therefore, the loss from behavioral differentiation and contrast learning is denoted as: ； In step 7, the total loss is L, where , , These represent the parameters that adjust different loss weights: 。 3. The information recommendation method according to claim 1, characterized in that, In step 9, the sorting algorithm used is the DPP algorithm.

Images