Sequence recommendation method based on uniform interaction frequency of anti-fact method

Abstract

The invention belongs to the technical field of recommendation systems and machine learning, and discloses a sequence recommendation method based on uniform interaction frequency of an anti-fact method, and the method specifically comprises the steps: extracting a user interaction sequence from a public data set; filtering and preprocessing the data; performing homogenization processing on the interaction timestamps, calculating the similarity between interacted items and target items by using cosine similarity, and replacing some items with the lowest score with items with the high score; the long-term interest and the short-term interest of the user are predicted by using Transfromer and Attentive Fism, and personalized sequence recommendation is carried out in combination with the long-term interest and the short-term interest of the user; constructing and training a model; and predicting a target item of the user through the trained model, and selecting items with relatively high scores to generate a recommendation set. According to the method, more accurate long-term interests and short-term interests are obtained by using an anti-fact method, so that the recommendation accuracy is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of recommendation system and machine learning, and in particular to a sequence recommendation method with uniform interaction frequency based on counterfactual method. Background Art

[0002] Sequential recommendation is an important research direction of recommendation system, which aims to provide users with personalized recommendation services by capturing the time dependency and sequentiality of user behavior. Compared with the traditional recommendation method based on static features, sequential recommendation can better model the dynamic changes of users' interests, and is particularly suitable for scenarios where user behavior is constantly changing and timeliness is strong, such as product recommendations on e-commerce platforms, video push on short video platforms, and song recommendations on music platforms.

[0003] With the rapid development of deep learning, sequential recommendation methods have gradually evolved from early rule-based and matrix decomposition methods to modeling using deep learning technology. Models based on recurrent neural networks (RNN) can capture the temporal dependencies in user behavior sequences, while the attention mechanism further enhances the model's ability to perceive key behaviors. In addition, the Transformer model in recent years has shown excellent performance in sequential recommendation tasks due to its powerful parallel processing capabilities and ability to model global dependencies of sequences. These deep learning methods have shown great potential in capturing users' short-term preferences and long-term interests.

[0004] Although deep learning methods have improved the accuracy of recommendation systems, sequential recommendations still face many challenges. For example, bias and noise in the data can affect the learning of the model, resulting in recommendation results that may not match the user's actual needs; the diversity and dynamic changes in user behavior make interest modeling more complex; in addition, the transparency and explainability of the recommendation system is also an important issue, especially in scenarios where explainable recommendation results need to be provided for user or business decisions. Summary of the invention

[0005] The present invention aims to provide a sequence recommendation method with uniform interaction frequency based on counterfactual method to solve the above technical problems.

[0006] In order to solve the above technical problems, the specific technical solution of the sequence recommendation method based on the uniform interaction frequency of the counterfactual method of the present invention is as follows:

[0007] A sequence recommendation method based on uniform interaction frequency of counterfactual method, comprising the following steps:

[0008] Step 1: Obtain a public dataset from the website and then preprocess the public dataset; group the user interaction records by user ID and then sort them by timestamp;

[0009] Step 2: Even out the timestamps of user interactions; generate an arithmetic sequence based on the minimum and maximum values ​​of each user's timestamp and the total number of interactions to replace the original timestamps;

[0010] Step 3: First, all items are embedded, and the similarity between the items that the user has interacted with and the target items is calculated, so that the items that the user has interacted with are divided into key, medium, and discardable items. The discardable items are replaced to obtain an embedded expression of the interaction sequence;

[0011] Step 4: Use the long-term interest encoder and the short-term interest encoder to encode the long-term interest and short-term interest from the embedding expression of the new interaction sequence;

[0012] Step 5: Combining long-term interests and short-term interests to predict target items;

[0013] Step 6: Use the trained model to predict the user's target item, which is better than previous models in many key indicators.

[0014] Furthermore, the step 2 includes the following specific steps:

[0015] Using the counterfactual method, the timestamp of the user interaction is artificially constructed into an arithmetic progression, and the timestamp is input: ({t 1 ,t 2 ,…,t n}) where t n >t n-1 >t n-2 >…>t 1

[0016] Timestamp after counterfactual construction:

[0017] [t′ i =t 1 +(i-1)·d,fori=1,2,…,n,], where Furthermore, the step 3 includes the following specific steps:

[0018] The last item in the interaction sequence is taken as the label, i.e. the ground-truth. The remaining items in the interaction sequence are calculated similarity with the last item, using cosine similarity:

[0019] A=softmax(W 2 tanh(W 1 X T)) T

[0020] p interest =A T φ θ (X,y) where φ θ =cos

[0021] Where W 1 and W 2 is a learnable parameter, X is the item in our interaction sequence, y is the target item, and we get a new interaction sequence;

[0022] The parts with the lowest similarity are considered discardable, and the items with the highest similarity scores are considered key items, and the discardable items are replaced;

[0023] Replacement process:

[0024] (p interest ={p 1 ,p 2 ,…,p n}), where (p 1 ≥p 2 ≥…≥p n )

[0025] The smallest score part of the project ({p n-k+1 ,…,p n})Use the items with the largest scores ({p 1 ,p 2 ,…,p k}):Perform random replacement;

[0026] Where X is the embedding vector representation of the input, which includes the user's interaction sequence embedding vector The I above represents item, and the subscript v 1 ,v 2 ,v L Represents a specific interaction item; introduces time information to embed it as the context of user interaction. Its X=[x 1 ,…,x L ], and each item in the list

[0027] Furthermore, the step 4 includes the following specific steps:

[0028] The timestamp is used as the context and combined with the embedding vector of the sequence item as the input vector to Transform to predict short-term interests. The short-term interests use a multi-head self-attention mechanism.

[0029]

[0030] SAB(X) = FFL(SAL(X)) = ReLU(X Att W 1 +b 1 )W 2 +b 2

[0031]

[0032] In the above formula, X is the input, W 1 ,W 2 ,b 1 b 2 are all learnable parameters, W o is the projection matrix, and A j The Gaussian mixture probability model is used for sampling. It learns the importance of items and their timing information from the sequence, and combines the above information to obtain the expression of short-term interests. This formula is intended to illustrate how relevant the item v is to the user's short-term at the time L+1; where B is the number of attentions, that is, how many times SAL and SAB have been calculated. This formula predicts how relevant the item v is to the user's short-term at the time L+1;

[0033] The Attentive FISM algorithm used to predict long-term interests is as follows:

[0034] First, we randomly select N items from the user interaction sequence and extract their embedding vectors separately. Then update m u The value of m is initialized randomly u , and then update according to the following formula

[0035]

[0036] After updating m u After that, a prediction of the user's long-term interest is made. The formula is as follows: This represents the correlation between item v and the user’s long-term interest at time L+1.

[0037] Furthermore, the step 5 includes the following specific steps:

[0038] The user's last interaction item is predicted by combining the user's long-term interest and short-term interest.

[0039]

[0040] In the above loss function expression, x∈{short,long}, V S represents positive instances, items that interact with users in the dataset, O S Represents negative instances, which are instances in the dataset that do not interact with the user.

[0041] Furthermore, the step 6 includes the following specific steps:

[0042] Through the trained model, the model is evaluated on three public datasets. The evaluation process is to prepare the model to retrieve the next item that the user will interact with from all items. The public datasets come from movie platforms, e-commerce platforms and music platforms to predict the target items.

[0043] The sequence recommendation method based on the counterfactual method with uniform interaction frequency of the present invention has the following advantages: the sequence recommendation method based on the counterfactual method with uniform interaction frequency of the present invention combines both the long-term interests and short-term interests between users and the relationship between the user's interaction frequency and the timestamp, while also taking into account that the items that the user has interacted with can be divided into discardable and key items, and uses deep learning to obtain more accurate user interest expressions, thereby providing more accurate predictions. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 It is a flow chart of a sequence recommendation method with uniform interaction frequency based on a counterfactual method of the present invention. DETAILED DESCRIPTION

[0045] In order to better understand the purpose, structure and function of the present invention, the sequence recommendation method with uniform interaction frequency based on the counterfactual method of the present invention is further described in detail below with reference to the accompanying drawings.

[0046] like Figure 1 As shown, the sequence recommendation method based on the counterfactual method uniform interaction frequency of the present invention comprises the following steps:

[0047] Step 1: Obtain a public dataset from the website and then preprocess the public dataset; group the user interaction records by user ID and then sort them by timestamp.

[0048] Step 2: Even out the timestamps of user interactions; generate an arithmetic sequence based on the minimum and maximum values ​​of each user's timestamp and the total number of interactions to replace the original timestamps.

[0049] Using the counterfactual method, the timestamp of user interaction is artificially constructed into an arithmetic progression. Input timestamp: ({t 1 ,t 2 ,…,tn}) where t n >t n-1 >t n-2 >…>t 1

[0050] Timestamp after counterfactual construction:

[0051] [t′ i =t 1 +(i-1)·d,fori=1,2,…,n,], where

[0052] Step 3: First, all items are embedded and the similarity between items that the user has interacted with and the target items is calculated, so that the items that the user has interacted with are divided into key, medium, and discardable items. The discardable items are replaced to obtain an embedded expression of the interaction sequence.

[0053] The last item in the interaction sequence is taken as the label, i.e. the ground-truth. The remaining items in the interaction sequence are calculated similarity with the last item. In this example, the cosine similarity is used:

[0054] A=softmax(W 2 tanh(W 1 X T )) T

[0055] p interest =A T φ θ (X,y) where φ θ =cos

[0056] Where W 1 and W 2 is a learnable parameter, X is the item in our interaction sequence, and y is the target item we think. We get a new interaction sequence.

[0057] The parts with the lowest similarity are considered discardable, and the items with the highest similarity scores are considered key items, and the discardable items are replaced.

[0058] Replacement process:

[0059] (p interest ={p 1 ,p 2 ,…,p n}), where (p 1 ≥p 2 ≥…≥p n )(Sort in descending order)

[0060] The smallest score part of the project ({p n-k+1 ,…,p n})Use the items with the largest scores ({p 1 ,p 2 ,…,p k}): Perform random replacement.

[0061] Where X is the embedding vector representation of the input, which mainly includes the user's interaction sequence embedding vector The I above represents item, and the subscript v 1 ,v 2 ,v L Represents the items of specific interactions. In addition, previous models often ignore the replication dependency between user interests and time. In this regard, our model introduces time information as the context of user interaction and embeds it. Its X=[x 1 ,…,x L ], and each item in the list

[0062] Step 4: Use the long-term interest encoder and short-term interest encoder to encode the long-term interest and short-term interest from the embedding representation of the new interaction sequence.

[0063] In the short term, user interests are greatly affected by context, so we use the timestamp as context and the embedding vector of the sequence item as the input vector to Transform to predict short-term interests. Among them, short-term interests mainly use the multi-head self-attention mechanism. Transformer has a multi-head attention mechanism, which can handle time series information well.

[0064]

[0065] SAB(X) = FFL(SAL(X)) = ReLU(X Att w 1 +b 1 ) 2 +b 2

[0066]

[0067] In the above formula, X is our input, W 1 ,W 2 ,b 1 b 2 are all learnable parameters.

[0068] W o is the projection matrix. A jWe use a Gaussian mixture probability model for sampling, which can learn the importance of items and their timing information from the sequence. Combining the above information, we can get the expression of short-term interest This formula is intended to illustrate how relevant item v is to the user in the short term at time L+1. Where B is the number of attentions, that is, how many times SAL and SAB have been calculated. This formula predicts how relevant item v is to the user in the short term at time L+1.

[0069] The Attentive FISM (Attention Item Similarity Matrix Decomposition) is mainly used to predict long-term interests. Previous models often use fixed vectors to represent long-term interests, while the method we use flexibly changes according to the representation of the current item. The specific algorithm process is as follows: First, we randomly select N items from the user interaction sequence, extract their embedding vectors separately, and use Then update m u The value of m is initialized randomly u , and then update according to the following formula

[0070]

[0071] After updating m u After that, we need to make a prediction about the user's long-term interests. The formula is as follows: This represents the correlation between item v and the user’s long-term interest at time L+1.

[0072] Step 5: Combine long-term interests and short-term interests to predict target items.

[0073] We combine the user's long-term interests and short-term interests to predict the item of the user's last user interaction.

[0074]

[0075] In the above loss function expression, x∈{short,long}, V S represents positive instances, items that interact with users in the dataset, O S Represents negative instances, which are instances in the dataset that do not interact with the user.

[0076] Step 6: Use the trained model to predict the user's target item, which is better than previous models in many key indicators.

[0077] We evaluated the trained model on three public datasets. The evaluation process mainly requires the model to be able to retrieve the next item that the user will interact with from all items. The public datasets come from movie platforms, e-commerce platforms, and music platforms. The datasets are divided into 60%, 20%, and 20% training sets, test sets, and validation sets to predict the target items. The model is superior to previous models in some key indicators of evaluation accuracy.

[0078] Example:

[0079] like Figure 1 As shown, a sequence recommendation method based on uniform interaction frequency of counterfactual method includes the following steps:

[0080] Step 1: Obtain the public dataset from the website and then preprocess the public dataset;

[0081] The interaction records in the data set are grouped by user_id, and then some users with too many or too few interaction records are filtered out. The remaining interaction records are sorted by timestamp.

[0082] Table 1 User-Item interaction record table

[0083] User_id Item_id Timestamp 1 1 978300019 1 13 978300055 1 15 978300172 1 22 978300275 2 5 978300719 2 7 978301368 2 10 978301398 2 11 978301570

[0084] Step 2: Even out the interaction timestamps of the user interaction items.

[0085] Step 2.1: Take the minimum and maximum values ​​of the timestamps of the user interaction items and the length of the interaction sequence as three parameters, generate an arithmetic sequence of the same length as the interaction sequence, and then replace the timestamp of the original interaction sequence.

[0086] Table 2 User-Item interaction records with timestamps

[0087]

[0088]

[0089] Step 3: Perform the second counterfactual operation, that is, assume that the user interacts with certain items and does not interact with certain items.

[0090] The following steps are involved:

[0091] Step 3.1: Take the last item in the interaction sequence as the target item. Use the other items in the interaction sequence to calculate the cosine similarity with it. The k items with the lowest similarity are considered to be discardable and replaced with the k key items with the lowest similarity.

[0092] A=softmax(W 2 tanh(W 1 X T )) T

[0093] p interest =A T φ θ (X,y) where φ θ =cos

[0094] Replacement process:

[0095] (p interest ={p 1 ,p 2 ,…,p n}), where (p 1 ≥p 2 ≥…≥p n )(Sort in descending order)

[0096] The smallest score part of the project ({p n-k+1 ,…,p n}): Use the items with the largest scores ({p 1 ,p 2 ,…,p k}): Perform random replacement.

[0097] Where W 1 and W 2 is a learnable parameter, X is the item in our interaction sequence, and y is the target item we think. We get a new interaction sequence.

[0098] Step 4: Use Transformer and Attentive FISM (Attention Item Similarity Matrix Factorization) to predict users’ short-term and long-term interests.

[0099] In the short term, user interests are greatly affected by context, so we use the timestamp as context and the embedding vector of the sequence item as the input vector to Transform to predict short-term interests.

[0100] Transformer has a multi-head attention mechanism that can handle temporal information well.

[0101]

[0102] SAB(X) = FFL(SAL(X)) = ReLU(X Att W 1 +b 1 )W 2 +b 2

[0103] In the above formula, X is our input, W 1 ,W 2 ,b 1 b 2 are all learnable parameters.

[0104] W o is the projection matrix. A j We use a Gaussian mixture probability model for sampling, which can learn the importance of items and their timing information from the sequence. Combining the above information, we can get the expression of short-term interest This formula is intended to illustrate how relevant the item v is to the user in the short term at time L+1. Where B is the number of times it is viewed, that is, how many times SAL and SAB have been calculated.

[0105] In the past, modules that predict long-term interests often predict long-term interests as a fixed vector, which cannot reflect long-term diversity. For this reason, we use FISM, which can make long-term interests related to specific items.

[0106] The specific algorithm process is as follows: First, we randomly select N items from the user interaction sequence and extract their embedding vectors separately. Then update m u The value of m is initialized randomly u , and then update according to the following formula

[0107]

[0108] After updating m u After that, we need to make a prediction about the user's long-term interests. The formula is as follows: This represents the correlation between item v and the user’s long-term interest at time L+1.

[0109] Step 5: Build and train the recommendation model, including the following steps:

[0110] Step 5.1: For the data set, partitioning is adopted, and the leave-one-out method is adopted, that is, for each interaction sequence of length n in the data, the first n-2 items are taken as the training set, the n-1th as the validation set, and the last one as the test set.

[0111] Step 5.2: Combine long-term interests and short-term interests together and define a cross-entropy-based loss function:

[0112]

[0113] In the above loss function expression, x∈{short,long}, V S represents positive instances, items that interact with users in the dataset, O S Represents negative instances, which are instances in the dataset that do not interact with the user.

[0114] Step 5.3: The number of iterations is increased by 1. When the loss function J is less than the preset value or the number of iterations reaches the maximum number of iterations, the convergence condition is met and the process goes to step 5.4. Otherwise, the long-term and short-term interests of the user are continuously predicted from the loss function J, and the loss function is trained using the gradient descent method.

[0115] Step 5.5: Output the converged long-term interests and short-term interests of the user, and the training process ends.

[0116] Step 6: Test the trained model on the test set. The test is to retrieve the user's target item from all items based on the user's long-term and short-term interests. Calculate some key indicators on the test set to effectively optimize some past models.

[0117] experiment:

[0118] 1. Dataset

[0119] The present invention uses three real-world datasets, Movie lens, Last-FM, and TMALL, to verify the recommendation performance. Movie lens is a commonly used benchmark dataset, which contains the user's rating history for various movies and the corresponding timestamps. Last-FM includes the types of music that users listen to in different time periods, and uses various music types as recommendation targets. TMALL contains users, merchants, products, and corresponding timestamps, which mainly reflect when the user purchased the corresponding product. The statistical information of the three datasets is shown in Table 2, and it can be seen that these datasets are essentially very sparse.

[0120] Table 2 Statistics of the dataset

[0121]

[0122] 2. Evaluation criteria

[0123] To make the experimental results more convincing, the leave-one-out method is used, that is, for each user interaction sequence of length n in the data, the first n-2 items are taken as the training set, the n-1th as the validation set, and the last one as the test set. For the final test results, the model is re-verified by selecting 5 different random values, and the final experimental result is the average of the 5 results.

[0124] The evaluation indicators used in this invention are hit rate, normalized discounted cumulative gain, and mean reciprocal rank as evaluation criteria. Similar to most recommendation systems, the candidate items are sorted by score, and the top N items are recommended. For each user, P@N and F1@N are defined as:

[0125]

[0126] in r uj is the relevance score of user u to item j calculated by the model, r′ uj is the relevance score of user u to item j in the actual data

[0127]

[0128] 3. Comparison methods and experimental results

[0129] The experiment selected 8 existing recommendation methods for comparison with this method, including FPMC, BERT4Rec, CORE, GRU4Rec, NextItNet, SASRec, SINE, and FEARec. For the Movie lens, Last-FM, and TMALL datasets, the hyperparameters set by this method are the parameters that achieve the best state, and other methods use the default parameters in the original literature. The experimental results of the 9 methods are compared in Table 3, where the underlined results are the best and suboptimal results in the baseline.

[0130] Table 3 Experimental results of different methods on three datasets

[0131]

[0132] The experimental results show that the hit rate, normalized discounted cumulative gain and average reciprocal ranking of the method of the present invention are superior to other methods in most cases under different N values. It can be proved that the sequence recommendation method based on the counterfactual method uniform interaction frequency proposed by the present invention can better optimize the implicit variables of users and items, improve the reliability of prediction and the accuracy of the recommendation system.

[0133] It is to be understood that the present invention is described by some embodiments, and it is known to those skilled in the art that various changes or equivalent substitutions may be made to these features and embodiments without departing from the spirit and scope of the present invention. In addition, under the teachings of the present invention, these features and embodiments may be modified to adapt to specific circumstances and materials without departing from the spirit and scope of the present invention. Therefore, the present invention is not limited by the specific embodiments disclosed herein, and all embodiments falling within the scope of the claims of this application are within the scope of protection of the present invention.

Claims

1. A sequence recommendation method based on uniform interaction frequency of counterfactual method, characterized in that: The steps include: Step 1: Obtain a public dataset from the website and then preprocess the public dataset; group the user interaction records by user ID and then sort them by timestamp; Step 2: Even out the timestamps of user interactions; Based on the minimum and maximum values ​​of each user's timestamp and the total number of interactions, an arithmetic sequence is generated to replace the original timestamp. Step 3: First, all items are embedded, and the similarity between the items that the user has interacted with and the target items is calculated, so that the items that the user has interacted with are divided into key, medium, and discardable items. The discardable items are replaced to obtain an embedded expression of the interaction sequence; Step 4: Use the long-term interest encoder and the short-term interest encoder to encode the long-term interest and short-term interest from the embedding expression of the new interaction sequence; Step 5: Combining long-term interests and short-term interests to predict target items; Step 6: Use the trained model to predict the user's target item, which is better than previous models in many key indicators.

2. The sequence recommendation method based on counterfactual method uniform interaction frequency according to claim 1, characterized in that: The step 2 comprises the following specific steps: Using the counterfactual method, the timestamp of the user interaction is artificially constructed into an arithmetic progression, and the timestamp is input: ({t1,t2,…,t n }) where t n >t n-1 >t n-2 >…>t1 Timestamp after counterfactual construction: [t′ i = t1 + (i-1) · d, for i = 1, 2, …, n,], where 3. The sequence recommendation method based on uniform interaction frequency of counterfactual method according to claim 1, characterized in that: The step 3 comprises the following specific steps: The last item in the interaction sequence is taken as the label, i.e. the ground-truth. The remaining items in the interaction sequence are calculated similarity with the last item, using cosine similarity: A=softmax(W2tanh(W1X T )) T p interest =A T f θ (X,y) among themf θ =cos Where W1 and W2 are learnable parameters, X is the item in our interaction sequence, y is the target item, and we get a new interaction sequence; The parts with the lowest similarity are considered discardable, and the items with the highest similarity scores are considered key items, and the discardable items are replaced; Replacement process: (p interest ={p1,p2,…,p n }), where (p1≥p2≥…≥p n ) The smallest score part of the project ({p n-k+1 ,…,p n }) Use the items with the largest scores ({p1,p2,…,p k }):Perform random replacement; Where X is the embedding vector representation of the input, which includes the user's interaction sequence embedding vector The I above represents item, and the subscripts v1, v2, v L Represents a specific interaction item; introduces time information to embed it as the context of user interaction. Where X=[x1,…,x L ], and each item in the list 4. The method for sequential recommendation based on uniform interaction frequency of counterfactual method according to claim 1, characterized in that: The step 4 comprises the following specific steps: The timestamp is used as the context and combined with the embedding vector of the sequence item as the input vector to Transform to predict short-term interests. The short-term interests use a multi-head self-attention mechanism. SAB(X)=FFL(SAL(X))=ReLU(X Att W1+b1)W2+b2 In the above formula, X is the input, W1, W2, b1b2 are all learnable parameters, and W o is the projection matrix, and A j The Gaussian mixture probability model is used for sampling. It learns the importance of items and their timing information from the sequence, and combines the above information to obtain the expression of short-term interests. This formula is intended to illustrate how relevant the item v is to the user's short-term at the time L+1; where B is the number of attentions, that is, how many times SAL and SAB have been calculated. This formula predicts how relevant the item v is to the user's short-term at the time L+1; The Attentive FISM algorithm used to predict long-term interests is as follows: First, we randomly select N items from the user interaction sequence and extract their embedding vectors separately. Then update m u The value of m is initialized randomly u , and then update according to the following formula After updating m u After that, a prediction of the user's long-term interest is made. The formula is as follows: This represents the correlation between item v and the user’s long-term interest at time L+1.

5. The sequence recommendation method based on counterfactual method uniform interaction frequency according to claim 1, characterized in that: The step 5 comprises the following specific steps: The item of the user's last user interaction is predicted by combining the user's long-term interest and short-term interest. In the above loss function expression, x∈{short,long}, V S represents positive instances, items that interact with users in the dataset, O S Represents negative instances, which are instances in the dataset that do not interact with the user.

6. The method for sequential recommendation based on uniform interaction frequency of counterfactual method according to claim 1, characterized in that: The step 6 comprises the following specific steps: Through the trained model, the model is evaluated on three public datasets. The evaluation process is to prepare the model to retrieve the next item that the user will interact with from all items. The public datasets come from movie platforms, e-commerce platforms and music platforms to predict the target items.

Images