Video recommendation method based on sequence recommendation SeqTransRepeatNet

Abstract

The invention provides a video recommendation method based on sequence recommendation SeqTransRepeatNet, and the method comprises the following steps: S1, obtaining a data set, carrying out the preprocessing of the data set, and dividing the preprocessed data set into a training set, a verification set and a test set; s2, constructing a SeqTransRepeatNet model, wherein the SeqTransRepeatNet model comprises an embedded module, a multi-layer cross attention encoder module and a repeated exploration recommendation module which are connected in sequence; s3, the constructed SeqTransRepeatNet model is trained by means of the training set and the verification set, and a finally trained SeqTransRepeatNet model is obtained; and S4, carrying out test evaluation on the finally trained SeqTransRepeatNet model by utilizing the test set, and obtaining a final recommendation list. The method is small in recommendation error and strong in generalization ability, and can accurately recommend the next video watched by the user.

Description

Technical Field

[0001] The present invention relates to the technical field of video recommendation, and in particular to a video recommendation method based on sequence recommendation SeqTransRepeatNet. Background Art

[0002] With the development of 5G technology and the popularity of smart phones, video applications have become an important part of people's lives. By utilizing advanced recommendation algorithms and user behavior data analysis, video platforms can provide more personalized and accurate video recommendations, thereby improving users' viewing experience and platform user stickiness. Therefore, in the face of abundant video resources and a large user base, how to accurately identify user needs and effectively recommend videos that users prefer has become an important issue for Internet service providers and video platform developers.

[0003] Nowadays, recommendation systems have been widely used in many fields such as e-commerce, advertising, and social media for personalized information filtering. The core of recommendation systems is to estimate the possibility of interaction between users and items based on their past actions. Traditional recommendation methods use collaborative filtering methods to solve this problem by assuming that users with similar behaviors show similar preferences for items; deep learning has powerful expressive power, and recommendation models based on deep learning have shown excellent performance in many scenarios. Sequential recommendation based on deep learning has always been a hot topic of research due to its practicality and high-precision capture of sequential information. The sequential recommendation algorithm first sorts the user's past behaviors to obtain an ordered sequence, and then sends it to different neural networks for training, such as recurrent neural networks (RNNs), convolutional neural networks (CNNs), and attention structures, to predict the next item of interest to the user.

[0004] Early work mainly used Markov models (MC) to model local and dynamic preferences and combine them with global preferences to make the next personalized recommendation based on the user's previous behavior. For example, the literature [Rendle S, Freudenthaler C, Schmidt-Thieme L. Factorizing Personalized Markov Chains for Next-Basket Recommendation [C] / / 19th international world wide web conference 2010. ACM, 2010.] proposed a method that combines the traditional matrix factorization (MF) model for decomposing a type of feedback matrix with the transfer matrix model generated by the personalized Markov chain; the literature [He R, McAuley J. Fusing similarity models with markov chains for sparse sequential recommendation [C] / / 2016 IEEE 16th international conference on data mining (ICDM). IEEE, 2016: 191-200] proposed a method based on high-order Markov chains and matrix factorization technology, specifically for capturing long-term and short-term dependencies in user behavior sequences. In recent years, more and more deep learning (DL)-based methods have been used to model dynamic interests. The paper [Nguyen TB, Takasu A. NPE: Neural Personalized Embedding for Collaborative Filtering [J]. 2018.] proposes a neural network model for recommendation systems, which aims to generate personalized embedding representations by combining the contextual information of users and items to improve the recommendation effect. The paper [Cho K, Van Merrienboer B, Gulcehre C, et al. Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation [J]. Computer Science, 2014.] is one of the earliest sequence recommendation algorithms that applied recurrent neural networks (RNNs). It is based on the gated recurrent unit (GRU) design in RNNs and is specifically used to process user behavior sequence data. It makes recommendations by capturing users' dynamic preferences.The paper [Huang K, Du Y, Li L, et al. Pairwise-Based Hierarchical Gating Networks for Sequential Recommendation [C] / / International Conference on Knowledge Science, Engineering and Management. Springer, Cham, 2020.] proposes a hierarchical gating network (HGN) combined with Bayesian personalized ranking to capture long-term and short-term user interests. However, because RNN imposes a strict execution order in training and learning, it faces global dependency problems when the sequence is too long, while CNN is usually used to capture local features in the sequence, emphasizing the influence of recent behaviors. The paper [Tang J, Wang K. Personalized Top-N Sequential Recommendation via Convolutional Sequence Embedding [C] / / Web Search and Data Mining. ACM, 2018.] proposes a convolutional sequence embedding recommendation model (Caser) that embeds a series of recent items into images in time and latent space, and uses convolutional filters to learn sequence patterns as local features of images. This method provides a unified and flexible network structure for capturing general preferences and sequential patterns. Recently, attention models have received extensive attention and research in the field of sequential recommendation. The self-attention model allows point-to-point feature interaction within an item sequence, solves the global dependency problem and allows the use of longer sequences and more information. The paper [Kang WC, Mcauley J. Self-Attentive Sequential Recommendation [C] / / 2018 IEEE International Conference on Data Mining (ICDM). IEEE, 2018.] is the first sequential recommendation model based on the self-attention mechanism. By introducing multi-head self-attention and position encoding, it can efficiently capture the long-term and short-term dependencies in user behavior sequences. Compared with the traditional RNN model, it supports parallel computing and has higher computational efficiency and recommendation accuracy.

[0005] However, the traditional self-attention model fails to accurately model the sequential relationship between items due to implicit position encoding. As the user sequence grows, the attention weights become more dispersed, making it difficult to effectively capture important temporal patterns. This not only affects the model's ability to capture long-term dependencies, but may also lead to neglect of key item relationships, thereby reducing the recommendation effect. At the same time, users may frequently receive recommendations for a certain type of video. This repetition may lead to dissatisfaction and form an information cocoon, reducing user experience and potentially leading to user churn. How to solve the above problems is a challenge. Summary of the invention

[0006] In view of the technical problems that existing methods cannot accurately model the sequential relationship between items, are difficult to effectively capture long-term dependencies, ignore the relationship between key items, and often repeatedly recommend a certain type of video, the present invention proposes a video recommendation method based on sequence recommendation SeqTransRepeatNet. In the constructed SeqTransRepeatNet model, the position encoding is separately decoupled to accurately model the sequential relationship between items; an interest focus block is added to focus on the user's interest information to enhance the model's ability to capture long-term dependencies; cross-attention is used to associate user item information with sequence position information to enhance the understanding of context; a repeated exploration mechanism module is introduced to help users perform repeated video recommendations at appropriate time nodes. The present invention has small recommendation errors and strong generalization ability, and can accurately recommend the user's next video to watch.

[0007] In order to achieve the above object, the technical solution of the present invention is achieved as follows:

[0008] A video recommendation method based on sequence recommendation SeqTransRepeatNet, comprising the following steps:

[0009] S1: Obtain a data set, preprocess the data set, and divide the preprocessed data set into a training set, a validation set, and a test set;

[0010] S2: Construct the SeqTransRepeatNet model, including the sequentially connected embedding module, the multi-layer cross-attention encoder module, and the repeated exploration recommendation module;

[0011] S3: Use the training set and the validation set to train the constructed SeqTransRepeatNet model to obtain the final trained SeqTransRepeatNet model;

[0012] S4: Use the test set to test and evaluate the final trained SeqTransRepeatNet model and obtain the final recommendation list.

[0013] The embedding module described in step S2 includes an embedding layer, a normalization layer I and a random dropout layer I connected in sequence;

[0014] The multi-layer cross-attention encoder module includes a plurality of sequentially connected encoding layers, each encoding layer includes a sequential multi-head self-attention mechanism and a position fully connected feed-forward network connected in sequence, and the sequential multi-head self-attention mechanism of the first encoding layer is connected to the random inactivation layer I;

[0015] The repeated exploration recommendation module includes a repeated exploration mechanism, a repeated module, an exploration module and a prediction layer. The input of the repeated exploration mechanism is connected to the position fully connected feedforward network of the last encoding layer, and the output of the repeated exploration mechanism is connected to the repeated module and the exploration module respectively. The outputs of the repeated module and the exploration module are connected and then connected to the prediction layer.

[0016] The sequence multi-head self-attention mechanism includes a parallel item coding input layer and a position coding input layer, the item coding input layer and the position coding input layer of the sequence multi-head self-attention mechanism of the first coding layer are respectively connected to the random inactivation layer I, the item coding input layer of the sequence multi-head self-attention mechanism of the subsequent coding layer is connected to the position fully connected feedforward network of the previous coding layer, and the position coding input layer of the sequence multi-head self-attention mechanism of the subsequent coding layer is connected to the input of the position coding input layer of the sequence multi-head self-attention mechanism of the previous coding layer; the output of the item coding input layer and the output of the position coding input layer are both connected to each attention head in the sequence multi-head self-attention mechanism, each attention head includes a linear layer I, an interest focus block and a cross-scaling dot product attention module connected in sequence, the linear layer I of each attention head is connected to the output of the item coding input layer and the output of the position coding input layer, the cross-scaling dot product attention module of each attention head is connected to the aggregation module, the aggregation module is connected to the normalization layer II, and the output of the item coding input layer is connected to the normalization layer II;

[0017] The position fully connected feedforward network includes a linear layer II, an activation layer, a linear layer III, a random inactivation layer II and a normalization layer III which are connected in sequence; the output of the normalization layer I is connected to the normalization layer III; the output of the normalization layer III of the position fully connected feedforward network of the first coding layer is connected to the input of the item coding input layer of the sequence multi-head self-attention mechanism of the subsequent coding layer, and the normalization layer III of the position fully connected feedforward network of the last coding layer is connected to the input of the repeated exploration mechanism.

[0018] The method for training the constructed SeqTransRepeatNet model using the training set and the validation set in step S3 is as follows: the training set and the validation set are respectively input into the constructed SeqTransRepeatNet model, and the embedding module is used to embed the video project sequence I of the user from the training set or the validation set.s Perform feature extraction to obtain item codes and position codes;

[0019] In each encoding layer of the multi-layer cross-attention encoder module, the interest focus block and the cross-scaled dot product attention module are used to obtain the attention weight information of the item encoding and the position encoding, and the position fully connected feedforward network is used to calculate the attention weight information of the item encoding and the position encoding. After calculations in multiple encoding layers, the output sequence of the multi-layer cross-attention encoder module is finally obtained;

[0020] In the repeated exploration recommendation module, the output prediction probability is calculated based on the output sequence of the multi-layer cross-attention encoder module;

[0021] The log-likelihood loss function is used to maximize the output prediction probability of a given input sequence according to the output prediction probability, the Adam optimizer is used to optimize the loss function, the optimal hyperparameters of each model are found through grid search, and each model is trained using the early stopping strategy.

[0022] The method for obtaining the item encoding is: taking the hidden dimension as d model , the video item sequence I is embedded into the item sequence I through the item embedding operation of the embedding layer s =i1,i2,…,i τ ,…,i t , Encoded into a dense latent vector representation, the dense latent vector representation is passed through the normalization layer I and the random inactivation layer to obtain the item encoding E item =e1,e2,…,e i ,…,e t , t is the sequence length;

[0023] The method for obtaining the position code is as follows: create an integer sequence from 0 to t-1 as a unique identifier for each position, and then convert the position identifier into a high-dimensional embedding vector through the position embedding operation of the embedding layer. After the high-dimensional embedding vector passes through the normalization layer I and the random inactivation layer, the position code E is obtained. pos =p1,p2,…,p i ,…,p t ,

[0024] The method of using the interest focus block and the cross-scaled dot product attention module to obtain the attention weight information of the item code and the position code is as follows: the item code E input into the item code input layer is converted into the item code E input into the item code input layer by a linear layer I. item Transformed into item query Q, item key K and item value V, the position encoding E of the position encoding input layer is converted into item query Q, item key K and item value V through linear layer I. posIt is converted into position query Qp and position key Kp, and the project key K, project value V and position key Kp are respectively input into the interest focus block for calculation to obtain the new project key K1, new project value V1 and new position key Kp1; the cross-scaled dot product attention module is used to calculate the attention weight about the project and the attention weight about the position and sum them up to get the attention weight information of each attention head, and the attention weight information of multiple heads of the sequential multi-head self-attention mechanism of each encoding layer is spliced ​​through the aggregation module to get the multi-head attention weight MultiHead(Q,K,V), and the multi-head attention weight MultiHead(Q,K,V) is residually connected with the project code and normalized to get the attention weight information X2 of the project code and position code.

[0025] The method of inputting into the interest focus block for calculation is: given an input embedding matrix Define the parameter matrix where d model is the embedding dimension, k is the number of potential interests, and the correlation distribution matrix D between items and interests is calculated matrix :

[0026] D matrix =Softmax(X·θ)

[0027] Among them, Softmax is the activation function;

[0028] Then use the Einstein summation convention to distribute the correlation matrix D between items and interests matrix Aggregate with the input embedding matrix X to obtain the interest block matrix

[0029] X1=Einsum(X,D matrix )

[0030] Among them, Einsum() represents the Einstein summation convention function;

[0031] The input embedding matrix X is the item key K, item value V and position key Kp; the obtained interest block matrix X1 is the new item key K1, new item value V1 and new position key Kp1.

[0032] The method of using the cross-scaled dot product attention module to calculate the attention weight about the item and the attention weight about the position and sum them up to obtain the attention weight information of each attention head is:

[0033] Calculate the attention weights about the items:

[0034]

[0035] Calculate the attention weights with respect to the position:

[0036]

[0037] Sum up to get the attention weight information of each attention head i :

[0038]

[0039] Among them, h is the number of attention heads, W i Q , W i V Respectively represent the weight matrix of the i-th attention head;

[0040] The method for splicing is:

[0041] MultiHead(Q,K,V)=Concat(head1,..,head h )W o

[0042] Among them, Concat is a concatenation operation, W o is the linear transformation matrix.

[0043] The method for calculating the attention weight information of the item code and the position code by using the position fully connected feedforward network is as follows: the attention weight information X2 of the item code and the position code is calculated by the fully connected neural network, and then passed through a random dropout layer, and then residually connected and normalized with the attention weight information X2 of the item code and the position code to obtain the output sequence H of each coding layer l , l represents the lth coding layer;

[0044] The method of calculating the fully connected neural network is:

[0045] x=X2W1+b1

[0046] FFN(X2)=Φ(x)W2+b2

[0047] Among them, Φ(x) is the activation function, W1 and W2 are weights, and b1 and b2 are biases;

[0048] The output sequence of the multi-layer cross-attention encoder module is: L ={h1,h2,...,h τ ,…,h t}, L is the number of encoder layers, where h τ is the τth hidden state.

[0049] The method for calculating the output prediction probability according to the output sequence of the multi-layer cross attention encoder module in the repeated exploration recommendation module is:

[0050] The repeated exploration mechanism outputs the sequence H with a multi-layer cross-attention encoder module. L As input, use the last hidden state h t With each hidden state h τ Perform matching to obtain repeated exploration importance scores Exploring importance scores for repetitions Perform normalization calculation to obtain repeated exploration attention weights For each hidden state h τ Perform weighted summation to obtain the video item sequence I s Context vector Then the softmax activation function is used to transform the video item sequence I s Context vector The probability distribution of the transition mode corresponds to the probability of executing in a repeated mode P(r|I s ) and the probability of executing in exploration mode P(e|I s );

[0051] The repetition module uses a repetition recommendation decoder to compute the recommended item i in the repetition mode j The probability P(i j |r,I s );

[0052] The exploration module uses the exploration recommendation decoder to calculate the recommended item i in the exploration mode j The probability P(i j |e,I s );

[0053] According to the probability P(r|I s ), the probability of executing in exploration mode P(e|I s ), recommend item i in repetition mode j The probability P(i j |r,I s ) and recommend items i in the exploration mode j The probability P(i j |e,I s ), calculate the probability P(i) of generating the next recommended video j |I s ).

[0054] The repeated exploration importance score The calculation method is:

[0055]

[0056] Among them, V re ,W re and U re is the weight parameter, Φ() is the activation function, r and e represent the repetition mode and exploration mode respectively; the repetition exploration attention weight and Video Project Sequence I s Context vector The calculation method is:

[0057]

[0058] The probability of executing in a repeated mode P(r|I s ) and the probability of executing in exploration mode P(e|I s ) is calculated as:

[0059]

[0060] in, is the weight matrix;

[0061] The calculation is the probability P(i) of recommending item i in the repetitive mode. j |r,I s ) is:

[0062]

[0063] Among them, V r ,W r and U r is the weight parameter, Φ() is the activation function, is the repetition importance score, Indicates project i j ∈I s The total number of occurrences, I is the sequence of all video items, P(i j |r,I s ) represents the probability of recommending item i in the repetitive mode;

[0064] The calculation is the probability P(i|e,I s ) is:

[0065]

[0066] in, is the weight matrix; V e ,W e and U e is a parameter, Φ() is an activation function, is the exploration importance score, represents the attention state in exploration mode, and then the last hidden state h τ and attention status Combining into video project sequence I s Mixed representation of P(i j |e,I s ) indicates recommending item i in exploration mode j probability;

[0067] The calculation generates the probability P(i) of the next recommended video. j |I s ) is:

[0068] P(i j |I s )=P(r|I s )P(i j |r,I s )+P(e|I s )P(i j |e,I s )

[0069] P(i j |I s ) indicates that in a given video item sequence I s Recommended items in case of j probability;

[0070] The log-likelihood loss function is:

[0071]

[0072] Among them, θ represents all parameters of the SeqTransRepeatNet model.

[0073] The beneficial effects of the present invention are:

[0074] 1) In order to solve the problem that the implicit position encoding information of the self-attention model cannot accurately model the sequential relationship between items, the position encoding is decoupled separately and put into a multi-layer cross-attention module together with the sequence information for self-attention learning, which can more accurately model the sequential relationship between items.

[0075] 2) In order to solve the problem that the model is difficult to effectively capture long-term dependencies, an interest focus block is added to focus on the user's interest information and enhance the model's ability to capture long-term dependencies.

[0076] 3) In order to solve the problem of the model neglecting key item relations, cross attention is used to associate user-item information with sequence position information to enhance the understanding of context.

[0077] 4) In order to solve the problem of user dissatisfaction caused by repeated recommendations, a repeated exploration mechanism module is introduced to help users perform repeated video recommendations at appropriate time nodes. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0079] Figure 1 This is the overall architecture diagram of the SeqTransRepeatNet model of the present invention.

[0080] Figure 2 This is the cross-scaling dot product attention module diagram of the present invention.

[0081] Figure 3 A module diagram is recommended for the iterative exploration of the present invention. DETAILED DESCRIPTION

[0082] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0083] A video recommendation method based on sequence recommendation SeqTransRepeatNet, such as Figure 1 As shown, the following steps are included:

[0084] S1: Obtain a data set, preprocess the data set, and divide the preprocessed data set into a training set, a validation set, and a test set.

[0085] Obtaining datasets: This example is conducted on three real-world and widely used video datasets, namely the dataset MovieLens 100K, the dataset MovieLens 1M, and the dataset KuaiRec. These datasets are very different in terms of data volume and sparsity. MovieLens is a well-known movie recommendation system dataset created by GroupLens Research Lab, and the two widely used versions are MovieLens 100K and MovieLens 1M. KuaiRec is a real dataset collected from the recommendation logs of the video sharing mobile application Kuaishou.

[0086] Preprocessing: The three datasets follow the same data preprocessing procedure. This embodiment converts each dataset into an implicit dataset by treating each rating or comment in the dataset as the existence of a user and item interaction. Then, the interactions are grouped by user ID and all groups are sorted by timestamp to form a video item sequence for each user, discarding short sequences with less than 5 interactions. Table 1 shows the statistics of the dataset after preprocessing.

[0087] Table 1 Statistics of the preprocessed dataset

[0088]

[0089] Among them, #User represents the number of users, #Items represents the number of videos, #Inters represents the number of interactions, #Avgu represents the average number of user operations, #Avgi represents the average number of item operations, and Sparsity represents sparsity.

[0090] The three datasets used are often used as public benchmark datasets and are of different sizes and densities, which makes it easy to test models on different types of datasets.

[0091] Dataset division: In order to evaluate the performance of each method, the leave-one-out method is used for each user's video item sequence, retaining the last video item for testing and the second-to-last video item for verification. The remaining video items are used for training, and each real video item in the test set is paired with 100 video items that have not interacted with the user (negative samples), and extracted by uniform sampling, which is more consistent with the real situation in the real world.

[0092] S2: Construct the SeqTransRepeatNet model, including the sequentially connected embedding module, the multi-layer cross-attention encoder module, and the repeated exploration recommendation module;

[0093] The embedding module is used to sequence the user's video items. s Perform feature extraction to obtain an item code for representing item embedding information and a position code for representing sequence embedding position information;

[0094] The multi-layer cross-attention encoder module adds an interest focus block on the basis of the Transformer decoder to focus on the user's interest information. In each encoding layer, the interest focus block and the cross-scaled dot product attention module are used to obtain the attention weight information of the item encoding and position encoding to enhance the understanding of the context. The position fully connected feedforward network is used to calculate the attention weight information of the item encoding and position encoding;

[0095] The repeated exploration recommendation model helps the model calculate the selection probability of repeated patterns or exploration patterns and make video recommendations at the right time.

[0096] The embedding module includes an embedding layer, a normalization layer I and a random dropout layer I connected in sequence;

[0097] The attention module has received extensive attention and research in the field of sequential recommendation. The self-attention model allows point-to-point feature interaction within the video item sequence, solves the global dependency problem and allows the use of longer sequences and more information. In this algorithm, the attention module of the model is the focus of attention. The network architecture of Transformer mainly includes two parts: encoder and decoder, each of which consists of multiple stacked identical layers. Each layer consists of two main sublayers: a multi-head self-attention mechanism and a feedforward neural network. The present invention mainly introduces its encoder structure. The encoder is stacked with multiple identical encoder layers, and each layer contains a multi-head self-attention mechanism and a simple position fully connected feedforward network. The multi-head self-attention mechanism includes a self-attention mechanism and a multi-head mechanism. The self-attention mechanism generates attention weights by calculating the query, key, and value vectors of each element in the input sequence. These weights are used to weighted sum the values ​​of all input elements to capture the dependencies between elements in the sequence. The multi-head mechanism captures information from different subspaces through multiple independent self-attention heads and splices their outputs together.

[0098] The multi-layer cross-attention encoder module includes multiple encoding layers, each encoding layer includes a sequential multi-head self-attention mechanism and a position fully connected feedforward network connected in sequence, and the sequential multi-head self-attention mechanism of the first encoding layer is connected to the random inactivation layer I; the sequential multi-head self-attention mechanism includes a parallel project coding input layer and a position coding input layer, and the project coding input layer and the position coding input layer are only used for the input of project coding and position coding, and no operation is performed therein, the project coding input layer and the position coding input layer of the sequential multi-head self-attention mechanism of the first encoding layer are respectively connected to the random inactivation layer I, the project coding input layer of the sequential multi-head self-attention mechanism of the subsequent encoding layer is connected to the position fully connected feedforward network of the previous encoding layer, and the position coding input layer of the sequential multi-head self-attention mechanism of the subsequent encoding layer is connected to the input of the position coding input layer of the sequential multi-head self-attention mechanism of the previous encoding layer; the output of the project coding input layer and the position coding input The output of the input layer is connected to each attention head in the sequence multi-head self-attention mechanism, each attention head includes a linear layer I, an interest focus block and a cross-scaling dot product attention module connected in sequence, the linear layer I of each attention head is connected to the output of the item encoding input layer and the output of the position encoding input layer, the cross-scaling dot product attention module of each attention head is connected to the aggregation module, the aggregation module is connected to the normalization layer II, and the output of the item encoding input layer is connected to the normalization layer II; the position fully connected feedforward network includes a linear layer II, an activation layer, a linear layer III, a random inactivation layer II and a normalization layer III connected in sequence; the output of the normalization layer I is connected to the normalization layer III; the output of the normalization layer III of the position fully connected feedforward network of the first encoding layer is connected to the input of the item encoding input layer of the sequence multi-head self-attention mechanism of the subsequent encoding layer, and the normalization layer III of the position fully connected feedforward network of the last encoding layer is connected to the input of the repeated exploration mechanism.

[0099] Duplicate recommendations refer to a situation where a user receives the same or similar recommended content multiple times. The processing of duplicate recommendations is crucial to the effectiveness evaluation of the recommendation system and the user experience. It is necessary to comprehensively consider the diversity, novelty and personalized needs of the recommendations to avoid excessive recommendation of the same content that may cause fatigue or dissatisfaction to users.

[0100] The repeated exploration recommendation module includes a repeated exploration mechanism, a repeated module, an exploration module and a prediction layer. The input of the repeated exploration mechanism is connected to the position fully connected feedforward network of the last encoding layer. The output of the repeated exploration mechanism is connected to the repeated module and the exploration module respectively. The outputs of the repeated module and the exploration module are connected and then connected to the prediction layer. The prediction layer is used to output a recommendation list according to the probability of the next recommended video. The model recommends videos in the user's history in the repeated module and recommends new videos to the user in the exploration module.

[0101] S3: Use the training set and the validation set to train the constructed SeqTransRepeatNet model to obtain the final trained SeqTransRepeatNet model;

[0102] The method for training the constructed SeqTransRepeatNet model using the training set and the validation set is as follows: the training set and the validation set are respectively input into the constructed SeqTransRepeatNet model, and the video project sequence I of the user from the training set or the validation set is embedded in the model using the embedding module. s Perform feature extraction to obtain item codes and position codes;

[0103] The method for obtaining the item encoding is: taking the hidden dimension as d model , the video item sequence I is embedded into the item sequence I through the item embedding operation of the embedding layer s =i1,i2,…,i τ ,…,i t , Encoded into a dense latent vector representation to reduce the amount of computation and improve representation learning, the dense latent vector representation is passed through the normalization layer I and the random inactivation layer to obtain the item encoding E item =e1,e2,…,e i ,…,e t , t is the sequence length, i τ Refers to a video project, Video Project Sequence I s Represents the set of all video items watched by a user in history, that is, a sequence containing all videos watched by a user. s | indicates video item sequence I s Total number of videos in

[0104] Traditional self-attention-based recommendation algorithms directly encode item E item and position code E pos Mixed introduction of sequence relationship, that is, (E item +E pos ), however, such forced fusion cannot distinguish the difference between items and positions well, which may limit the model's ability to capture sequence relationships. Therefore, the present invention decouples the position encoding and models it simultaneously with the item encoding, so as to better capture the potential relationship between items.

[0105] The method for obtaining the position code is as follows: create an integer sequence from 0 to t-1 as a unique identifier for each position, and then convert the position identifier into a high-dimensional embedding vector through the position embedding operation of the embedding layer to capture the semantic information of the position. After the high-dimensional embedding vector passes through the normalization layer I and the random inactivation layer, the position code E is obtained. pos =p1,p2,…,p i ,…,p t ,

[0106] In each encoding layer of the multi-layer cross-attention encoder module, the interest focus block and the cross-scaled dot product attention module are used to obtain the attention weight information of the item encoding and position encoding: the item encoding E input to the item encoding input layer is converted into the item encoding E input by the linear layer I. item Transformed into item query Q, item key K and item value V, the position encoding E of the position encoding input layer is converted into item query Q, item key K and item value V through linear layer I. pos Convert it into position query Qp and position key Kp, and input the item key K, item value V and position key Kp into the interest focus block for calculation, and obtain the new item key K1, new item value V1 and new position key Kp1;

[0107] Assuming that most users have no more than t categories in their historical interests, t is a configurable hyperparameter, which can be [5, 10, 15, 20, 25]. Then the video item sequence I can be s Aggregate into potential blocks of interest. For example, the number of video item sequences |I s | is 50, and the interest block is set to 20, then these 50 historical video items can be turned into 20 related interest blocks, making Reduced calculation amount.

[0108] The method of inputting into the interest focus block for calculation is: given an input embedding matrix Define the parameter matrix where d model is the embedding dimension, k is the number of potential interests, and the correlation distribution matrix D between items and interests is calculated matrix , represents the weight distribution of each item for different potential interests:

[0109] D matrix =Softmax(X·θ)

[0110] Among them, Softmax is the activation function;

[0111] Then use the Einstein summation convention to distribute the correlation matrix D between items and interests matrix Aggregate with the embedding matrix X to obtain the interest block matrix

[0112] X1=Einsum(X,D matrix )

[0113] Among them, Einsum() represents the Einstein summation convention function;

[0114] The input embedding matrix X is the item key K, item value V and position key Kp; the obtained interest block matrix X1 is the new item key K1, new item value V1 and new position key Kp1.

[0115] like Figure 2 As shown, the attention weights of the items and the attention weights of the positions are calculated using the cross-scaled dot product attention module and summed up to obtain the attention weight information of each attention head. The attention weight information of multiple heads of the sequence multi-head self-attention mechanism of each encoding layer is spliced ​​through the aggregation module to obtain the multi-head attention weight MultiHead(Q, K, V);

[0116] In this embodiment, there are two left and right cross-scaling dot product attention blocks that simultaneously calculate different attention weights, and use cross attention to associate different information, thereby enhancing the understanding of the context.

[0117] Compute the attention weights for the items in :

[0118]

[0119] Calculate the attention weights with respect to the position:

[0120]

[0121] Sum up to get the attention weight information of each attention head i :

[0122]

[0123] Among them, h is the number of attention heads, W i Q , W i V Respectively represent the weight matrix of the i-th attention head;

[0124] The method for splicing is:

[0125] MultiHead(Q,K,V)=Concat(head1,..,head h )W o

[0126] Among them, Concat is a concatenation operation, W ois the linear transformation matrix.

[0127] The multi-head attention weight MultiHead(Q, K, V) is residually connected to the item code and normalized to obtain the attention weight information X2 of the item code and position code.

[0128] The position fully connected feedforward network is used to calculate the attention weight information of the item code and the position code: the attention weight information X2 of the item code and the position code is calculated by the fully connected neural network, and then passes through a random dropout layer and performs residual connection and normalization operation with the attention weight information X2 of the item code and the position code to obtain the output sequence H of each encoding layer l , l represents the lth coding layer;

[0129] The method for calculating the fully connected neural network is:

[0130] x=X2W1+b1

[0131] GeLU: erf is the error function

[0132] ReLU: Φ = max(0,x)

[0133] Swish:

[0134] Tanh:

[0135] Sigmoid:

[0136] FFN(X2)=ΦW2+b2

[0137] W1 and W2 are weights, b1 and b2 are biases;

[0138] The position fully connected feedforward neural network is a fully connected neural network FFN consisting of two linear transformations and a function hidden_act that encapsulates five activation functions (GeLU, ReLU, Swish, Tanh and Sigmoid). The activation function Φ is an adjustable parameter.

[0139] After multiple encoding layers are calculated, the output sequence of the multi-layer cross attention encoder module is finally obtained: L ={h1,h2,...,h τ ,…,h t}, L is the number of encoder layers, where h τ is the τth hidden state.

[0140] like Figure 3As shown, finally, the cross-attention encoder module transforms the video item sequence I s Encoded as output sequence H L ={h1,h2,...,h τ ,…,h t} where h τ represents the hidden state at timestamp τ. The repeated exploration mechanism uses the output sequence H of the multi-layer cross-attention encoder module L As input, it predicts the probability of executing the repetition mode or exploration mode. The repetition recommendation decoder takes the output sequence H of the multi-layer cross-attention encoder module as L As input, predict the video item sequence I s The probability of repeated recommendation of the clicked item in . The exploration recommendation decoder is an output sequence H of the multi-layer cross-attention encoder module. L As input, we predict the video item sequence (II s ), where I is the sequence of all video items.

[0141] In the repeated exploration recommendation module, the output prediction probability is calculated according to the output sequence of the multi-layer cross attention encoder module to help users perform video recommendations at the appropriate time node. The repeated exploration mechanism can be regarded as a binary classifier, which is based on the output sequence H of the multi-layer cross attention encoder module. L ={h1,h2,...,h τ ,…,h t} to predict the recommendation mode. The specific steps are as follows:

[0142] The repeated exploration mechanism uses the multi-layer cross-attention encoder module output sequence as H L As input, use the last hidden state h t With each hidden state h τ Perform matching to obtain repeated exploration importance scores

[0143]

[0144] Among them, V re ,W re and U re is the weight parameter, Φ is the activation function, r and e represent the repetition mode and exploration mode respectively;

[0145] Exploring importance scores for repetitions Perform normalization calculation to obtain repeated exploration attention weights

[0146]

[0147] For each hidden state hτ Perform weighted summation to obtain the video item sequence I s Context vector

[0148]

[0149] Then the softmax activation function is used to transform the video item sequence I s Context vector The probability distribution of the transition mode corresponds to

[0150] The probability of repeating the pattern execution P(r|I s ) and the probability of executing in exploration mode P(e|I s ):

[0151]

[0152] in, is the weight matrix;

[0153] Repeat Module Evaluating Re-clicking Video Item Sequences Using Repeat Recommendation Decoder I s The probability of the item in the calculation is to recommend item i in the repetitive mode j The probability P(i j |r,I s ):

[0154]

[0155] Where V r ,W r and U r is the weight parameter, Φ is the activation function, is the repetition importance score, Indicates project i j ∈I s The total number of occurrences, I is the sequence of all video items, P(i j |r,I s ) indicates recommending item i in repeated mode j probability;

[0156] The exploration module uses the exploration recommendation decoding evaluation to click on the video item sequence I s The probability of a new item not existing in the , and the probability of recommending item i in the exploration mode P(i j |e,I s ):

[0157]

[0158] in, is the weight matrix; where V e ,We and U e is a parameter, Φ is an activation function, is the exploration importance score, represents the attention state in exploration mode, and then the last hidden state h τ and attention status Combining into video project sequence I s Mixed representation of P(i j |e,I s ) indicates recommending item i in exploration mode j The probability that an item exists in the video item sequence I is s , then the probability of it being recommended in the exploration module is 0.

[0159] According to the probability P(r|I s ), the probability of executing in exploration mode P(e|I s ), recommend item i in repetition mode j The probability P(i j |r,I s ) and the probability P(i) of recommending item i in the exploration mode j |e,I s ) Calculate the probability P(i) of generating the next recommended video j |I s ):

[0160] P(i j |I s )=P(r|I s )P(i j |r,I s )+P(e|I s )P(i j |e,I s )

[0161] P(i j |I s ) indicates that in a given video item sequence I s Recommended items in case of j probability.

[0162] The prediction layer is based on the probability P(i j |I s ) to output a list of recommendations.

[0163] The output prediction probability of a given input sequence is maximized using the log-likelihood loss function based on the output prediction probability:

[0164]

[0165] Among them, θ represents all parameters of the SeqTransRepeatNet model.

[0166] The Adam optimizer is used to optimize the loss function, and the learning rate is a hyperparameter of lr∈{0.01,0.005,0.001,0.0005,0.0001}. The normal distribution is used to initialize the parameters, with a mean of 0 and a standard deviation of 0.02. The weight of the normalization layer is initialized to 1.0, and the bias is initialized to 0; the bias of the linear layer is initialized to 0.

[0167] The model is implemented in Pytorch, and grid search is used to find the best parameters for each model; the hidden dimension d is selected model ∈{16,32,64,96,128}; dropout rate d∈{0.2,0.3,0.4,0.5 activation function type Φ∈{′GeLU′,′ReLU′,′Swish′,′TanH′,′Sigmoid′}; number of potential interests k∈{5,10,15,20,25}; learning rate lr∈{0.01,0.005,0.001,0.0005,0.0001}; number of attention heads h∈{2,4,8} for grid search. For the STRN model, the best hyperparameters for each benchmark are:

[0168] ml-100k: {d model =32, d=0.4, L=2, Φ='RELU', k=15, lr=0.01, h=8}

[0169] ml-1m: {d model =96, d=0.4, L=2, Φ=′GELU′, k=25, lr=0.0005, h=8}

[0170] KuaiRec: model =128, d=0.2, L=2, Φ=′Tanh′, k=25, lr=0.001, h=2}

[0171] To ensure fairness, the video project sequence is uniformly set. s The maximum sequence length is 50, the number of layers of the multi-layer cross-attention encoder module is L=2, and grid search is used to search for the optimal result.

[0172] At the same time, an early stopping strategy is used to train each model. When the verification accuracy does not improve within 10 epochs, the model will be terminated early. All models are trained on a single NVIDIA GeForce RTX 3090 GPU.

[0173] S4: Use the test set to test and evaluate the final trained SeqTransRepeatNet model and obtain the final recommendation list.

[0174] In order to score the ranking list, the present invention adopts three common evaluation indicators: recall, mean reciprocal rank (MRR) and normalized discounted cumulative gain (NDCG), which are explained as follows:

[0175] Recall measures the proportion of items that users actually like among all related items. The formula is as follows.

[0176]

[0177] Among them, U represents the user set, represents the top K recommendation list of user u, and R(u) represents the set of items actually liked by user u.

[0178] The mean reciprocal rank (MRR) measures the ranking position of the first relevant item in the recommendation results, and its formula is shown below.

[0179]

[0180] Among them, U represents the user set, Indicates the ranking position of the first relevant item of user u in the recommendation list.

[0181] Normalized discounted cumulative gain (NDCG) is a measure of ranking quality where the position is logarithmically discounted, and it accounts for the position of hits by assigning higher scores to top-ranked hits, and its formula is:

[0182]

[0183] Among them, U represents the user set, pi represents the position of the target item of user i in the recommendation list, and the higher the position, the larger pi is.

[0184] When the true item ranks higher in the top-K list (i.e., the recommendation list), Recall@K, MRR@K, and NDCG@K all have larger values, where K = (5, 10).

[0185] In order to verify the superior performance of the proposed method, 9 typical recommendation algorithms were selected as baseline models, and the results of all baseline models on three datasets were verified in Table 2. Among them, STRN is the recommendation algorithm proposed in the text.

[0186] GRU4Rec: A conversational recommender system based on recurrent neural networks.

[0187] NextItNet: A simple convolutional generative network for next item recommendation.

[0188] SASRec: A sequential recommendation model based on Transformer using self-attention mechanism.

[0189] Bert4Rec: A sequence recommendation model based on bidirectional Transformer.

[0190] LightSANs: A Low-Rank Factorized Self-Attention Network for Next Item Recommendation.

[0191] FEARec: A frequency-enhanced hybrid attention network for sequential recommendation.

[0192] AdaMCT: A CNN-Transformer Adaptive Hybrid Network for Sequence Recommendation.

[0193] SINE: A Sparse Interest Network for Sequential Recommendation.

[0194] CORE: A simple and effective session-based recommender system with a unified representation space.

[0195] Table 2 Performance comparison of different models

[0196]

[0197]

[0198] The score in bold is the best score in each row, and the score in underline is the second best score.

[0199] Table 2 shows the recommendation performance of the proposed model on three datasets and summarizes the best results of all baseline models on the three datasets. In the ml-100k dataset, SASRec and LightSANs, represented by the attention mechanism, effectively capture the long-range dependencies in user behavior through a powerful attention mechanism, thus performing outstandingly on the ml-100k dataset; in the ml-1m dataset, GRU4Rec, represented by RNN, leads SASRec by a slight margin by leveraging its powerful temporal information modeling capability of recurrent neural networks; and for the relatively sparse KuaiRec dataset, the CORE model significantly improves the recommendation effect by constructing a unified representation space, especially in the case of sparse data, showing strong adaptability and robustness.

[0200] For STRN, the best results were achieved on each dataset, with an average improvement of 13.12%, 4.61%, and 15.62% in the three datasets, respectively. This result shows that the STRN model not only performs well when processing dense datasets, but also shows strong adaptability and superior performance when processing sparse datasets. This further proves that the method of the present invention has wide applicability and effectiveness in various scenarios, and provides new directions and possibilities for future research and application of video recommendation.

[0201] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A video recommendation method based on sequence recommendation SeqTransRepeatNet, characterized in that: The following steps are involved: S1: Obtain a data set, preprocess the data set, and divide the preprocessed data set into a training set, a validation set, and a test set; S2: Construct the SeqTransRepeatNet model, including the sequentially connected embedding module, the multi-layer cross-attention encoder module, and the repeated exploration recommendation module; S3: Use the training set and the validation set to train the constructed SeqTransRepeatNet model to obtain the final trained SeqTransRepeatNet model; S4: Use the test set to test and evaluate the final trained SeqTransRepeatNet model and obtain the final recommendation list.

2. The video recommendation method based on sequence recommendation SeqTransRepeatNet according to claim 1, characterized in that: The embedding module described in step S2 includes an embedding layer, a normalization layer I and a random dropout layer I connected in sequence; The multi-layer cross-attention encoder module includes a plurality of sequentially connected encoding layers, each encoding layer includes a sequential multi-head self-attention mechanism and a position fully connected feed-forward network connected in sequence, and the sequential multi-head self-attention mechanism of the first encoding layer is connected to the random inactivation layer I; The repeated exploration recommendation module includes a repeated exploration mechanism, a repeated module, an exploration module and a prediction layer. The input of the repeated exploration mechanism is connected to the position fully connected feedforward network of the last encoding layer, and the output of the repeated exploration mechanism is connected to the repeated module and the exploration module respectively. The outputs of the repeated module and the exploration module are connected and then connected to the prediction layer.

3. The video recommendation method based on sequence recommendation SeqTransRepeatNet according to claim 2, characterized in that: The sequence multi-head self-attention mechanism includes a parallel item coding input layer and a position coding input layer, the item coding input layer and the position coding input layer of the sequence multi-head self-attention mechanism of the first coding layer are respectively connected to the random inactivation layer I, the item coding input layer of the sequence multi-head self-attention mechanism of the subsequent coding layer is connected to the position fully connected feedforward network of the previous coding layer, and the position coding input layer of the sequence multi-head self-attention mechanism of the subsequent coding layer is connected to the input of the position coding input layer of the sequence multi-head self-attention mechanism of the previous coding layer; the output of the item coding input layer and the output of the position coding input layer are both connected to each attention head in the sequence multi-head self-attention mechanism, each attention head includes a linear layer I, an interest focus block and a cross-scaling dot product attention module connected in sequence, the linear layer I of each attention head is connected to the output of the item coding input layer and the output of the position coding input layer, the cross-scaling dot product attention module of each attention head is connected to the aggregation module, the aggregation module is connected to the normalization layer II, and the output of the item coding input layer is connected to the normalization layer II; The position fully connected feed-forward network includes a linear layer II, an activation layer, a linear layer III, a random inactivation layer II and a normalization layer III connected in sequence; The output of normalization layer I is connected to normalization layer III; the output of normalization layer III of the positional fully connected feedforward network of the first encoding layer is connected to the input of the item encoding input layer of the sequential multi-head self-attention mechanism of the subsequent encoding layer, and the normalization layer III of the positional fully connected feedforward network of the last encoding layer is connected to the input of the repeated exploration mechanism.

4. The video recommendation method based on sequence recommendation SeqTransRepeatNet according to claim 3 is characterized in that: The method for training the constructed SeqTransRepeatNet model using the training set and the validation set in step S3 is as follows: the training set and the validation set are respectively input into the constructed SeqTransRepeatNet model, and the embedding module is used to embed the video project sequence I of the user from the training set or the validation set. s Perform feature extraction to obtain item codes and position codes; In each encoding layer of the multi-layer cross-attention encoder module, the interest focus block and the cross-scaled dot product attention module are used to obtain the attention weight information of the item encoding and the position encoding, and the position fully connected feedforward network is used to calculate the attention weight information of the item encoding and the position encoding. After calculations in multiple encoding layers, the output sequence of the multi-layer cross-attention encoder module is finally obtained; In the repeated exploration recommendation module, the output prediction probability is calculated based on the output sequence of the multi-layer cross-attention encoder module; The log-likelihood loss function is used to maximize the output prediction probability of a given input sequence according to the output prediction probability, the Adam optimizer is used to optimize the loss function, the optimal hyperparameters of each model are found through grid search, and each model is trained using the early stopping strategy.

5. The video recommendation method based on sequence recommendation SeqTransRepeatNet according to claim 4, characterized in that: The method for obtaining the item encoding is: taking the hidden dimension as d model , the video item sequence is embedded into the item sequence through the embedding layer Encoded into a dense latent vector representation, the dense latent vector representation is passed through the normalization layer I and the random inactivation layer to obtain the item encoding t is the sequence length; The method for obtaining the position code is as follows: creating an integer sequence from 0 to t-1 as a unique identifier for each position, and then converting the position identifier into a high-dimensional embedding vector through the position embedding operation of the embedding layer. After the high-dimensional embedding vector passes through the normalization layer I and the random inactivation layer, the position code is obtained.

6. The video recommendation method based on sequence recommendation SeqTransRepeatNet according to claim 5, characterized in that: The method of using the interest focus block and the cross-scaled dot product attention module to obtain the attention weight information of the item code and the position code is as follows: the item code E input into the item code input layer is converted into the item code E input into the item code input layer by a linear layer I. item Transformed into item query Q, item key K and item value V, the position encoding E of the position encoding input layer is converted into item query Q, item key K and item value V through linear layer I. pos Convert it into position query Qp and position key Kp, and input the item key K, item value V and position key Kp into the interest focus block for calculation, and obtain the new item key K1, new item value V1 and new position key Kp1; use The cross-scaled dot product attention module calculates the attention weight about the item and the attention weight about the position and sums them up to obtain the attention weight information of each attention head. The aggregation module concatenates the attention weight information of multiple heads of the sequential multi-head self-attention mechanism of each encoding layer to obtain the multi-head attention weight MultiHead(Q, K, V). The multi-head attention weight MultiHead(Q, K, V) is residually connected with the item code and normalized to obtain the attention weight information X2 of the item code and position code.

7. The video recommendation method based on sequence recommendation SeqTransRepeatNet according to claim 6, characterized in that: The method of inputting into the interest focus block for calculation is: given an input embedding matrix Define the parameter matrix where d model is the embedding dimension, k is the number of potential interests, and the correlation distribution matrix D between items and interests is calculated matrix : D matrix =Softmax(X·θ) Among them, Softmax is the activation function; Then use the Einstein summation convention to distribute the correlation matrix D between items and interests matrix Aggregate with the input embedding matrix X to obtain the interest block matrix X1=Einsum(X,D matrix ) Among them, Einsum() represents the Einstein summation convention function; The input embedding matrix X is the item key K, item value V and position key Kp; the obtained interest block matrix X1 is the new item key K1, new item value V1 and new position key Kp1.

8. The video recommendation method based on sequence recommendation SeqTransRepeatNet according to claim 6 or 7, characterized in that: The method of using the cross-scaled dot product attention module to calculate the attention weight about the item and the attention weight about the position and sum them up to obtain the attention weight information of each attention head is: Calculate the attention weights about the items: Calculate the attention weights with respect to the position: Sum up to get the attention weight information of each attention head i : Among them, h is the number of attention heads, Respectively represent the weight matrix of the i-th attention head; The method for splicing is: MultiHead(Q,K,V)=Concat(head1,..,head h )W o Among them, Concat is a concatenation operation, W o is the linear transformation matrix; The method for calculating the attention weight information of the item code and the position code by using the position fully connected feedforward network is as follows: the attention weight information X2 of the item code and the position code is calculated by the fully connected neural network, and then passed through a random dropout layer, and then residually connected and normalized with the attention weight information X2 of the item code and the position code to obtain the output sequence H of each coding layer l , l represents the lth coding layer; The method of calculating the fully connected neural network is: x=X2W1+b1 FFN(X2)=Φ(x)W2+b2 Among them, Φ(x) is the activation function, W1 and W2 are weights, and b1 and b2 are biases; The output sequence of the multi-layer cross-attention encoder module is: L ={h1,h2,...,h τ ,…,h t }, L is the number of encoder layers, where h τ is the τth hidden state.

9. The video recommendation method based on sequence recommendation SeqTransRepeatNet according to claim 8, characterized in that: The method for calculating the output prediction probability according to the output sequence of the multi-layer cross attention encoder module in the repeated exploration recommendation module is: The repeated exploration mechanism outputs the sequence H with a multi-layer cross-attention encoder module. L As input, use the last hidden state h t With each hidden state h τ Perform matching to obtain repeated exploration importance scores Exploring importance scores for repetitions Perform normalization calculation to obtain repeated exploration attention weights For each hidden state h τ Perform weighted summation to obtain the video item sequence I s Context vector Then the softmax activation function is used to transform the video item sequence I s Context vector The probability distribution of the transition mode corresponds to the probability of executing in a repeated mode P(r|I s ) and the probability of executing in exploration mode P(e|I s ); The repetition module uses a repetition recommendation decoder to compute the recommended item i in the repetition mode j The probability P(i j |r,I s ); The exploration module uses the exploration recommendation decoder to calculate the recommended item i in the exploration mode j The probability P(i j |e,I s ); According to the probability P(r|I s ), the probability of executing in exploration mode P(e|I s ), recommend item i in repetition mode j The probability P(i j |r,I s ) and recommend items i in the exploration mode j The probability P(i j |e,I s ), calculate the probability P(i) of generating the next recommended video j |I s ).

10. The video recommendation method based on sequence recommendation SeqTransRepeatNet according to claim 9, characterized in that: The repeated exploration importance score The calculation method is: Among them, V re ,W re and U re is the weight parameter, Φ() is the activation function, r and e represent the repetition mode and exploration mode respectively; The repeated exploration attention weights and Video Project Sequence I s Context vector The calculation method is: The probability of executing in a repeated mode P(r|I s ) and the probability of executing in exploration mode P(e|I s ) is calculated as: in, is the weight matrix; The calculation is the probability P(i) of recommending item i in the repetitive mode. j |r,I s ) is: Among them, V r ,W r and U r is the weight parameter, Φ() is the activation function, is the repetition importance score, Indicates project i j ∈I s The total number of occurrences, I is the sequence of all video items, P(i j |r,I s ) represents the probability of recommending item i in the repetitive mode; The calculation is the probability P(i|e,I s ) is: in, is the weight matrix; V e ,W e and U e is a parameter, Φ() is an activation function, is the exploration importance score, represents the attention state in exploration mode, and then the last hidden state h τ and attention status Combining into video project sequence I s Mixed representation of P(i j |e,I s ) indicates recommending item i in exploration mode j probability; The calculation generates the probability P(i) of the next recommended video. j |I s ) is: P(i j |I s )=P(r|I s )P(i j |r,I s )+P(e|I s )P(i j |e,I s ) P(i j |I s ) indicates that in a given video item sequence I s Recommended items in case of j probability; The log-likelihood loss function is: Among them, θ represents all parameters of the SeqTransRepeatNet model.

Images