A video recommendation method based on Sequence Recommendation SeqTransRepeatNet

Abstract

This invention proposes a video recommendation method based on SeqTransRepeatNet, comprising the following steps: S1: Obtain the dataset, preprocess the dataset, and divide the preprocessed dataset into a training set, a validation set, and a test set; S2: Construct a SeqTransRepeatNet model, including a sequentially connected embedding module, a multi-layer cross-attention encoder module, and a repeated exploration recommendation module; S3: Train the constructed SeqTransRepeatNet model using the training and validation sets to obtain the final trained SeqTransRepeatNet model; S4: Test and evaluate the final trained SeqTransRepeatNet model using the test set and obtain the final recommendation list. This invention's method has small recommendation error, strong generalization ability, and can accurately recommend the user's next video to watch.

Description

Technical Field

[0001] This invention relates to the technical field of video recommendation, and in particular to a video recommendation method based on Sequence Recommendation (SeqTransRepeatNet). Background Technology

[0002] With the development of 5G technology and the widespread adoption of smartphones, video applications have become an important part of people's lives. By utilizing advanced recommendation algorithms and user behavior data analysis, video platforms can provide more personalized and accurate video recommendations, thereby improving the user viewing experience and platform user stickiness. Therefore, given the abundance of video resources and the large user base, accurately identifying user needs and effectively recommending videos that users prefer has become a crucial issue for internet service providers and video platform developers.

[0003] Today, recommender systems are widely used in many fields such as e-commerce, advertising, and social media for personalized information filtering. Their core principle is to estimate the probability of user-item interaction based on past user actions. Traditional recommendation methods use collaborative filtering, assuming that users with similar behaviors exhibit similar preferences for items to address this issue. Deep learning, with its powerful expressive capabilities, has demonstrated superior performance in many scenarios for its recommendation models. Sequence recommendation based on deep learning has been a research hotspot due to its practicality and high accuracy in capturing sequential information. Sequence recommendation algorithms first sort the user's past behaviors to obtain an ordered sequence, then feed this sequence into different neural networks for training, such as recurrent neural networks (RNNs), convolutional neural networks (CNNs), and attention structures, to predict the user's next item of interest.

[0004] Early work primarily used Markov models (MCs) to model local and dynamic preferences and combine them with global preferences to propose the next personalized recommendation based on the user's previous behavior. For example, the paper [Rendle S, Freudenthaler C, Schmidt-Thieme L. Factorizing Personalized Markov Chains for Next-Basket Recommendation[C] / / 19th international world wide web conference2010.ACM,2010.] proposed a method that combines a traditional matrix factorization (MF) model for decomposing a class of feedback matrices with a model for decomposing transition matrices generated by personalized Markov chains; the paper [He R, McAuley J. Fusing similarity models with markov chains for sparse sequential recommendation[C] / / 2016 IEEE 16th international conference on data mining (ICDM).IEEE,2016:191-200] proposed a method based on high-order Markov chains and matrix factorization techniques specifically designed to capture long-term and short-term dependencies in user behavior sequences. In recent years, an increasing number of deep learning (DL)-based methods have been used to model dynamic interests. The paper [Nguyen TB, Takasu A. NPE: Neural Personalized Embedding for Collaborative Filtering[J]. 2018.] proposes a neural network model for recommender systems, aiming to improve recommendation performance by generating personalized embedding representations by combining contextual information about users and items. The paper [Cho K, VanMerrienboer B, Gulcehre C, et al. Learning Phrase Representations using RNNEncoder-Decoder for Statistical Machine Translation[J]. Computer Science, 2014.] is one of the earliest sequence recommendation algorithms to apply recurrent neural networks (RNNs). Based on the gated recurrent unit (GRU) design in RNNs, it is specifically designed to process user behavior sequence data and makes recommendations by capturing users' dynamic preferences.The paper [Huang K, Du Y, Li L, et al. Pairwise-Based Hierarchical Gating Networks for Sequential Recommendation[C] / / International Conference on Knowledge Science, Engineering and Management. Springer, Cham, 2020.] proposes a hierarchical gating network (HGN) combined with Bayesian personalized ranking to capture the interests of long-term and short-term users. However, because RNNs impose a strict execution order during training, they face the problem of global dependencies when the sequence is too long, while CNNs are usually used to capture local features in the sequence, emphasizing the influence of recent behavior. The paper [Tang J, Wang K. Personalized Top-N Sequential Recommendation via Convolutional Sequence Embedding[C] / / Web Search and DataMining. ACM, 2018.] proposes a convolutional sequence embedding recommendation model (Caser) that embeds a series of recent items into the image in time and latent space, and uses convolutional filters to learn sequence patterns as local features of the image. This approach provides a unified and flexible network structure for capturing general preferences and sequential patterns. Recently, attention models have received widespread attention and research in the field of sequential recommendation. Self-attention models allow point-to-point feature interactions within a sequence of items, solving the global dependency problem and allowing the use of longer sequences and more information. The paper [Kang WC, Mcauley J. Self-Attentive Sequential Recommendation[C] / / 2018IEEE International Conference on Data Mining(ICDM).IEEE,2018.] is the first sequential recommendation model based on the self-attention mechanism. By introducing multi-head self-attention and positional encoding, it can efficiently capture long-short-term dependencies in user behavior sequences. Compared with traditional RNN models, it supports parallel computing and has higher computational efficiency and recommendation accuracy.

[0005] However, traditional self-attention models, due to their implicit positional encoding, fail to accurately model the sequential relationships between items. As the user sequence grows, attention weights become more dispersed, making it difficult to effectively capture important temporal patterns. This not only affects the model's ability to capture long-term dependencies but may also lead to the neglect of key item relationships, thus reducing recommendation effectiveness. Simultaneously, users may frequently receive recommendations for certain types of videos. This repetition can lead to dissatisfaction, create information cocoons, degrade user experience, and potentially cause user churn. Addressing these issues remains a challenge. Summary of the Invention

[0006] To address the technical problems of existing methods, such as inaccurate modeling of sequential relationships between items, difficulty in effectively capturing long-term dependencies, neglect of key item relationships, and frequent repetition of certain video types, this invention proposes a video recommendation method based on Sequence Recommendation (SeqTransRepeatNet). In the constructed SequenceRepeatNet model, positional encoding is decoupled to accurately model the sequential relationships between items; an interest-focusing block is added to focus on user interest information, enhancing the model's ability to capture long-term dependencies; cross-attention is used to associate user item information with sequence position information, thereby enhancing the understanding of context; and a repetition exploration mechanism module is introduced to help users perform repeated video recommendations at appropriate time points. This invention exhibits small recommendation errors, strong generalization ability, and can accurately recommend the user's next video to watch.

[0007] To achieve the above objectives, the technical solution of the present invention is implemented as follows:

[0008] A video recommendation method based on SeqTransRepeatNet includes the following steps:

[0009] S1: Obtain the dataset, preprocess the dataset, and divide the preprocessed dataset into training set, validation set and test set;

[0010] S2: Construct the SeqTransRepeatNet model, which includes a sequentially connected embedding module, a multi-layer cross-attention encoder module, and a repeat exploration recommendation module;

[0011] S3: Train the constructed SeqTransRepeatNet model using the training set and validation set to obtain the final trained SeqTransRepeatNet model;

[0012] S4: Use the test set to test and evaluate the finally trained SeqTransRepeatNet model and obtain the final recommendation list.

[0013] The embedding module described in step S2 includes an embedding layer, a normalization layer I, and a random deactivation layer I connected in sequence;

[0014] The multi-layer cross-attention encoder module includes multiple sequentially connected encoding layers. Each encoding layer includes a sequentially connected multi-head self-attention mechanism and a positionally fully connected feedforward network. The sequential multi-head self-attention mechanism of the first encoding layer is connected to the random deactivation layer I.

[0015] The repeated exploration recommendation module includes a repeated exploration mechanism, a repeated module, an exploration module, and a prediction layer. The input of the repeated exploration mechanism is connected to the fully connected feedforward network at the last coding layer. The output of the repeated exploration mechanism is connected to the repeated module and the exploration module, respectively. The outputs of the repeated module and the exploration module are connected and then connected to the prediction layer.

[0016] The sequence multi-head self-attention mechanism includes a parallel item encoding input layer and a position encoding input layer. The item encoding input layer and the position encoding input layer of the sequence multi-head self-attention mechanism in the first encoding layer are respectively connected to the random deactivation layer I. The item encoding input layer of the sequence multi-head self-attention mechanism in subsequent encoding layers is connected to the position fully connected feedforward network of the previous encoding layer. The position encoding input layer of the sequence multi-head self-attention mechanism in subsequent encoding layers is connected to the input of the position encoding input layer of the sequence multi-head self-attention mechanism in the previous encoding layer. The outputs of the item encoding input layer and the position encoding input layer are both connected to each attention head in the sequence multi-head self-attention mechanism. Each attention head includes a linear layer I, an interest focusing block, and a cross-scaling dot product attention module connected in sequence. The linear layer I of each attention head is connected to the outputs of the item encoding input layer and the position encoding input layer. The cross-scaling dot product attention module of each attention head is connected to the aggregation module. The aggregation module is connected to the normalization layer II. The output of the item encoding input layer is connected to the normalization layer II.

[0017] The positionally fully connected feedforward network comprises a linear layer II, an activation layer, a linear layer III, a random deactivation layer II, and a normalization layer III connected in sequence; the output of normalization layer I is connected to normalization layer III; the output of normalization layer III of the positionally fully connected feedforward network of the first coding layer is connected to the input of the item encoding input layer of the sequence multi-head self-attention mechanism of the subsequent coding layer; and the normalization layer III of the positionally fully connected feedforward network of the last coding layer is connected to the input of the repeated exploration mechanism.

[0018] The method for training the constructed SeqTransRepeatNet model using the training set and validation set in step S3 is as follows: Input the training set and validation set into the constructed SeqTransRepeatNet model respectively, and use the embedding module to train the video item sequence I from the training set or validation set.s Perform feature extraction to obtain the item code and location code;

[0019] In each encoding layer of the multilayer cross-attention encoder module, interest-focusing blocks and cross-scaling dot product attention modules are used to obtain attention weight information for item encoding and position encoding. The attention weight information for item encoding and position encoding is calculated using a position fully connected feedforward network. After calculation through multiple encoding layers, the output sequence of the multilayer cross-attention encoder module is finally obtained.

[0020] In the repeated exploration recommendation module, the output prediction probability is calculated based on the output sequence of the multi-layer cross-attention encoder module;

[0021] The output prediction probability is maximized using the log-likelihood loss function based on the output prediction probability. The Adam optimizer is used to optimize the loss function. The optimal hyperparameters for each model are found through grid search. Each model is trained using an early stopping strategy.

[0022] The method for obtaining the project code is as follows: using a hidden dimension of d... model The video item sequence I is embedded through the item embedding operation of the embedding layer. s =i1,i2,…,i τ ,…,i t , The encoding is a dense latent vector representation, which is then passed through a normalization layer I and a random deactivation layer to obtain the item code E. item =e1,e2,…,e i ,…,e t , t is the sequence length;

[0023] The method for obtaining the position code is as follows: An integer sequence from 0 to t-1 is created as a unique identifier for each position. Then, the position identifier is converted into a high-dimensional embedding vector through a position embedding operation in the embedding layer. After passing through a normalization layer I and a random deactivation layer, the high-dimensional embedding vector yields the position code E. pos =p1,p2,…,p i ,…,p t ,

[0024] The method for obtaining attention weight information of item code and position code using interest-focusing blocks and cross-scaling dot product attention modules is as follows: the item code is input into the linear layer I, and the item code E is input into the layer. item The data is transformed into a project query Q, a project key K, and a project value V, and then the position code E input to the position code input layer is passed through linear layer I. posThe values ​​are converted into location queries Qp and location keys Kp. The item key K, item value V, and location key Kp are then input into the interest focusing block for calculation to obtain new item key K1, new item value V1, and new location key Kp1. The attention weights for items and locations are calculated using the cross-scaling dot product attention module and summed to obtain the attention weight information for each attention head. The attention weight information of multiple heads of the sequence multi-head self-attention mechanism of each encoding layer is concatenated by the aggregation module to obtain the multi-head attention weight MultiHead(Q,K,V). The multi-head attention weight MultiHead(Q,K,V) is then residually concatenated with the item encoding and normalized to obtain the attention weight information X2 for the item encoding and location encoding.

[0025] The method for inputting the input into the interest-focusing block for calculation is as follows: given an input embedding matrix Define parameter matrix Where d model Here, k is the embedding dimension, k is the number of potential interests, and D is the correlation distribution matrix between items and interests. matrix :

[0026] D matrix =Softmax(X·θ)

[0027] Where Softmax is the activation function;

[0028] Then, the correlation distribution matrix D between projects and interests is analyzed using the Einstein summation convention. matrix The interest block matrix is ​​obtained by aggregating the input embedding matrix X with the interest block matrix.

[0029] X1 = Einsum(X,D) matrix )

[0030] Where Einsum() represents the Einstein summation convention function;

[0031] The input embedding matrix X consists of item key K, item value V, and position key Kp; the obtained interest block matrix X1 consists of new item key K1, new item value V1, and new position key Kp1.

[0032] The method for calculating the attention weights related to the item and the attention weights related to the position using the cross-scaling dot product attention module and then summing them to obtain the attention weight information for each attention head is as follows:

[0033] Calculate attention weights for the project:

[0034]

[0035] Calculate the attention weights with respect to location:

[0036]

[0037] Summing is performed to obtain the attention weight information for each attention head. i :

[0038]

[0039] Where h is the number of attention heads, W i Q , W i V Let represent the weight matrices of the i-th attention head;

[0040] The method for splicing is as follows:

[0041] MultiHead(Q,K,V)=Concat(head1,..,head h W o

[0042] Where Concat is the concatenation operation, W o Let be a linear transformation matrix.

[0043] The method for calculating the attention weight information of item encoding and location encoding using a position-fully connected feedforward network is as follows: the attention weight information X2 of item encoding and location encoding is calculated by a fully connected neural network, then passed through a random deactivation layer, and finally residually connected and normalized with the attention weight information X2 of item encoding and location encoding to obtain the output sequence H of each encoding layer. l , l represents the l-th coding layer;

[0044] The computation method of a fully connected neural network is as follows:

[0045] x = X²W₁ + b₁

[0046] FFN(X2)=Φ(x)W2+b2

[0047] Where Φ(x) is the activation function, W1 and W2 are weights, and b1 and b2 are biases;

[0048] The output sequence of the multi-layer cross-attention encoder module is: H L ={h1,h2,...,h τ ,…,h t}, where L is the number of encoder layers, and h τ Let τ be the τth hidden state.

[0049] The method for calculating the output prediction probability based on the output sequence of the multi-layer cross-attention encoder module in the repeated exploration recommendation module is as follows:

[0050] The repeated exploration mechanism outputs sequence H using a multi-layer cross-attention encoder module. L For input, use the last hidden state h. t With each hidden state h τ Matching to obtain the importance score of repeated exploration Importance score for repeated exploration Normalization calculation is performed to obtain the attention weights for repeated exploration. For each hidden state h τ The video project sequence I is obtained by performing a weighted summation. s Context vector Then, the softmax activation function is used to process the video item sequence I. s Context vector The probability distribution of the switching mode corresponds to the probability P(r|I) of executing the repeating mode. s ) and the probability P(e|I) of executing in exploration mode s );

[0051] The repetition module uses the repetition recommendation decoder to compute recommended item i in the repetition pattern. j The probability P(i) j |r,I s );

[0052] The exploration module uses the exploration recommendation decoder to calculate recommended item i in exploration mode. j The probability P(i) j |e,I s );

[0053] Based on the probability P(r|I) of executing in the repetitive mode s The probability P(e|I) of executing in exploration mode s Recommended items i in repeat mode j The probability P(i) j |r,I s ) and recommended items in exploration mode i j The probability P(i) j |e,I s ), calculate the probability P(i) of generating the next recommended video. j |I s ).

[0054] The importance score of repeated exploration The calculation method is as follows:

[0055]

[0056] Among them, V re W re and U re Here, Φ is the weight parameter, Φ() is the activation function, and r and e represent the repetition mode and exploration mode, respectively; the repetition exploration attention weight... And video project sequence I s Context vector The calculation method is as follows:

[0057]

[0058] The probability P(r|I) of executing in the repetitive mode s ) and the probability P(e|I) of executing in exploration mode s The calculation method for ) is as follows:

[0059]

[0060] in, It is a weight matrix;

[0061] The calculation of the probability P(i) of recommending item i in the repetitive pattern. j |r,I s The method is as follows:

[0062]

[0063] Among them, V r W r and U r These are the weight parameters, and Φ() is the activation function. It is the score of repetition importance. Indicates item i j ∈I s The total number of occurrences, where I represents the sequence of all video items, and P(i j |r,I s ) represents the probability of recommending item i in the repeat pattern;

[0064] The calculation of the probability P(i|e,I) of recommending item i in exploration mode. s The method is as follows:

[0065]

[0066] in, It is the weight matrix; V e W e and U e Here are the parameters, and Φ() is the activation function. It explores the importance score. This represents the attention state in exploration mode, and then the last hidden state h is... τ and attention state Combined into video project sequence I s Hybrid representation P(i j |e,I s ) indicates that item i is recommended in exploration mode. j probability;

[0067] The probability P(i) of generating the next recommended video is calculated. j |I s The method is as follows:

[0068] P(i j |I s )=P(r|I s )P(i j |r,I s )+P(e|I s )P(i j |e,I s )

[0069] P(i j |I s ) indicates that in a given video item sequence I s Recommended project i j probability;

[0070] The log-likelihood loss function is:

[0071]

[0072] Where θ represents all parameters of the SeqTransRepeatNet model.

[0073] The beneficial effects of this invention are as follows:

[0074] 1) In order to solve the problem that the implicit positional encoding information of the self-attention model cannot accurately model the sequential relationship between items, the positional encoding is decoupled and put into a multi-layer cross-attention module along with the sequence information for self-attention learning, which can more accurately model the sequential relationship between items.

[0075] 2) To address the problem that the model struggles to effectively capture long-term dependencies, an interest-focusing block is added to focus on the user's interest information, thereby enhancing the model's ability to capture long-term dependencies.

[0076] 3) To address the issue of the model neglecting key project relationships, cross-attention is used to associate user project information with sequence position information, thereby enhancing the understanding of the context.

[0077] 4) To address user dissatisfaction caused by duplicate recommendations, a duplicate exploration mechanism module was introduced to help users perform duplicate video recommendations at appropriate times. Attached Figure Description

[0078] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0079] Figure 1 This is a diagram of the overall architecture of the SeqTransRepeatNet model of the present invention.

[0080] Figure 2 This is a diagram of the cross-scaling dot product attention module of the present invention.

[0081] Figure 3 Recommended module diagram for repeated exploration in this invention. Detailed Implementation

[0082] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0083] A video recommendation method based on Sequence Recommendation SeqTransRepeatNet, such as Figure 1 As shown, it includes the following steps:

[0084] S1: Obtain the dataset, preprocess the dataset, and divide the preprocessed dataset into training set, validation set, and test set.

[0085] Dataset Acquisition: This example was conducted on three real-world and widely used video datasets: MovieLens 100K, MovieLens 1M, and KuaiRec. These datasets differ significantly in terms of data volume and sparsity. MovieLens is a well-known movie recommendation system dataset created by GroupLens Research Labs, with two widely used versions: MovieLens 100K and MovieLens 1M. KuaiRec is a real-world dataset collected from recommendation logs of the video-sharing mobile application Kuaishou.

[0086] Preprocessing: All three datasets followed the same data preprocessing procedure. In this embodiment, each dataset was transformed into an implicit dataset by treating each rating or comment in the dataset as an interaction between the user and the item. The interactions were then grouped by user ID and all groups were sorted by timestamp to form a sequence of video items for each user, discarding short sequences with fewer than 5 interactions. Table 1 shows the statistics for the preprocessed datasets.

[0087] Table 1. Statistical data of the preprocessed dataset

[0088]

[0089] Where #User represents the number of users, #Items represents the number of videos, #Inters represents the number of interactions, #Avgu represents the average number of operations per user, #Avgi represents the average number of operations per item, and Sparsity represents the sparsity.

[0090] The three datasets used are frequently used as public benchmark datasets, and they differ in size and density, making it convenient to test the model on different types of datasets.

[0091] Dataset partitioning: To evaluate the performance of each method, a leave-one-out partitioning method was used for each user's video item sequence, retaining the last video item for testing and the second-to-last video item for validation. The remaining video items were used for training. Each real video item in the test set was paired with 100 video items that did not involve user interaction (negative samples) using uniform sampling, which more closely reflects real-world scenarios.

[0092] S2: Construct the SeqTransRepeatNet model, which includes a sequentially connected embedding module, a multi-layer cross-attention encoder module, and a repeat exploration recommendation module;

[0093] The embedding module is used for the user's video project sequence I s Perform feature extraction to obtain the item code used to represent item embedding information and the position code used to represent sequence embedding position information;

[0094] The multi-layer cross-attention encoder module adds an interest-focusing block to the Transformer decoder to focus on the user's interest information. In each encoding layer, the interest-focusing block and the cross-scaling dot product attention module are used to obtain the attention weight information of item encoding and position encoding, thereby enhancing the understanding of the context. The position fully connected feedforward network is used to calculate the attention weight information of item encoding and position encoding.

[0095] Repeated exploration recommendation model helps the model calculate the probability of selecting a repeated pattern or an exploration pattern, and makes video recommendations at appropriate times.

[0096] The embedding module includes an embedding layer, a normalization layer I, and a random deactivation layer I connected in sequence.

[0097] Attention modules have received widespread attention and research in the field of sequential recommendation. Self-attention models allow point-to-point feature interactions within a sequence of video items, solving the global dependency problem and allowing the use of longer sequences and more information. In this algorithm, the attention module of the model is the focus. The Transformer network architecture mainly consists of an encoder and a decoder, each composed of multiple stacked identical layers. Each layer consists of two main sub-layers: a multi-head self-attention mechanism and a feedforward neural network. This invention mainly introduces its encoder structure, which consists of multiple stacked identical encoder layers, each containing a multi-head self-attention mechanism and a simple positionally fully connected feedforward network. The multi-head self-attention mechanism includes a self-attention mechanism and a multi-head mechanism. The self-attention mechanism generates attention weights by calculating the query, key, and value vectors for each element in the input sequence. These weights are used to weighted sum the values ​​of all input elements, thereby capturing the dependencies between elements in the sequence. The multi-head mechanism captures information from different subspaces through multiple independent self-attention heads and concatenates their outputs together.

[0098] The multi-layer cross-attention encoder module includes multiple encoding layers. Each encoding layer includes a sequential multi-head self-attention mechanism and a position-fully connected feedforward network connected in sequence. The sequential multi-head self-attention mechanism of the first encoding layer is connected to the random deactivation layer I. The sequential multi-head self-attention mechanism includes a parallel item encoding input layer and a position encoding input layer. The item encoding input layer and the position encoding input layer are only used for the input of item encoding and position encoding, and no operation is performed there. The item encoding input layer and the position encoding input layer of the sequential multi-head self-attention mechanism of the first encoding layer are respectively connected to the random deactivation layer I. The item encoding input layer of the sequential multi-head self-attention mechanism of subsequent encoding layers is connected to the position-fully connected feedforward network of the previous encoding layer, and the position encoding input layer of the sequential multi-head self-attention mechanism of subsequent encoding layers is connected to the input of the position encoding input layer of the sequential multi-head self-attention mechanism of the previous encoding layer. The output of the item encoding input layer and the position encoding input layer are connected. The outputs of each input layer are connected to each attention head in the sequence multi-head self-attention mechanism. Each attention head includes a linear layer I, an interest focusing block, and a cross-scaling dot product attention module connected in sequence. The linear layer I of each attention head is connected to the outputs of the item encoding input layer and the position encoding input layer. The cross-scaling dot product attention module of each attention head is connected to the aggregation module. The aggregation module is connected to the normalization layer II. The output of the item encoding input layer is connected to the normalization layer II. The position fully connected feedforward network includes a linear layer II, an activation layer, a linear layer III, a random deactivation layer II, and a normalization layer III connected in sequence. The output of the normalization layer I is connected to the normalization layer III. The output of the normalization layer III of the position fully connected feedforward network of the first encoding layer is connected to the input of the item encoding input layer of the sequence multi-head self-attention mechanism in the subsequent encoding layers. The normalization layer III of the position fully connected feedforward network of the last encoding layer is connected to the input of the repeated exploration mechanism.

[0099] Repeated recommendations refer to the method of recommending the same or similar content to users multiple times. The handling of repeated recommendations is crucial for the effectiveness evaluation of recommendation systems and user experience. It is necessary to comprehensively consider the diversity, novelty and personalization of recommendations in order to avoid over-recommending the same content and causing user fatigue or dissatisfaction.

[0100] The repeated exploration recommendation module includes a repeated exploration mechanism, a repeated module, an exploration module, and a prediction layer. The input of the repeated exploration mechanism is connected to a fully connected feedforward network at the last encoding layer. The output of the repeated exploration mechanism is connected to both the repeated module and the exploration module. The outputs of the repeated module and the exploration module are then connected to the prediction layer, which outputs a recommendation list based on the probability of the next recommended video. The model recommends videos from the user's history in the repeated module and recommends new videos to the user in the exploration module.

[0101] S3: Train the constructed SeqTransRepeatNet model using the training set and validation set to obtain the final trained SeqTransRepeatNet model;

[0102] The method for training the constructed SeqTransRepeatNet model using training and validation sets is as follows: The training and validation sets are input into the constructed SeqTransRepeatNet model respectively, and the embedding module is used to train the user's video item sequence I from the training or validation set. s Perform feature extraction to obtain the item code and location code;

[0103] The method for obtaining the project code is as follows: using a hidden dimension of d... model The video item sequence I is embedded through the item embedding operation of the embedding layer. s =i1,i2,…,i τ ,…,i t , The encoding is performed as a dense latent vector representation to reduce computation and improve representation learning. The dense latent vector representation is passed through a normalization layer I and a random deactivation layer to obtain the item encoding E. item =e1,e2,…,e i ,…,e t , t is the sequence length, i τ It refers to a video project. Video Project Sequence I s This represents the set of all video items a user has viewed in their history; it's a sequence containing all the videos a user has watched. s | Represents video project sequence I s The total number of videos within;

[0104] Traditional self-attention-based recommendation algorithms directly encode items into E. item and position code E pos The mixed introduction of sequence relations, i.e. (E item +E pos However, such forced fusion cannot effectively distinguish between items and locations, potentially limiting the model's ability to capture sequence relationships. Therefore, this invention decouples location encoding and models it simultaneously with item encoding, thereby better capturing the potential connections between items.

[0105] The method for obtaining the position code is as follows: An integer sequence from 0 to t-1 is created as a unique identifier for each position. Then, the position identifier is converted into a high-dimensional embedding vector through a position embedding operation in the embedding layer to capture the semantic information of the position. After the high-dimensional embedding vector passes through a normalization layer I and a random deactivation layer, the position code E is obtained. pos =p1,p2,…,p i ,…,p t ,

[0106] In each encoding layer of the multi-layer cross-attention encoder module, attention weight information for item codes and position codes is obtained using interest-focusing blocks and cross-scaling dot product attention modules: the item code E is input to the linear layer I. item The data is transformed into a project query Q, a project key K, and a project value V, and then the position code E input to the position code input layer is passed through linear layer I. pos Convert the data into location query Qp and location key Kp, and input the item key K, item value V, and location key Kp into the interest focus block for calculation to obtain the new item key K1, new item value V1, and new location key Kp1;

[0107] Assuming that most users have no more than t categories in their historical interests, where t is a configurable hyperparameter that can take values ​​of [5, 10, 15, 20, 25], then the video item sequence I can be... s Aggregates into potential interest blocks. For example, the number of video project sequences | I s If we set the number of historical video items to 50 and the number of interest blocks to 20, we can transform these 50 items into 20 related interest blocks. This reduces the amount of computation.

[0108] The method for inputting the input into the interest-focusing block for calculation is as follows: given an input embedding matrix Define parameter matrix Where d model Here, k is the embedding dimension, k is the number of potential interests, and D is the correlation distribution matrix between items and interests. matrix This represents the weight distribution of each project for different potential interests:

[0109] D matrix =Softmax(X·θ)

[0110] Where Softmax is the activation function;

[0111] Then, the correlation distribution matrix D between projects and interests is analyzed using the Einstein summation convention. matrix Aggregate with the embedding matrix X to obtain the interest block matrix.

[0112] X1 = Einsum(X,D) matrix )

[0113] Where Einsum() represents the Einstein summation convention function;

[0114] The input embedding matrix X consists of item key K, item value V, and position key Kp; the obtained interest block matrix X1 consists of new item key K1, new item value V1, and new position key Kp1.

[0115] like Figure 2 As shown, the attention weights of items and positions are calculated using the cross-scaling dot product attention module and summed to obtain the attention weight information of each attention head. The attention weight information of multiple heads of the sequence multi-head self-attention mechanism of each coding layer is concatenated by the aggregation module to obtain the multi-head attention weight MultiHead(Q,K,V).

[0116] In this embodiment, two cross-scaled dot product attention blocks are used to calculate different attention weights simultaneously, and cross attention is used to associate different information, thereby enhancing the understanding of the context.

[0117] Calculate the attention weights for the items:

[0118]

[0119] Calculate the attention weights with respect to location:

[0120]

[0121] Summing is performed to obtain the attention weight information for each attention head. i :

[0122]

[0123] Where h is the number of attention heads, W i Q , W i V Let represent the weight matrices of the i-th attention head;

[0124] The method for splicing is as follows:

[0125] MultiHead(Q,K,V)=Concat(head1,..,head h W o

[0126] Where Concat is the concatenation operation, W oLet be a linear transformation matrix.

[0127] The attention weights MultiHead(Q,K,V) are residually concatenated with the item codes and then normalized to obtain the attention weight information X2 for the item codes and position codes.

[0128] Attention weights for item and location encodings are calculated using a position-fully connected feedforward network: The attention weights X2 for item and location encodings are processed by the fully connected neural network, then passed through a random deactivation layer, and finally residually connected to and normalized with the attention weights X2 for item and location encodings to obtain the output sequence H of each encoding layer. l , l represents the l-th coding layer;

[0129] The method for calculating the fully connected neural network is as follows:

[0130] x = X²W₁ + b₁

[0131] GeLU: erf is the error function

[0132] ReLU: Φ = max(0,x)

[0133] Swish:

[0134] Tanh::

[0135] Sigmoid:

[0136] FFN(X2)=ΦW2+b2

[0137] W1 and W2 are weights, and b1 and b2 are biases;

[0138] The positionally fully connected feedforward neural network is a fully connected neural network FFN consisting of two linear transformations and a function hidden_act that encapsulates five activation functions (GeLU, ReLU, Swish, Tanh, and Sigmoid). The activation function Φ is an adjustable parameter.

[0139] After calculations across multiple coding layers, the final output sequence of the multi-layer cross-attention encoder module is obtained: H L ={h1,h2,...,h τ ,…,h t}, where L is the number of encoder layers, and h τ Let τ be the τth hidden state.

[0140] like Figure 3As shown, finally, the cross-attention encoder module will process the video item sequence I. s Encoded as output sequence H L ={h1,h2,...,h τ ,…,h t}where h τ This represents the hidden state at timestamp τ. The repeated exploration mechanism uses the output sequence H of a multi-layer cross-attention encoder module. L Given the input, predict the probability of executing either the repetitive or exploratory mode. The repetitive recommendation decoder uses the output sequence H of the multi-layer cross-attention encoder module. L As input, predict video item sequence I s The probability of repeated recommendations for clicked items. Exploring the recommendation decoder using the output sequence H of a multi-layer cross-attention encoder module. L As input, predict the video item sequence (II). s The probability of exploring and recommending unclicked items in the sequence is given by I, where I represents the sequence of all video items.

[0141] In the repeated exploration recommendation module, the output prediction probability is calculated based on the output sequence of the multi-layer cross-attention encoder module to help users perform video recommendations at appropriate time points. The repeated exploration mechanism can be viewed as a binary classifier, which calculates the output prediction probability based on the output sequence H of the multi-layer cross-attention encoder module. L ={h1,h2,...,h τ ,…,h t This is used to predict recommendation patterns. The specific steps are as follows:

[0142] The repeated exploration mechanism uses a multi-layer cross-attention encoder module to output sequence H. L For input, use the last hidden state h. t With each hidden state h τ Matching to obtain the importance score of repeated exploration

[0143]

[0144] Among them, V re W re and U re Φ is the weight parameter, Φ is the activation function, and r and e represent the repeat mode and the exploration mode, respectively.

[0145] Importance score for repeated exploration Normalization calculation is performed to obtain the attention weights for repeated exploration.

[0146]

[0147] For each hidden state hτ The video project sequence I is obtained by performing a weighted summation. s Context vector

[0148]

[0149] Then, the softmax activation function is used to process the video item sequence I. s Context vector The probability distribution of the conversion mode corresponds to the following:

[0150] The probability of repeating the pattern P(r|I) s ) and the probability P(e|I) of executing in exploration mode s ):

[0151]

[0152] in, It is a weight matrix;

[0153] The repeat module uses the repeat recommendation decoder to evaluate the re-clicked video item sequence I. s The probability of a middle item is calculated, and the recommended item i is determined under the repetition pattern. j The probability P(i) j |r,I s ):

[0154]

[0155] Where V r W r and U r Here, Φ is the weight parameter, and Φ is the activation function. It is the score of repetition importance. Indicates item i j ∈I s The total number of occurrences, where I represents the sequence of all video items, and P(i j |r,I s ) indicates that item i is recommended in repeat mode. j probability;

[0156] The exploration module uses the exploration recommendation decoding evaluation click video item sequence I s Given the probability of a new item not existing in the current database, calculate the probability P(i) of recommending item i in exploration mode. j |e,I s ):

[0157]

[0158] in, It is the weight matrix; where V e We and U e Here, Φ is the parameter, and Φ is the activation function. It explores the importance score. This represents the attention state in exploration mode, and then the last hidden state h is... τ and attention state Combined into video project sequence I s Hybrid representation P(i j |e,I s ) indicates that item i is recommended in exploration mode. j The probability, since exp(-∞)=0, is given by the probability that an item exists in the video item sequence I. s If it is in the middle, then the probability of it being recommended in the exploration module is 0.

[0159] Based on the probability P(r|I) of executing in the repetitive mode s The probability P(e|I) of executing in exploration mode s Recommended items i in repeat mode j The probability P(i) j |r,I s ) and the probability P(i) of recommending item i in exploration mode. j |e,I s ) Calculate the probability P(i) of generating the next recommended video. j |I s ):

[0160] P(i j |I s )=P(r|I s )P(i j |r,I s )+P(e|I s )P(i j |e,I s )

[0161] P(i j |I s ) indicates that in a given video item sequence I s Recommended project i j The probability of.

[0162] The prediction layer determines the probability P(i) of the next recommended video. j |I s Output a recommended list.

[0163] Maximize the output prediction probability for a given input sequence using the log-likelihood loss function based on the output prediction probability:

[0164]

[0165] Where θ represents all parameters of the SeqTransRepeatNet model.

[0166] The Adam optimizer is used to optimize the loss function, with a learning rate of lr∈{0.01,0.005,0.001,0.0005,0.0001} for the hyperparameter. A normal distribution is used for parameter initialization, with a mean of 0 and a standard deviation of 0.02. The weights of the normalized layers are initialized to 1.0, and the biases are initialized to 0; the biases of the linear layers are initialized to 0.

[0167] The model is implemented in PyTorch and uses a grid search to find the optimal parameters for each model; the hidden dimension d is selected. model The grid search is performed using the following parameters: ∈{16,32,64,96,128}; dropout rate d∈{0.2,0.3,0.4,0.5}; activation function type Φ∈{′GeLU′,′ReLU′,′Swish′,′TanH′,′Sigmoid′}; number of latent interests k∈{5,10,15,20,25}; learning rate lr∈{0.01,0.005,0.001,0.0005,0.0001}; number of attention heads h∈{2,4,8}. For the STRN model, the optimal hyperparameters for each baseline are:

[0168] ml-100k: {d model =32, d=0.4, L=2, Φ='RELU', k=15, lr=0.01, h=8}

[0169] ml-1m:{d model =96, d=0.4, L=2, Φ=′GELU′, k=25, lr=0.0005, h=8}

[0170] KuaiRec:{d model =128, d=0.2, L=2, Φ=′Tanh′, k=25, lr=0.001, h=2}

[0171] To ensure fairness, a unified video project sequence I is set up. s The maximum sequence length is 50, the number of layers in the multi-layer cross-attention encoder module is L=2, and grid search is used to search for the optimal result.

[0172] An early stopping strategy is used to train each model, which terminates early if the validation accuracy does not improve within 10 epochs. All models are trained on a single NVIDIA GeForce RTX 3090 GPU.

[0173] S4: Use the test set to test and evaluate the finally trained SeqTransRepeatNet model and obtain the final recommendation list.

[0174] To score the ranking list, this invention employs three common evaluation metrics: Recall, Mean Reciprocal Ranking (MRR), and Normalized Discounted Cumulative Gain (NDCG), which are explained below:

[0175] Recall measures the percentage of items a user actually likes among all relevant items, and its formula is shown below.

[0176]

[0177] Where U represents the user set, Let R(u) represent the list of the top K recommendations for user u, and let R(u) represent the set of items that user u actually likes.

[0178] The mean reciprocal rank (MRR) measures the rank of the first relevant item in the recommendation results, and its formula is shown below.

[0179]

[0180] Where U represents the user set, This indicates the ranking position of user u's first relevant item in the recommendation list.

[0181] Normalized Discounted Cumulative Gain (NDCG) is a metric that measures the quality of rankings, where positions are discounted logarithmically. It explains the position of a hit by assigning higher scores to hits with higher rankings, and its formula is as follows.

[0182]

[0183] Where U represents the set of users, and pi represents the position of user i's target item in the recommendation list. The higher the position, the larger pi is.

[0184] When the true item ranks highly in the top-K list (i.e., the recommendation list), Recall@K, MRR@K, and NDCG@K all have large values, where K = (5, 10).

[0185] To verify the superior performance of the proposed method, nine typical recommendation algorithms were selected as baseline models, and the results of all baseline models on three datasets are validated in Table 2. STRN is the recommendation algorithm proposed for the text.

[0186] GRU4Rec: A conversational recommendation system based on recurrent neural networks.

[0187] NextItNet: A simple convolutional generative network for recommending next projects.

[0188] •SASRec: A sequence recommendation model based on Transformer and using a self-attention mechanism.

[0189] Bert4Rec: A sequence recommendation model based on bidirectional Transformer.

[0190] LightSANs: A low-rank decomposition self-attention network for the next recommendation.

[0191] FEARec: A frequency-enhanced hybrid attention network for sequence recommendation.

[0192] • AdaMCT: A CNN-Transformer adaptive hybrid network for sequence recommendation.

[0193] SINE: A sparse interest network for sequence recommendation.

[0194] CORE: A simple and efficient session-based recommender system with a unified representation space.

[0195] Table 2 Performance Comparison of Different Models

[0196]

[0197]

[0198] The bolded fractions are the best fractions in each row, and the underlined fractions are the second best fractions.

[0199] Table 2 shows the recommendation performance of the model of this invention on three datasets and summarizes the best results of all baseline models on the three datasets. In the ml-100k dataset, SASRec and LightSANs, represented by attention mechanisms, effectively capture long-range dependencies in user behavior through powerful attention mechanisms, thus performing outstandingly on the ml-100k dataset. In the ml-1m dataset, GRU4Rec, represented by RNN, leverages its powerful temporal information modeling capabilities to slightly outperform SASRec. For the relatively sparse KuaiRec dataset, the CORE model significantly improves the recommendation effect by constructing a unified representation space, especially in the case of sparse data, demonstrating strong adaptability and robustness.

[0200] For STRN, state-of-the-art results were achieved on each dataset, with average improvements of 13.12%, 4.61%, and 15.62% respectively across the three datasets. This result demonstrates that the STRN model not only performs exceptionally well on dense datasets but also exhibits strong adaptability and superior performance on sparse datasets. This further proves the broad applicability and effectiveness of the method in various scenarios, providing new directions and possibilities for future research and applications in video recommendation.

[0201] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A video recommendation method based on Sequence Recommendation (SeqTransRepeatNet), characterized in that, Includes the following steps: S1: Obtain the dataset, preprocess the dataset, and divide the preprocessed dataset into training set, validation set and test set; S2: Construct the SeqTransRepeatNet model, which includes a sequentially connected embedding module, a multi-layer cross-attention encoder module, and a repeat exploration recommendation module; S3: Train the constructed SeqTransRepeatNet model using the training set and validation set to obtain the final trained SeqTransRepeatNet model; S4: Use the test set to test and evaluate the finally trained SeqTransRepeatNet model and obtain the final recommendation list; The method for training the constructed SeqTransRepeatNet model using the training set and validation set in step S3 is as follows: Input the training set and validation set into the constructed SeqTransRepeatNet model respectively, and use the embedding module to train the video item sequence I from the training set or validation set. s Perform feature extraction to obtain the item code and location code; In each encoding layer of the multilayer cross-attention encoder module, interest-focusing blocks and cross-scaling dot product attention modules are used to obtain attention weight information for item encoding and position encoding. The attention weight information for item encoding and position encoding is calculated using a position fully connected feedforward network. After calculation through multiple encoding layers, the output sequence of the multilayer cross-attention encoder module is finally obtained. In the repeated exploration recommendation module, the output prediction probability is calculated based on the output sequence of the multi-layer cross-attention encoder module; Based on the output prediction probability, the log-likelihood loss function is used to maximize the output prediction probability of a given input sequence. The Adam optimizer is used to optimize the loss function. The best hyperparameters for each model are found through grid search. Each model is trained using an early stopping strategy. The method for calculating the output prediction probability based on the output sequence of the multi-layer cross-attention encoder module in the repeated exploration recommendation module is as follows: The repeated exploration mechanism outputs sequence H using a multi-layer cross-attention encoder module. L For input, use the last hidden state h. t With each hidden state h τ Matching to obtain the importance score of repeated exploration Importance score for repeated exploration Normalization calculation is performed to obtain the attention weights for repeated exploration. For each hidden state h τ The video project sequence I is obtained by performing a weighted summation. s Context vector Then, the softmax activation function is used to process the video item sequence I. s Context vector The probability distribution of the switching mode corresponds to the probability P(r|I) of executing the repeating mode. s ) and the probability P(e|I) of executing in exploration mode s The repetition module uses the repetition recommendation decoder to compute recommended item i in the repetition pattern. j The probability P(i) j |r,I s The exploration module uses the exploration recommendation decoder to calculate recommended item i in exploration mode. j The probability P(i) j |e,I s ); Based on the probability P(r|I) of executing in the repetitive mode s The probability P(e|I) of executing in exploration mode s Recommended items i in repeat mode j The probability P(i) j |r,I s ) and recommended items in exploration mode i j The probability P(i) j |e,I s ), calculate the probability P(i) of generating the next recommended video. j |I s ).

2. The video recommendation method based on Sequence Recommendation SeqTransRepeatNet according to claim 1, characterized in that, The embedding module described in step S2 includes an embedding layer, a normalization layer I, and a random deactivation layer I connected in sequence; The multi-layer cross-attention encoder module includes multiple sequentially connected encoding layers. Each encoding layer includes a sequentially connected multi-head self-attention mechanism and a positionally fully connected feedforward network. The sequential multi-head self-attention mechanism of the first encoding layer is connected to the random deactivation layer I. The repeated exploration recommendation module includes a repeated exploration mechanism, a repeated module, an exploration module, and a prediction layer. The input of the repeated exploration mechanism is connected to the fully connected feedforward network at the last coding layer. The output of the repeated exploration mechanism is connected to the repeated module and the exploration module, respectively. The outputs of the repeated module and the exploration module are connected and then connected to the prediction layer.

3. The video recommendation method based on Sequence Recommendation SeqTransRepeatNet according to claim 2, characterized in that, The sequence multi-head self-attention mechanism includes a parallel item encoding input layer and a position encoding input layer. The item encoding input layer and the position encoding input layer of the sequence multi-head self-attention mechanism in the first encoding layer are respectively connected to the random deactivation layer I. The item encoding input layer of the sequence multi-head self-attention mechanism in subsequent encoding layers is connected to the position fully connected feedforward network of the previous encoding layer. The position encoding input layer of the sequence multi-head self-attention mechanism in subsequent encoding layers is connected to the input of the position encoding input layer of the sequence multi-head self-attention mechanism in the previous encoding layer. The outputs of the item encoding input layer and the position encoding input layer are both connected to each attention head in the sequence multi-head self-attention mechanism. Each attention head includes a linear layer I, an interest focusing block, and a cross-scaling dot product attention module connected in sequence. The linear layer I of each attention head is connected to the outputs of the item encoding input layer and the position encoding input layer. The cross-scaling dot product attention module of each attention head is connected to the aggregation module. The aggregation module is connected to the normalization layer II. The output of the item encoding input layer is connected to the normalization layer II. The location-fully connected feedforward network includes a linear layer II, an activation layer, a linear layer III, a random deactivation layer II, and a normalization layer III connected in sequence. The output of normalized layer I is connected to normalized layer III; the output of normalized layer III of the positionally fully connected feedforward network of the first coding layer is connected to the input of the item coding input layer of the sequential multi-head self-attention mechanism of the subsequent coding layers; and the normalized layer III of the positionally fully connected feedforward network of the last coding layer is connected to the input of the repeated exploration mechanism.

4. The video recommendation method based on Sequence Recommendation SeqTransRepeatNet according to claim 3, characterized in that, The method for obtaining the project code is as follows: using a hidden dimension of d... model The video item sequence I is embedded through the item embedding operation of the embedding layer. s =i1,i2,…,i τ ,…,i t , The encoding is a dense latent vector representation, which is then passed through a normalization layer I and a random deactivation layer to obtain the item code E. item =e1,e2,…,e i ,…,e t , t is the sequence length; The method for obtaining the position code is as follows: An integer sequence from 0 to t-1 is created as a unique identifier for each position. Then, the position identifier is converted into a high-dimensional embedding vector through a position embedding operation in the embedding layer. After passing through a normalization layer I and a random deactivation layer, the high-dimensional embedding vector yields the position code E. pos =p1,p2,…,p i ,…,p t , 5. The video recommendation method based on Sequence Recommendation SeqTransRepeatNet according to claim 4, characterized in that, The method for obtaining attention weight information of item code and position code using interest-focusing blocks and cross-scaling dot product attention modules is as follows: The item code E input to the input layer is passed through linear layer I. item The data is transformed into a project query Q, a project key K, and a project value V, and then the position code E input to the position code input layer is passed through linear layer I. pos Convert the data into a location query Qp and a location key Kp, and input the item key K, item value V, and location key Kp into the interest focus block for calculation to obtain a new item key K1, a new item value V1, and a new location key Kp1; then use... The cross-scaling dot product attention module calculates the attention weights for items and the attention weights for positions, and sums them to obtain the attention weight information for each attention head. The aggregation module concatenates the attention weight information of multiple heads of the sequence multi-head self-attention mechanism of each encoding layer to obtain the multi-head attention weight MultiHead(Q,K,V). The multi-head attention weight MultiHead(Q,K,V) is then residually concatenated with the item encoding and normalized to obtain the attention weight information X2 for the item encoding and the position encoding.

6. The video recommendation method based on Sequence Recommendation SeqTransRepeatNet according to claim 5, characterized in that, The method for inputting the input into the interest-focusing block for calculation is as follows: given an input embedding matrix Define parameter matrix Where d model Here, k is the embedding dimension, k is the number of potential interests, and D is the correlation distribution matrix between items and interests. matrix : D matrix =Softmax(X·θ) Where Softmax is the activation function; Then, the correlation distribution matrix D between projects and interests is analyzed using the Einstein summation convention. matrix The interest block matrix is ​​obtained by aggregating the input embedding matrix X with the interest block matrix. X1=Einsum(X,D matrix ) Where Einsum() represents the Einstein summation convention function; The input embedding matrix X consists of item key K, item value V, and position key Kp; the obtained interest block matrix X1 consists of new item key K1, new item value V1, and new position key Kp1.

7. The video recommendation method based on Sequence Recommendation SeqTransRepeatNet according to claim 5 or 6, characterized in that, The method for calculating the attention weights related to the item and the attention weights related to the position using the cross-scaling dot product attention module and then summing them to obtain the attention weight information for each attention head is as follows: Calculate attention weights for the project: Calculate the attention weights with respect to location: Summing is performed to obtain the attention weight information for each attention head. i : head i =Attention(QpW i Q ,K1W i K ,V1W i V )+Attention(QW i Q ,Kp1W i K ,V1W i V ) Where h is the number of attention heads, W i Q W i K W i V Let represent the weight matrices of the i-th attention head; The method for splicing is as follows: MultiHead(Q,K,V)=Concat(head1,..,head h )W o Where Concat is the concatenation operation, W o It is a linear transformation matrix; The method for calculating the attention weight information of item encoding and location encoding using a position-fully connected feedforward network is as follows: the attention weight information X2 of item encoding and location encoding is calculated by a fully connected neural network, then passed through a random deactivation layer, and finally residually connected and normalized with the attention weight information X2 of item encoding and location encoding to obtain the output sequence H of each encoding layer. l , l represents the l-th coding layer; The computation method of a fully connected neural network is as follows: x = X²W₁ + b₁ FFN(X2)=Φ(x)W2+b2 Where Φ(x) is the activation function, W1 and W2 are weights, and b1 and b2 are biases; The output sequence of the multi-layer cross-attention encoder module is: H L ={h1,h2,...,h τ ,…,h t }, where L is the number of encoder layers, and h τ Let τ be the τth hidden state.

8. The video recommendation method based on Sequence Recommendation SeqTransRepeatNet according to claim 7, characterized in that, The importance score of repeated exploration The calculation method is as follows: Among them, V re W re and U re is the weight parameter, 𝛷() is the activation function, and r and e represent the repeat mode and the exploration mode, respectively; The repeated exploration attention weight And video project sequence I s Context vector The calculation method is as follows: The probability P(r|I) of executing in the repetitive mode s ) and the probability P(e|I) of executing in exploration mode s The calculation method for ) is as follows: in, It is a weight matrix; The calculation of the probability P(i) of recommending item i in the repetitive pattern. j |r,I s The method is as follows: Among them, V r W r and U r These are the weight parameters, and Φ() is the activation function. It is the score of repetition importance. Indicates item i j ∈I s The total number of occurrences, where I represents the sequence of all video items, and P(i j |r,I s ) represents the probability of recommending item i in the repeat pattern; The calculation of the probability P(i|e,I) of recommending item i in exploration mode. s The method is as follows: in, It is the weight matrix; V e W e and U e Here are the parameters, and Φ() is the activation function. It explores the importance score. This represents the attention state in exploration mode, and then the last hidden state h is... τ and attention state Combined into video project sequence I s Hybrid representation P(i j |e,I s ) indicates that item i is recommended in exploration mode. j The probability of; The probability P(i) of generating the next recommended video is calculated. j |I s The method is as follows: P(i j |I s )=P(r|I s )P(i j |r,I s )+P(e|I s )P(i j |e,I s ) P(i j |I s ) indicates that in a given video item sequence I s Recommended project i j The probability of; The log-likelihood loss function is: Where θ represents all parameters of the SeqTransRepeatNet model.

Images