Sequence recommendation method, system, device and medium

Abstract

A sequence recommendation method is disclosed, including: obtaining a user's content item interaction history and generating a historical content item sequence of the user; generating a set of interval representations based on the historical content item sequence of the user; interacting each content item in the historical content item sequence with each interval representation in the set to determine a self-attention encoding matrix of the historical content item sequence; and predicting content items that the user may interact with based on the historical content item sequence and the self-attention encoding matrix. Other methods, corresponding systems, devices and media related to sequence recommendation are also disclosed.

Description

Technical Field

[0001] The present application relates to sequence recommendation, and in particular to a sequence recommendation method, system, device and medium. Background Art

[0002] Nowadays, the application of sequential recommendation models is becoming more and more widespread. For example, in content consumption applications such as news, video, and audio, the next recommended content item is generated based on the interaction history between the user and the content item. For another example, in shopping applications, the next recommended content item is generated based on the interaction history between the user and content items such as goods or services.

[0003] However, existing sequential recommendation models generally assume that user preferences remain unchanged, and thus use the entire interaction history between users and content items for modeling. That is, there is a bias in existing models that only considers users' long-term preferences.

[0004] Due to this long-term preference bias in the existing technology, users' short-term preferences are ignored, resulting in room for improvement in the performance of sequential recommendation models.

[0005] Therefore, a solution that can improve the performance of sequence recommendation is needed. Summary of the invention

[0006] In order to overcome the defects of the prior art, one or more embodiments of the present specification improve the performance of the sequence recommendation model by considering the local characteristics in the sequence of historical content items in the sequence recommendation model, taking the user's short-term preferences into consideration.

[0007] One or more embodiments of the present specification achieve the above-mentioned purpose through the following technical solutions.

[0008] In one aspect, a sequence recommendation method is provided, comprising: obtaining a user's content item interaction history, and generating a historical content item sequence of the user based on the obtained content item interaction history; generating a set of interval representations representing the user's interests in different time intervals based on the user's historical content item sequence; interacting each content item in the historical content item sequence with each interval representation in the set to determine a self-attention encoding matrix for the historical content item sequence; and predicting content items that the user may interact with based on the historical content item sequence and the self-attention encoding matrix.

[0009] Preferably, generating a set of a series of interval representations representing the interests of the user in different time intervals based on the historical content item sequence of the user includes: performing convolution processing on the historical content item sequence to generate the set.

[0010] Preferably, performing convolution processing on the sequence of historical content items to generate the set includes: generating an input matrix based on the sequence of historical content items; generating a query matrix Q, a key matrix K and a value matrix V based on the input matrix; performing downsampling convolution on the key matrix K and the value matrix V to generate a convolved key matrix and the convolved value matrix , as the set.

[0011] Preferably, the interaction of each content item in the historical content item sequence with each interval representation in the set to determine the self-attention encoding matrix of the historical content item sequence comprises: and the convolved value matrix Input to the dot product attention mechanism to generate the self-attention encoding matrix of the historical content item sequence.

[0012] Preferably, performing down-sampling convolution on the key matrix K and the value matrix V comprises: performing down-sampling convolution on the key matrix K and the value matrix V using two convolution layers.

[0013] Preferably, the stride in each convolutional layer is equal to the convolution kernel size.

[0014] Preferably, the method further includes: generating an asymmetric position coding matrix based on the positions of content items in the historical content item sequence, wherein content item-position interactions are encoded in both a key matrix and a query matrix used when generating the asymmetric position coding matrix, but content item-position interactions are not encoded in a value matrix used when generating the asymmetric position coding matrix, wherein based on the historical content item sequence and the self-attention coding matrix, predicting content items with which the user may interact includes: predicting content items with which the user may interact based on the historical content item sequence, the self-attention coding matrix and the asymmetric position coding matrix.

[0015] Preferably, generating an asymmetric position encoding matrix based on the positions of content items in the historical content item sequence comprises: generating a position embedding matrix based on the historical content item sequence; combining the input matrix and the position embedding matrix to generate a hybrid input matrix; generating a hybrid query matrix based on the hybrid input matrix and mixed key matrix ; and the mixed query matrix , Mixed Key Matrix And the value matrix V is input into the dot product attention mechanism to generate an asymmetric position encoding matrix for the historical content item sequence.

[0016] Preferably, predicting the content items with which the user may interact based on the historical content item sequence, the self-attention encoding matrix and the asymmetric position encoding matrix includes: combining the self-attention encoding matrix and the asymmetric position encoding matrix to output a fused encoding matrix; and predicting the content items with which the user may interact based on the historical content item sequence and the fused encoding matrix.

[0017] On the other hand, a sequence recommendation system is provided, including: an input processing module for acquiring a user's content item interaction history, and generating a historical content item sequence of the user based on the acquired content item interaction history; an interval representation generation module for generating a set of interval representations representing the user's interests in different time intervals based on the user's historical content item sequence; a dot product attention mechanism module for interacting each content item in the historical content item sequence with each interval representation in the set to determine a self-attention encoding matrix of the historical content item sequence; and a prediction module for predicting content items that the user may interact with based on the historical content item sequence and the self-attention encoding matrix.

[0018] Preferably, the sequence recommendation system further includes: an asymmetric position encoding module, used to generate an asymmetric position encoding matrix based on the positions of content items in the historical content item sequence, wherein content item-position interactions are encoded in both the key matrix and the query matrix used when generating the asymmetric position encoding matrix, but content item-position interactions are not encoded in the value matrix used when generating the asymmetric position encoding matrix, wherein the prediction module is used to predict content items with which the user may interact based on the historical content item sequence, the self-attention encoding matrix and the asymmetric position encoding matrix.

[0019] Preferably, the sequence recommendation system further includes: a fusion encoding module, used to combine the self-attention encoding matrix and the asymmetric position encoding matrix to output a fusion encoding matrix, wherein the prediction module is used to predict the content items that the user may interact with based on the historical content item sequence and the fusion encoding matrix.

[0020] On the other hand, a method for sequence recommendation is provided, comprising: obtaining content item interaction histories of multiple users, and generating multiple historical content item sequences of the multiple users based on the content item interaction histories of the multiple users; and using the multiple historical content item sequences to train a sequence recommendation model, wherein the sequence recommendation model includes an encoding layer, the encoding layer includes a self-attention encoding module, wherein the self-attention encoding module is used to: perform convolution processing on the historical content item sequence to generate a set of a series of interval representations representing the interests of the user in different time intervals; and interact each content item in the historical content item sequence with each interval representation in the set to determine a self-attention encoding matrix of the historical content item sequence, wherein the sequence recommendation model is trained at least in part based on the self-attention encoding matrix.

[0021] In yet another aspect, a sequence recommendation apparatus is provided, comprising: a memory; and a processor, wherein the processor is configured to execute the above method.

[0022] In yet another aspect, a computer-readable storage medium storing instructions is provided, which instructions, when executed by a computer, cause the computer to perform the above method.

[0023] Compared with the prior art, one or more embodiments of the present specification can achieve one or more of the following technical effects:

[0024] It can take users’ short-term patterns into consideration and tap into their short-term preferences.

[0025] Ability to gradually capture short-term patterns and better balance long-term and short-term preferences;

[0026] Uncover richer underlying correlations; and / or

[0027] Taking both long-term and short-term preferences into consideration improves system performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The above invention content and the following detailed description will be better understood when read in conjunction with the accompanying drawings. It should be noted that the accompanying drawings are only examples of the claimed invention. In the accompanying drawings, the same reference numerals represent the same or similar elements.

[0029] Figure 1 A simplified block diagram showing the self-attention encoding module of a traditional sequence recommendation model.

[0030] Figure 2 An exemplary simplified block diagram of a self-attention encoding module of a sequence recommendation model according to an embodiment of the present specification is shown.

[0031] Figure 3A block diagram showing an example implementation of a downsampling convolution module according to an embodiment of the present specification.

[0032] Figure 4 A simplified schematic block diagram of a coding layer including an asymmetric position coding module and a fusion coding module according to a preferred embodiment of the present specification is shown.

[0033] Figure 5 A schematic flowchart of an example method involving sequence recommendation according to an embodiment of the present specification is shown.

[0034] Figure 6 A schematic flow chart showing an example process for generating self-attention encodings for content items according to an embodiment of the present disclosure.

[0035] Figure 7 A schematic flow chart showing an example process for generating an asymmetric position code according to an embodiment of the present disclosure.

[0036] Figure 8 A schematic flow chart showing another example method involving sequence recommendation according to an embodiment of the present specification.

[0037] Fig. 9 A schematic block diagram of an example system involving sequence recommendation according to an embodiment of the present specification is shown.

[0038] Fig.10 A schematic block diagram showing an apparatus for implementing a method according to one or more embodiments of the present specification. DETAILED DESCRIPTION

[0039] The contents of the following specific implementations are sufficient to enable any technical person skilled in the art to understand the technical contents of one or more embodiments of this specification and implement them accordingly, and based on the description, claims and drawings disclosed in this specification, those skilled in the art can easily understand the purposes and advantages related to one or more embodiments of this specification.

[0040] Nowadays, the application of sequential recommendation models is becoming more and more widespread. For example, in content consumption applications such as news, video, and audio, the next recommended content item is generated based on the interaction history between the user and the content item. For another example, in shopping applications, the next recommended content item is generated based on the interaction history between the user and content items such as goods or services.

[0041] Sequential content item recommendation (or simply "sequential recommendation") refers to predicting content items that a user is likely to interact with. This prediction is usually based on content items that the user (and possibly other users) have previously interacted with, i.e., the user's content item interaction history.

[0042] In the embodiments of this specification, a content item may refer to any information or thing that can be digitized. Examples of content items may include, but are not limited to, goods, services, applications, articles, videos, audios, games, characters or virtual characters, and other information fragments, as well as any combination of the above various content items, etc.

[0043] The specific content of a content item can be defined by a user (e.g., a developer) according to specific needs. For example, in a shopping application, a content item may refer to a recommended product or service. In a news application, a content item may refer to news. In a multimedia (e.g., video, short video, or audio) application, a content item may refer to multimedia information or a fragment thereof. In a social networking application, a content item may refer to a person (their digital representation) or the content they publish.

[0044] In the embodiments of the present specification, the historical content item sequence refers to a sequence of content items with which a user has interacted. Generally, the historical content item sequence is a sequence of content items sorted in chronological order.

[0045] In the embodiments of this specification, interaction refers to any form of user operation on content items. For example, for shopping applications, interaction with goods or services may include operations such as purchase, click, browse, search, favorite, subscribe, and add to shopping cart. For news or multimedia applications, interaction may include operations such as playing, reading, clicking, searching, favorite, and subscribing to news or multimedia information. For social network applications, interaction may include following a person, viewing related information about the person, browsing, favorite, liking, and forwarding the content posted by the person, etc.

[0046] Existing sequence recommendation models (such as various Transformer-based models) generally assume that user preferences remain unchanged, and thus use the entire interaction history between users and content items for modeling, that is, only consider the user's long-term preferences. In other words, there is a long-term preference bias in the existing technology.

[0047] The inventors realize that users may have not only long-term preferences but also short-term preferences. Long-term preferences reflect the long-term needs and / or interests of users, while short-term preferences reflect the short-term needs and / or interests of users. For example, a user may be a long-term long-distance running enthusiast, and thus has always been interested in content items related to running, while the user may have an interest in cooking for a period of time and then lose interest. During this period of time, the user is interested in content items related to cooking.

[0048] Similar to a user's long-term preferences, a user's short-term preferences may generate patterns worth mining in the local part of the user's historical content item sequence.

[0049] In addition, even if those skilled in the art can overcome the long-term preference bias and realize that they need to utilize the local characteristics of the historical content item sequence, they can only think of simply applying the convolution algorithm to the traditional sequence recommendation model, that is, dividing the entire historical content item sequence into several segments and performing the convolution operation. However, this simple segmentation will damage the long-term characteristics of the historical content item sequence.

[0050] To solve the above problems, in the embodiments of this specification, firstly, a specially developed convolution module is used to segment the overall long-term historical content item sequence into a series of local sequences, and secondly, these local subsequences are interacted with each content item, thereby generating a locality-aware self-attention representation. Benefiting from the limited content item-subsequence interaction, the embodiments of this specification can progressively capture local short-term patterns, thereby considering both local short-term patterns and global long-term patterns with appropriate weights, thereby achieving better performance with nearly linear complexity.

[0051] In a more preferred embodiment of the present specification, in order to further enhance the learning of global context information, a new local encoding strategy is proposed. The strategy uses position encoding with an asymmetric structure to help mine richer content item-content item correlations.

[0052] In a more preferred embodiment of the present specification, the above-mentioned locally aware self-attention representation is also fused with the asymmetric encoded position-aware representation to achieve better performance.

[0053] The Transformer model has been widely used due to its excellent performance and ease of training. The sequence recommendation model in one or more embodiments of this specification may be a significant modification and / or supplement to the traditional Transformer model. For some technical details not mentioned in this specification, refer to the specific implementation of the traditional Transformer model. Relevant information about the traditional Transformer model can be found, for example, in the paper "Attention Is All You Need" published by Ashish Vaswani et al. in 2017 (see, for example, https: / / arxiv.org / abs / 1706.03762).

[0054] In addition, various variants or improvements of the Transformer model have also appeared, which are collectively referred to as Transformer-based models in this article. Examples of Transformer-based models may include, but are not limited to, Transformer models, BERT models and their variants, GPT, GPT-2, GPT-3 models and their variants, T5 models and their variants, etc. The embodiments of this specification are also applicable to further modification and / or supplementation of these variants or improvements.

[0055] In the following, we will first briefly introduce the traditional sequence recommendation model (taking the Transformer model as an example), especially the general implementation of its scaling self-attention mechanism, and then explain the significant modifications and / or supplements of the embodiments of this specification based on it.

[0056] However, it should be appreciated that the embodiments of the present specification are not limited to the Transformer model, but can be used in conjunction with any sequence recommendation model that employs a self-attention mechanism.

[0057] See also Figure 1 , which shows a simplified block diagram of the self-attention encoding module 100 of the traditional sequence recommendation model, which adopts the dot product attention mechanism.

[0058] like Figure 1 As shown at 102 in FIG. 1 , the self-attention encoding module 100 has an input matrix .

[0059] For ease of explanation, assume that there is a collection of content items I , the content item collection I It is the collection of all content items that can be interacted with by users in the application served by the sequence recommendation system. ,in For user interaction n content items and The purpose of the sequence recommendation model is to predict the next content item, that is, In terms of form, create a content item coding matrix To represent the entire set of content items I ,in Content Item Set I The total number of content items in d is the latent dimension. Given the historical content item sequence, we can construct n content item embedding to generate the input matrix ,in .

[0060] In such Figure 1In the conventional self-attention encoding module 100 shown in FIG. 1 , the input matrix It can be encoded as the output according to the following formula 1 :

[0061]

[0062] in are different linear projections, and is a scaling term, the purpose of which is to avoid the value of the attention matrix being too large. As can be understood by those skilled in the art, Q, K, and V are the query matrix, key matrix, and value matrix representing the set of "query" vectors, "key" vectors, and "value" vectors, respectively, and all belong to .

[0063] The above process Figure 1 As shown more clearly in Figure 1 104, 106 and 108, the input matrix Through different linear projections (in Figure 1 Q, K and V are obtained as shown in the figure. Then, Figure 1 As shown in 110, matrix multiplication (MatMul) and Softmax operation (in Figure 1 Preferably, a scaling operation may also be performed on it, that is, divided by ( Figure 1 ). Figure 1 As shown in 112, the above operation can obtain a matrix. Figure 1 114, the matrix is ​​multiplied with V (in Figure 1 The final output is shown as MatMul in (like Figure 1 shown in 116).

[0064] The details of the above operations are known to those skilled in the art (for example, see the relevant implementation of the Transformer model), and will not be repeated here. In the following text, the operations that are the same as the above operations are simplified to avoid blurring the key points of the embodiments of this specification.

[0065] It can be seen that in Figure 1 In the traditional model, only the content items and the correlation between content items in the entire historical content item sequence are considered.

[0066] See also Figure 2 , which shows an exemplary simplified block diagram of the self-attention encoding module 200 of the sequence recommendation model according to an embodiment of the present specification.

[0067] In the embodiment of this specification, the historical content item sequence Segment into multiple subsequences, e.g. . The inventors found that coherent subsequences can still reflect the user's interests and contain valuable local sequence patterns. In the embodiments of this specification, such local sequence patterns are extracted through a sliding window operation via a shared convolution kernel. Taking into account the irregular preferences and time information between each subsequence, a convolution module (and preferably a multi-layer convolution module) is used to filter out redundant information and capture high-level local patterns. Through the convolution operation, a set of a series of interval representations is obtained. The interval representation can represent the user's interests in different time intervals. Preferably, in the embodiments of this specification, it is proposed to use content item-interval representation interaction to replace the content item-content item interaction in the traditional solution.

[0068] In such Figure 2 In the conventional self-attention encoding module 200 shown in FIG. 1 , the input matrix It can be encoded as the output according to the following formula 2 :

[0069]

[0070] in and are obtained by performing downsampling convolution on the key matrix K and the value matrix V through the convolution module, and m represents the number of content items included in each subsequence. m can be determined by the user (such as the developer or user of the sequence recommendation system) according to needs or experiments. Usually, m is a constant, and its value is a small integer (such as 2 or 3). The meanings of other quantities or functions in Formula 2 can refer to Formula 1.

[0071] The above process Figure 2 As shown more clearly in Figure 2 204, 206 and 208, the input matrix at 202 Through different linear projections (in Figure 2 The query matrix Q, key matrix K and value matrix V are obtained by performing an embedding operation on a sequence of historical content items and then performing a linear projection. The specific implementation can refer to the traditional Transformer implementation.

[0072] Afterwards, with Figure 1 Different from directly performing matrix multiplication on Q and K in Figure 2 As shown in 218, firstly, a down-sampling convolution operation is performed on K (in Figure 2 In order to generate the convolution key matrix , then Figure 2The 210 shown on Q and Perform matrix multiplication (MatMul) and Softmax operation (in Figure 2 Preferably, a scaling operation may also be performed on it, that is, divided by ( Figure 2 ). Figure 2 As shown in 212, the above operation can obtain an nx m matrix. In some embodiments, similar to the traditional Transformer solution, other operations such as mask operations can be performed.

[0073] Similarly, with Figure 1 In contrast, directly performing matrix multiplication on this matrix and V is different, as Figure 2 As shown in 220, first, a downsampling convolution operation is performed on V (in Figure 2 DownConv) to generate a convolved value matrix , then Figure 2 The 214 shown for this nxm matrix and Perform matrix multiplication (in Figure 2 The final output is shown as MatMul in (like Figure 2 216), the output This is the sequence of historical content items The self-attention encoding matrix.

[0074] In other words, Figure 1 In the above example, the query matrix Q, key matrix K, and value matrix V are input to the dot product attention mechanism, while in Figure 2 In the example, the query matrix Q and the convolution key matrix and the convolved value matrix Input to the dot product attention mechanism. The specific implementation of the dot product attention mechanism can be the same as the traditional Transformer model, or other appropriate dot product attention mechanisms can be used.

[0075] Through the above operations, the complexity of the algorithm is reduced from the traditional self-attention mechanism to Reduce to Since m (for example, it is 2, 3 or other small integers) is usually much smaller than n (the historical content item sequence may include a large number of content items), the algorithm complexity is greatly reduced (reduced to nearly linear complexity).

[0076] It can be seen that the essence of the above self-attention encoding is to convert each content item in the historical content item sequence (which is represented by the input matrix The query matrix Q obtained by linear projection is then combined with each interval representation in a series of interval representations (which are obtained by down-convolution sampling from the key matrix K and the value matrix V, respectively). and ) (this interaction is reflected in operations such as matrix multiplication and Softmax). In this way, the local characteristics in the historical content item sequence can be taken into account, so that the user's short-term preferences can be gradually captured and both short-term preferences and long-term preferences can be captured at the same time, improving the performance of the system.

[0077] The following describes the specific implementation of the downsampling convolution (or "downsampling" for short) operation.

[0078] See also Figure 3 , which shows a block diagram of an example implementation of the downsampling convolution module 300 according to an embodiment of the present specification. Since the downsampling convolution module 300 can generate a set of a series of interval representations representing the interests of the user in different time intervals, it can also be called an interval representation generation module.

[0079] like Figure 3 As shown, in a preferred embodiment of the present specification, the down-sampling convolution module 300 (which can be represented as ) includes two convolutional layers 302 and 304. Taking K as an example, the downsampling convolution module 300 is To generate its downsampled convolution output from K In the above formula, Used to perform a 1-D convolution operation with a specific stride.

[0080] Preferably, in order to generate non-overlapping subsequences, the stride can be set to always be equal to the convolution kernel size. In this case, the appropriate convolution kernel size (and thus the stride size) can be selected according to the specific situation, and the appropriate model complexity can be brought accordingly.

[0081] In the above formula, ,in and are the convolution kernel sizes of convolution layer 302 and convolution layer 304, respectively, and is a round-up function to ensure m The value of is an integer. Usually, k1 and k2 are selected so that the value of m is a small integer, such as 2, 3 or 4.

[0082] In a preferred example, there may be a ReLU activation module ( Figure 3 (not shown in the figure) to increase the nonlinearity of the downsampling convolution module 300 and improve the performance of the module. Alternatively, other activation modules may be used, or no activation module may be used.

[0083] The above describes the use of downsampling convolution to obtain K The specific implementation can be done according to In the same way, V The resulting convolved key matrix and the convolved value matrix It represents the interests of the user in different time intervals, and constitutes a set of interval representations that represent the interests of the user in different time intervals.

[0084] It should be appreciated that the above description is only a preferred embodiment of downsampling convolution. In practical applications, other implementations may be selected. For example, other numbers of convolutional layers (e.g., a single layer or more than 2 layers) may be used. For example, the step size and the size of the convolution kernel may be different. For example, the step size and the convolution kernel may each adopt other appropriate values, and so on.

[0085] In this way, the embodiments of the present specification can convert the historical content item sequence into a set of interval representations to better capture the user's interests in different time intervals. Based at least on the historical content item sequence and the self-attention encoding matrix, the content items that the user may interact with can be predicted, thereby providing recommendations for content items to the user.

[0086] The above examples introduce an embodiment of a sequence recommendation model including self-attention encoding. In some examples, the sequence recommendation model may not include a position encoding module. Failure to encode the position may result in a loss of absolute position information, making it impossible to provide a global order relationship between content items, and performance will be affected, but it is still achievable. In other examples, the sequence recommendation model may include traditional position encoding, such as the position encoding scheme used in the traditional Transformer model, which is a symmetric position encoding. However, for the self-attention mechanism of this specification that considers the interaction between content item-interval representations, the use of traditional symmetric position encoding is not optimal and may lead to conflicts between local convolution operators.

[0087] Therefore, in a more preferred embodiment of the present specification, an asymmetric position encoding scheme different from the traditional symmetric position encoding scheme is proposed. The asymmetric position encoding scheme is combined with the self-attention encoding scheme including convolution in the embodiment of the present specification, and the performance of the embodiment of the present specification can be further improved by cooperating with each other.

[0088] See also Figure 4, which shows a simplified schematic block diagram of the encoding layer 400 according to a preferred embodiment of the present specification. The encoding layer 400 may include a self-attention encoding module, an asymmetric position encoding module, and a fusion encoding module. It can be appreciated that some other components of the sequence recommendation model are omitted in the figure to avoid blurring the focus of the embodiment of the present specification. For these components, reference can be made to the specific implementation of the traditional Transformer model.

[0089] Figure 4 Components 402, 404, 406, 408, 410, 412, 414, 416, 418 and 420 in the above constitute a self-attention encoding module, and its implementation details can be referred to above for Figure 2 The description of components 202, 204, 206, 208, 210, 212, 214, 216, 218 and 220 are not repeated here. Figure 4 Components 422, 424, 426, 428, 430, 432 and 434 in constitute an asymmetric position encoding module. Figure 4 Component 436 in illustrates a fusion encoding module.

[0090] like Figure 4 As shown, in order to encode position information and learn more robust position representations, the proposed sequence recommendation model uses an asymmetric approach to process the positions and content items corresponding to the "query", "key" and "value" vectors. Specifically, the embodiment of this specification retains the content item-position interactions in the key matrix and the query matrix (by combining the content item embedding with the position embedding), but removes the content item-position interactions in the value matrix (by inheriting the original "value"). In this way, the global order information between positions and the contextual information of the content items can be learned. Unlike the traditional self-attention mechanism, the position-aware context output of the embodiment of this specification can be expressed as:

[0091]

[0092] in ; is the position embedding matrix; is an additional linear projection; It is the same as the meaning in Formula 1 above. It can be understood that although the above embodiment applies The simple addition of and P, but the two can also be combined in other ways, such as taking their weighted sum to reflect their different weights.

[0093] The above process Figure 4 As shown more clearly in Figure 4 As shown in 426 and 428, the input matrix The mixed input matrix consisting of the position embedding matrix P Through different linear projections (in Figure 1 is shown as Linear) and . Linear projection is obtained by taking the mixed input matrix (rather than just the input matrix), thus mixing the key matrix and the hybrid query matrix The content item-position interaction is realized. Since the value matrix V is derived from the input matrix It is obtained directly without involving the position embedding matrix P, so the content item-position interaction is not reflected in the value matrix V.

[0094] Then, if Figure 4 As shown in 428, and Perform matrix multiplication (MatMul) and Softmax operation (in Figure 4 Preferably, a scaling operation may also be performed on it, that is, divided by ( Figure 4 ). Figure 4 As shown in 430, the above operation can obtain an nxn matrix. Figure 4 432, the nxn matrix is ​​multiplied with V (in Figure 4 The asymmetric position encoding matrix can be obtained by (like Figure 4 shown in 434).

[0095] Therefore, based on the historical content item sequence, the self-attention encoding matrix and the asymmetric position encoding matrix, the content items that the user is likely to interact with can be predicted, thereby providing recommendations for content items to the user.

[0096] Compared with the symmetric coding in the traditional scheme, the asymmetric coding in the embodiment of this specification can capture richer underlying correlations, thereby achieving better performance.

[0097] Then, if Figure 4 As shown in 436 and 438 in , the self-attention encoding matrix and the asymmetric position encoding matrix are combined (as shown in Figure 4 Add) is the fusion coding matrix ,in is a linear projection; g is a mapping formed by two layers of feedforward networks. The implementation of this combined operation can refer to the traditional Transformer model. In other words, it can be used Figure 4 (or Figure 2) replaces the dot product attention mechanism in the traditional Transformer model (or other Transformer-based models), and can optionally be used Figure 4 The asymmetric positional encoding module in replaces the positional encoding module in the traditional Transformer model (or other Transformer-based models), and optionally replaces Figure 4 The fusion coding module in the embodiment replaces the feedforward network and the combination and normalization layer (Add&Norm) in the traditional Transformer model (or other models based on Transformer), so as to use the coding layer of the embodiment of this specification to implement a base layer in the encoder in the traditional Transformer model (or other models based on Transformer).

[0098] The self-attention encoding matrix and the asymmetric position encoding matrix may also be combined in any other appropriate manner, such as by a simple summation, a weighted summation, etc.

[0099] Figure 4 The fused coding matrix O generated by the coding layer 400 in is the fusion output, which is equivalent to the output of each base layer of the Transformer-based model. Accordingly, the coding layer 400 of the embodiment of the present specification can be regarded as a base layer, which can be referred to as a progressive self-attention and asymmetric position coding layer, or PAUP layer for short, in this document. The PAUP layer can be used to replace the base layer (for example, the 6 base layers in the encoder of the Transformer model) in the Transformer model (specifically, the encoder in the Transformer-based model). In some preferred embodiments, a multi-head mechanism can also be used for the PAUP layer. The implementation method of the multi-head mechanism can refer to the Transformer-based model. For example, the number of heads in the multi-head mechanism is 2.

[0100] In the complete sequence recommendation model, L The input sequence of stacked PAUP layers is encoded and the output is . This output may be equivalent to the output of the encoder of a Transformer-based model, for example. L The number may be 6, for example.

[0101] Specifically, The n OK, that is , for the input sequence The last item in The predictive representation of . Subsequently, the sequence recommendation model may be trained in any appropriate manner. Preferably, the sequence recommendation model may be trained in an end-to-end manner. In the embodiment of the present specification, a cross entropy loss function may be used during training, namely:

[0102]

[0103] in for and any content item i The dot product between ; Represents trainable parameters, including embedding matrices E, P, and j (in ) network parameters in the layer By minimizing the cross entropy loss function, a trained sequence recommendation model can be obtained. Any other appropriate loss function that can be conceived by those skilled in the art can also be used.

[0104] Thus, based on the historical content item sequence and the fused coding matrix, the content items that the user may interact with can be predicted, so as to be provided to the user as recommendations for content items. After describing the composition of the sequence recommendation model above, those skilled in the art can know how to train the sequence recommendation model and how to use the trained sequence recommendation model to generate predictions for sequence recommendation, which will not be described in detail here.

[0105] In fact, through the convolution mechanism of the embodiment of this specification plus the self-attention mechanism (content item-interval representation interaction), it is possible to achieve a gradual increase in attention weights from far to near over time on different local segments. In other words, the attention mechanism of the embodiment of this specification pays different attention to different local segments. The closer the local segment is to the current time, the greater the attention weight will be, and the farther the local segment is from the current time, the smaller the attention weight will be. In other words, the attention mechanism of the embodiment of this specification can not only capture user preferences in different time periods, but also better focus on users' recent short-term preferences, which further improves the accuracy of predicting recent user patterns.

[0106] In addition, since the self-attention mechanism of the embodiment of this specification does not only focus on any local fragment, but gradually considers the information of all local fragments, the behavior patterns of all local fragments will be taken into account in general. Therefore, the attention mechanism of the embodiment of this specification can consider the user's long-term preferences and short-term preferences at the same time, and give better weights to both, thereby improving the accuracy of modeling user behavior patterns, thereby improving the performance of sequential recommendations.

[0107] See also Figure 5 , which shows a schematic flow chart of an example sequence recommendation method 500 according to an embodiment of this specification. Figure 5 Some details of the method can be found in the above Figure 2-4 Description.

[0108] like Figure 5 As shown, method 500 may include: at operation 502, a content item interaction history of a user may be acquired, and a historical content item sequence of the user may be generated based on the acquired content item interaction history.

[0109] As mentioned above, the content item in this article may refer to any information or thing that can be digitized. The interaction between the user and the content item may refer to any form of operation of the user on the content item. The content item interaction history may include the original data of the user's operation in the application, such as a record of all operations of the user in the application.

[0110] The user's content item interaction history may be obtained from various data sources. Such data sources may include user devices, servers associated with content items, third-party services, or a combination thereof. For example, for shopping applications, the content item interaction history may be obtained from various data stores storing the user's interaction history with goods or services; for news or multimedia applications, the content item interaction history may be obtained from the user's device. The user's content item interaction history may also be obtained from third-party services.

[0111] After obtaining the content item interaction history, multiple historical content item sequences of the multiple users may be generated based on the content item interaction history of the multiple users. For example, the obtained content item interaction history may be processed to extract the content items interacted by the users, and the content items are arranged in chronological order to form the historical content item sequence of the user.

[0112] The user's content item interaction history may be obtained and / or a historical content item sequence may be generated in any other manner conceivable by those skilled in the art.

[0113] Method 500 may include: at operation 504, a set of interval representations representing the interests of the user at different time intervals may be generated based on the historical content item sequence of the user. For example, the historical content item sequence may be convolved to generate the set. Specifically, the convolved key matrix K and the value matrix V generated from the input matrix may be convolved by a convolution module to generate the convolved key matrix and the convolved value matrix To form the set. For example, the operation 504 may be performed by an interval representation generation module. The specific details of the operation 504 may refer to the description of the process 600 below.

[0114] Method 500 may further include: at operation 506, each content item in the sequence of historical content items may be interacted with each interval representation in the set to determine a self-attention encoding matrix for the sequence of historical content items. Figure 2 as well as Figure 4 For example, the query matrix Q, the convolution key matrix and the convolved value matrix Input to the dot product attention mechanism to generate the self-attention encoding matrix of the historical content item sequence. The dot product attention mechanism can refer to the implementation of the traditional Transformer model. As described above, the dot product attention mechanism can first perform an operation on the query matrix Q and the convolved key matrix Perform matrix multiplication and Softmax operation (and optional scaling operation) to generate a matrix and add this matrix to the convolved value matrix Perform a matrix multiplication operation to obtain the self-attention encoding matrix.

[0115] Method 500 may further include: at operation 508, based on the historical content item sequence and the self-attention encoding matrix, predicting the content items that the user may interact with. After obtaining the original historical content item sequence and the self-attention encoding matrix, the sequence recommendation model may be used to generate content items that the user may interact with. The details of this operation can be referred to the implementation of the traditional Transformer-based model, which will not be repeated here.

[0116] Preferably, in addition to using the self-attention encoding matrix, an asymmetric position encoding matrix can also be used, as described above with reference to Figure 4 In this case, the method 500 may further include: generating an asymmetric position encoding matrix based on the positions of the content items in the historical content item sequence, wherein the content item-position interactions are encoded in both the key matrix and the query matrix used when generating the asymmetric position encoding matrix, but the content item-position interactions are not encoded in the value matrix used when generating the asymmetric position encoding matrix.

[0117] In this case, the content items that the user may interact with may be predicted based on the historical content item sequence, the self-attention encoding matrix, and the asymmetric position encoding matrix. For example, the self-attention encoding matrix and the asymmetric position encoding matrix may be combined to output a fused encoding matrix, and the content items that the user may interact with may be predicted based on the historical content item sequence and the fused encoding matrix.

[0118] The details of this operation can be found below. Figure 7 Description.

[0119] The method 500 may further include: outputting the recommendation for the content item to the user based on the generated recommendation (not shown in the figure). For example, the recommendation may be displayed or otherwise prompted at a user device of the user for selection by the user.

[0120] See also Figure 6 , which shows a flowchart of an example process 600 for generating a set of interval representations according to an embodiment of the present description. Figure 2-4 (especially Figure 3 ) description.

[0121] like Figure 6 As shown, the method 600 may include: at operation 602, an input matrix may be generated based on the historical content item sequence. For example, the input matrix may be generated by performing an embedding operation on the content items.

[0122] The method 600 may further include: at operation 604, based on the input matrix, a query matrix Q, a key matrix K, and a value matrix V may be generated. For example, the query matrix Q, the key matrix K, and the value matrix V may be generated by different linear projections of the input matrix.

[0123] The method 600 may further include: at operation 606, performing a down-sampling convolution on the key matrix K and the value matrix V to generate a convolved key matrix and the convolved value matrix . Preferably, two convolutional layers can be used to perform downsampling convolution on the key matrix K and the value matrix V. Preferably, each convolutional layer uses 1-D convolution. Preferably, the step size in each convolutional layer is equal to the convolution kernel size. In this way, non-overlapping subsequences can be obtained, thereby improving the performance of the system. The specific details of this operation can be referred to the above Figure 3 Description.

[0124] See also Figure 7 , which shows a flowchart of an example process 700 for generating an asymmetric position encoding according to an embodiment of the present description. The example process can be, for example, referred to in Figure 4 The details of this example process can be found in the description of the asymmetric position encoding module. Figure 4 Description.

[0125] Process 700 may include: At operation 702, a position embedding matrix may be generated based on the historical content item sequence. For example, the position embedding matrix may be generated by performing an embedding operation on the positions of each content item in the historical content item sequence. The details of this operation may refer to the implementation in the traditional Transformer model.

[0126] The process 700 may further include: at operation 704, the input matrix and the position embedding matrix may be combined to generate a mixed input matrix. For example, the two may be combined by a simple summation, a weighted summation, or other appropriate manners.

[0127] Process 700 may also include: at operation 706, generating a hybrid query matrix based on the hybrid input matrix and mixed key matrix For example, the mixed query matrix can be generated by different linear projections of the mixed input matrix and mixed key matrix , in the same way that the query matrix and key matrix are generated from the input matrix.

[0128] Process 700 may also include: at operation 708, the hybrid query matrix , Mixed Key Matrix The sum matrix V is input to the dot product attention mechanism to generate an asymmetric position encoding matrix for the historical content item sequence. Preferably, the dot product attention mechanism can be a dot product self-attention mechanism based on the Transformer model. For details, please refer to the above Figure 4 For example, and Matrix multiplication (MatMul) and Softmax operation (and optional scaling operation) are performed to obtain a matrix, and the matrix is ​​multiplied with V to obtain the asymmetric position encoding matrix.

[0129] See also Figure 8 , which shows a schematic flowchart of an example method 800 involving sequence recommendation according to an embodiment of the present specification. Figure 8 Some details of the method can be found in the above Figure 2-4 Description.

[0130] like Figure 8 As shown, method 800 may include: at operation 802, content item interaction histories of multiple users may be obtained, and multiple historical content item sequences of the multiple users may be generated based on the content item interaction histories of the multiple users. For details of this operation, reference may be made to the description of operation 502 above.

[0131] Method 800 may include: at operation 804, the plurality of historical content item sequences may be used to train a sequence recommendation model. The sequence recommendation model includes an encoding layer, the encoding layer includes a self-attention encoding module, wherein the self-attention encoding module may be used to: perform convolution processing on the historical content item sequence to generate a set of a series of interval representations representing the interests of the user at different time intervals; and interact each content item in the historical content item sequence with each interval representation in the set to determine a self-attention encoding matrix of the historical content item sequence, wherein the sequence recommendation model is trained at least in part based on the self-attention encoding matrix. Preferably, the sequence recommendation model may be implemented by replacing the attention mechanism in the Transformer-based model with the self-attention encoding module of the embodiment of the present specification.

[0132] More preferably, in addition to this, the position encoding module in the Transformer-based model can be replaced by an asymmetric position encoding module in the embodiment of the present specification. The asymmetric position encoding module is used to generate an asymmetric position encoding matrix based on the position of the content item in the historical content item sequence, wherein the asymmetric position encoding matrix encodes the content item-position interaction in the key matrix and the query matrix, but does not encode the content item-position interaction in the value matrix. The specific operation of the asymmetric position encoding module can be referred to below. Figure 7 Description.

[0133] In this case, preferably, the encoding layer further includes a fusion encoding module, which combines the self-attention encoding matrix and the asymmetric position encoding matrix to output a fusion encoding matrix. For example, the output of the fusion encoding module can be the sum of the self-attention encoding matrix and the asymmetric position encoding matrix after linear projection and then passing through a two-layer feedforward network.

[0134] Alternatively, the encoding layer may not include a position encoding module. In this case, the output of the self-attention encoding module is used as the output of the encoding layer.

[0135] Alternatively, the encoding layer may include the same encoding module as the Transformer-based model. In this case, the output of the encoding layer is a combination of the output of the self-attention encoding module and the encoding module.

[0136] Preferably, the sequence recommendation module comprises a plurality of encoding layers, and the plurality of encoding layers are stacked to implement an encoder of the sequence recommendation model. Preferably, the sequence recommendation model can be trained based on the fused encoding matrix using a cross entropy loss function.

[0137] See also Fig. 9 , which shows a schematic block diagram of a sequence recommendation system 900 according to an embodiment of the present specification. The specific implementation of the system 900 can refer to the above Figure 2-4Description.

[0138] like Fig. 9 As shown, the system 900 may include an input processing module 902. The input processing module may be used to obtain content item interaction histories of multiple users; and generate multiple historical content item sequences of the multiple users based on the content item interaction histories of the multiple users. The specific details of the operation of this module may refer to the description of operation 502 above.

[0139] The system 900 may further include an interval representation generation module 904. The interval representation generation module 904 may be used to generate a set of interval representations representing the interests of the user in different time intervals based on the historical content item sequence of the user. For specific details of this operation, reference may be made to the description of operation 504 above. For example, the interval representation generation module 904 may include a matrix generation module (not shown in the figure) for generating a query matrix Q, a key matrix K, and a value matrix V and a convolution module (such as Figure 2 and Figure 3 as shown).

[0140] The system 900 may also include a dot product attention mechanism module 906. The dot product attention mechanism module 906 may be used to interact each content item in the sequence of historical content items with each interval representation in the set to determine a self-attention encoding matrix for the sequence of historical content items. The specific details of the operation of this module may refer to the description of operation 506 above. For example, the dot product attention mechanism module may include a matrix multiplication module, a Softmax module, a scaling module, etc., which adopts the method described in reference Figure 2 The interval representation generation module 904 and the dot product attention mechanism module 906 can together constitute a self-attention encoding module 910.

[0141] The system 900 may also include a prediction module 908. The prediction module 908 may be used to predict content items that the user may interact with based on the historical content item sequence and the self-attention encoding matrix. The specific details of the operation of this module may refer to the description of operation 508 above.

[0142] Preferably, the system 900 may further include an optional asymmetric position encoding module 912. The asymmetric position encoding module 910 may be used to generate an asymmetric position encoding matrix based on the position of the content item in the historical content item sequence, wherein the content item-position interaction is encoded in both the key matrix and the query matrix used in generating the asymmetric position encoding matrix, but the content item-position interaction is not encoded in the value matrix used in generating the asymmetric position encoding matrix. The specific details of the operation of this module may be referred to in the description of Figure 4In this case, the prediction module may be used to predict the content items that the user may interact with based on the historical content item sequence, the self-attention encoding matrix and the asymmetric position encoding matrix.

[0143] Preferably, the system 900 may further include an optional fusion encoding module 914. The fusion encoding module 912 may be used to combine the self-attention encoding matrix and the asymmetric position encoding matrix to output a fusion encoding matrix. The specific details of the operation of this module may be referred to in Figure 4 In this case, the prediction module may be used to predict the content item with which the user may interact based on the historical content item sequence and the fused coding matrix.

[0144] Preferably, the interval representation generation module, the dot product attention mechanism module, the asymmetric position encoding module (if any) and the fusion encoding module (if any) can constitute the encoding layer 916, and multiple encoding layers can constitute an encoder, and finally constitute a sequence recommendation model 918.

[0145] Fig.10 A schematic block diagram of an apparatus 1000 for implementing a method according to one or more embodiments of the present specification is shown. The apparatus may be used to implement, for example, any of the methods described herein (e.g., methods 500-800). The apparatus may also be implemented as any computing node or computing node cluster described herein (e.g., through virtualization, etc.). The apparatus may include a processor 1010 and a memory 1015, the processor being configured to execute any of the methods described above. The memory may include memory and / or persistent storage to store user interaction history, etc. The memory may also be used to store any instructions, variables, intermediate data, etc. that may be used during the execution of the method.

[0146] The device 1000 may include a network connection element 1025, for example, a network connection device that can be connected to other devices through a wired connection or a wireless connection. The wireless connection can be, for example, a WiFi connection, a Bluetooth connection, a 3G / 4G / 5G network connection, etc. For example, the network connection element can be used to connect to a network to obtain data (such as user interaction history, etc.) and other various data. The network connection element can also receive input from a user from other devices or transmit data to other devices for display.

[0147] The device may also optionally include other peripheral components 1020, such as input devices (such as keyboards, mice), output devices (such as displays), etc. For example, in a method based on user input, the user may perform input operations via the input device. Corresponding information may also be output to the user via the output device, such as content items to be recommended to the user.

[0148] Each of these modules may communicate with each other directly or indirectly, for example, via one or more buses (eg, bus 1005 ).

[0149] Moreover, the present application also discloses a computer-readable storage medium including computer-executable instructions stored thereon, and when the computer-executable instructions are executed by a processor, the processor is caused to perform the methods of the embodiments described herein.

[0150] In addition, the present application also discloses a device, which includes a processor and a memory storing computer-executable instructions, and when the computer-executable instructions are executed by the processor, the processor executes the methods of the embodiments described herein.

[0151] In addition, the present application also discloses a system, which includes a device for implementing the methods of the embodiments described herein.

[0152] It can be understood that the method according to one or more embodiments of the present specification can be implemented by software, firmware or a combination thereof.

[0153] It should be understood that the various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device and system embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0154] It should be understood that the above description of specific embodiments of this specification is provided. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0155] It should be understood that an element described in singular form herein or shown in the drawings as only one does not limit the number of the element to one. In addition, modules or elements described or shown as separate herein may be combined into a single module or element, and modules or elements described or shown as single herein may be split into multiple modules or elements.

[0156] It should also be understood that the terms and expressions used herein are for descriptive purposes only, and one or more embodiments of this specification should not be limited to these terms and expressions. The use of these terms and expressions does not mean to exclude any equivalent features of the illustrations and descriptions (or parts thereof), and it should be recognized that various modifications that may exist should also be included in the scope of the claims. Other modifications, changes and substitutions may also exist. Accordingly, the claims should be deemed to cover all such equivalents.

[0157] Similarly, it should be pointed out that although the description has been made with reference to the current specific embodiments, ordinary technicians in this technical field should realize that the above embodiments are only used to illustrate one or more embodiments of this specification, and various equivalent changes or substitutions may be made without departing from the spirit of the invention. Therefore, as long as the changes and modifications to the above embodiments are within the scope of the essential spirit of the invention, they will fall within the scope of the claims of this application.

Claims

1. A sequence recommendation method, comprising: Acquire a content item interaction history of a user, and generate a historical content item sequence of the user based on the acquired content item interaction history; Based on the historical content item sequence of the user, a set of interval representations representing the interests of the user in different time intervals is generated, including: generating an input matrix based on the historical content item sequence; generating a query matrix Q, a key matrix K and a value matrix V based on the input matrix; and performing downsampling convolution on the key matrix K and the value matrix V to generate a convolved key matrix K conv And the convolution value matrix V conv , as the set; Interacting each content item in the historical content item sequence with each interval representation in the set to determine a self-attention encoding matrix of the historical content item sequence, including: conv And the convolution value matrix V conv Input to the dot product attention mechanism to generate a self-attention encoding matrix of the historical content item sequence; and Based on the historical content item sequence and the self-attention encoding matrix, predict the content items that the user may interact with.

2. The method of claim 1, wherein performing down-sampling convolution on the key matrix K and the value matrix V comprises: Two convolutional layers are used to perform down-sampling convolution on the key matrix K and the value matrix V.

3. The method of claim 2, wherein the stride in each convolutional layer is equal to the convolution kernel size.

4. The method of claim 1, further comprising: generating an asymmetric position encoding matrix based on the positions of content items in the sequence of historical content items, wherein content item-position interactions are encoded in both a key matrix and a query matrix used in generating the asymmetric position encoding matrix, but not in a value matrix used in generating the asymmetric position encoding matrix, Predicting the content items that the user may interact with based on the historical content item sequence and the self-attention encoding matrix includes: Based on the historical content item sequence, the self-attention encoding matrix and the asymmetric position encoding matrix, the content items that the user may interact with are predicted.

5. The method of claim 4, wherein generating an asymmetric position encoding matrix based on the positions of the content items in the sequence of historical content items comprises: generating a position embedding matrix based on the sequence of historical content items; combining the input matrix and the position embedding matrix to generate a hybrid input matrix; Generate a hybrid query matrix Q based on the hybrid input matrix mix and the mixed key matrix K mix ;as well as The mixed query matrix Q mix , mixed key matrix K mix And the value matrix V is input into the dot product attention mechanism to generate an asymmetric position encoding matrix of the historical content item sequence.

6. The method of claim 5, wherein based on the historical content item sequence, the self-attention encoding matrix and the asymmetric position encoding matrix, predicting the content item that the user may interact with comprises: Combining the self-attention encoding matrix and the asymmetric position encoding matrix to output a fused encoding matrix; as well as The content items that the user may interact with are predicted based on the historical content item sequence and the fused coding matrix.

7. A sequential recommendation system, comprising: An input processing module, configured to obtain a user's content item interaction history, and generate a historical content item sequence of the user based on the obtained content item interaction history; An interval representation generation module is used to generate a set of interval representations representing the interests of the user in different time intervals based on the historical content item sequence of the user, including: generating an input matrix based on the historical content item sequence; generating a query matrix Q, a key matrix K and a value matrix V based on the input matrix; and performing downsampling convolution on the key matrix K and the value matrix V to generate a convolved key matrix K conv And the convolution value matrix V conv , as the set; A dot product attention mechanism module is used to interact each content item in the historical content item sequence with each interval representation in the set to determine a self-attention encoding matrix of the historical content item sequence, including: conv And the convolution value matrix V conv Input to the dot product attention mechanism to generate a self-attention encoding matrix of the historical content item sequence; and A prediction module is used to predict content items that the user may interact with based on the historical content item sequence and the self-attention encoding matrix.

8. The sequential recommendation system according to claim 7, further comprising: an asymmetric position encoding module for generating an asymmetric position encoding matrix based on the positions of content items in the sequence of historical content items, wherein content item-position interactions are encoded in both a key matrix and a query matrix used in generating the asymmetric position encoding matrix, but content item-position interactions are not encoded in a value matrix used in generating the asymmetric position encoding matrix, The prediction module is used to predict the content items that the user may interact with based on the historical content item sequence, the self-attention encoding matrix and the asymmetric position encoding matrix.

9. The sequential recommendation system of claim 8, further comprising: a fusion encoding module, configured to combine the self-attention encoding matrix and the asymmetric position encoding matrix to output a fusion encoding matrix, The prediction module is used to predict the content items that the user may interact with based on the historical content item sequence and the fusion coding matrix.

10. A method for sequence recommendation, comprising: Acquire content item interaction histories of a plurality of users, and generate a plurality of historical content item sequences of the plurality of users based on the content item interaction histories of the plurality of users; as well as The plurality of historical content item sequences are used to train a sequence recommendation model, wherein the sequence recommendation model comprises an encoding layer, the encoding layer comprises a self-attention encoding module, wherein the self-attention encoding module is used to: The historical content item sequence is convolved to generate a set of interval representations representing the user's interests at different time intervals, including: generating an input matrix based on the historical content item sequence; generating a query matrix Q, a key matrix K and a value matrix V based on the input matrix; and performing downsampling convolution on the key matrix K and the value matrix V to generate a convolved key matrix K. conv And the convolution value matrix V conv , as the set; and Interacting each content item in the sequence of historical content items with each interval representation in the set to determine a self-attention encoding matrix for the sequence of historical content items, wherein training the sequence recommendation model is at least partially based on the self-attention encoding matrix, comprising: conv And the convolution value matrix V conv Input to the dot product attention mechanism to generate the self-attention encoding matrix of the historical content item sequence.

11. A device for sequence recommendation, comprising: Memory; as well as A processor, wherein the processor is configured to execute the method according to any one of claims 1 to 6.

12. A computer-readable storage medium storing instructions, wherein when the instructions are executed by a computer, the computer performs the method according to any one of claims 1 to 6.

Images