A sequential recommendation method based on multi-faceted item features and absolute time relationships

Abstract

This invention proposes a sequential recommendation method based on multifaceted item features and absolute temporal relationships, including: S1, obtaining the target user's historical interaction sequence, embedding the multifaceted item features into the user's historical interaction sequence through an embedding layer, modeling the user's preferences for multiple attributes, and simultaneously enhancing the input historical sequence to obtain an enhanced input sequence embedded with the multifaceted item features. Furthermore, the absolute temporal relationship between each interaction is derived through an absolute temporal relationship layer; S2, inputting the absolute temporal relationship and interaction sequence into a multi-layer self-attention network sequential recommendation model based on absolute temporal relationships for recommendation learning, thereby obtaining a representation of the user's multifaceted preferences; and S3, using this representation and item combination embedding to predict the user's score for each item. This method not only considers the relative position of items in the sequence but also the absolute temporal relationship between any two items, thereby improving the accuracy of the model's predictions.

Description

Technical Field

[0001] The present invention relates to a sequence recommendation method, and in particular to a sequence recommendation method based on multi-faceted features of items and absolute time relationships. Background Art

[0002] As the amount of information available online and the ways people access it increase, the problem of information overload is becoming increasingly serious. To address this issue, many platforms have incorporated personalized recommendation systems (RSs) that filter and prioritize information. Traditional recommendation algorithms treat user preferences as static, but users may be interested in completely different items at different times and locations, meaning that user preferences change dynamically over time. Consequently, sequential recommendation systems (SRSs), which leverage temporal information to dynamically model user preferences, have seen rapid development in recent years. These systems combine historical user behavior with the concept of "context" based on recent user behavior to predict the next item a user might interact with.

[0003] To model the sequential relationships of user interactions, early research relied on Markov chains (MCs) and factorization machines. With the rapid development of deep learning, various deep learning-based sequential recommendation methods have significantly optimized model performance and enhanced their ability to learn user preference representations. Therefore, deep learning-based sequential recommendation methods have become the mainstream approach for sequential recommendation. Since the introduction of the Transformer framework, the self-attention mechanism has been widely used in various recommendation models. Furthermore, numerous studies have shown that user interaction data is large in scale, and long-term and short-term dependent behavioral patterns coexist in user interaction sequences. The SASRec model is the first to apply the adaptive attention mechanism (Self-Attention) to user behavior sequence modeling. It can handle longer data and simultaneously model both long-term and short-term dependencies in user behavior. AttRec restricts the length of user input sequences and uses the self-attention mechanism to learn and model specific users' short-term preferences. It also models specific users' long-term preferences through metric learning. SSE-PT incorporates user embedding information into the model. Its variant, SSE-PT++, utilizes dynamic sequences to process long data and employs the random shared embedding (SSE) regularization method to mitigate overfitting.

[0004] While these methods have achieved good performance and demonstrated their effectiveness in improving sequential recommendation models, several key issues remain unresolved. Sequential recommendation models model interactions through sequential relationships, emphasizing the sequential nature of interactions. However, in real life, some users do not interact frequently or have not yet had time to interact, resulting in sparse historical data. Furthermore, users have varying preferences for various aspects of different items. Therefore, ignoring item features and relying solely on historical user interaction sequences for prediction cannot accurately capture users' dynamic interests and preferences for specific aspects of items. Furthermore, to improve performance metrics such as model accuracy, sequential recommendation systems often leverage relative positional relationships to build prediction models. However, in real life, even if two users have the same user-item interaction sequence within a short period of time, their recommendations should differ if the time interval between interactions is different. Items with closer time intervals will clearly have a greater impact on the next item. Therefore, even if two users' interacted items have the same relative ordinal position, they should still have different impacts on the next item if the time interval between them is different. Traditional sequential recommendation systems treat these two scenarios as equivalent and often only consider the ordinal position of each interaction in the sequence. Summary of the Invention

[0005] The present invention aims to at least solve the technical problems existing in the prior art, and in particular innovatively proposes a sequence recommendation method based on multi-faceted features of items and absolute time relationships.

[0006] To achieve the above-mentioned object of the present invention, the present invention provides a sequence recommendation method based on multiple features of items and absolute time relationships, comprising the following steps:

[0007] S1: Obtain the target user's historical interaction sequence, which includes multiple item features and interaction timestamps. Use the embedding layer to embed the multiple item features into the user's historical interaction sequence, modeling the user's preferences for multiple attributes. Simultaneously, enhance the input historical sequence to obtain an enhanced input sequence embedded with the multiple item features. Use the absolute time relationship layer to derive the absolute time relationship between interactions.

[0008] S2, inputs the absolute time relationship and interaction sequence into the multi-layer self-attention network sequence recommendation model based on absolute time relationship for recommendation learning, and obtains the user's multi-faceted preference representation;

[0009] S3 uses this representation and item combination embedding to predict the score given by the user to each item, and generates a list of the K items with the highest predicted scores and recommends them to the user.

[0010] Furthermore, the length of the interaction sequence is fixed:

[0011] Determine whether the length of the interaction sequence exceeds the set interaction sequence length value n. If it exceeds this specific value, only the most recent n interactions are selected; if it is less than this specific value, padding items 0 are added to the left of the sequence until the length is n; at the same time, the timestamp sequence corresponding to the interaction sequence is also transformed accordingly, and padding items are added to the left of the timestamp sequence. The padding items are all the timestamps corresponding to the earliest interaction item l1 in the input sequence l.

[0012] Furthermore, the input sequence after embedding multiple features of the object and enhancing it is:

[0013]

[0014] in To embed items With items i The corresponding first attribute feature embedding and the second attribute feature embedding Make a connection;

[0015] represents the item embedding matrix of the i-th interaction;

[0016] l i represents the i-th interactive item;

[0017] The absolute time relationship is expressed as:

[0018]

[0019] Where R is the user's absolute time relationship matrix;

[0020] r nn It represents the element in the nth row and nth column of the matrix R, that is, the relative time interval between the nth interaction and the nth interaction of the user, where n is the set interaction sequence length.

[0021] Furthermore, the first attribute feature is embedded and the second attribute feature embedding It compares the attention size of various aspects of the item features, selects the two attributes with the largest attention weights and deletes the rest.

[0022] Considering the limitation of time complexity, only two attributes are selected, and the attributes of other weights are some attributes that users do not pay special attention to, so they can be deleted to reduce the amount of calculation.

[0023] Furthermore, the multi-layer self-attention network sequence recommendation model is composed of several attention layers and feedforward network layers stacked alternately, and each layer of the self-attention layer and the point-by-point feedforward network layer must undergo layer normalization, residual connection and Dropout operations.

[0024] Furthermore, normalization is defined as:

[0025]

[0026] Where LayerNorm(x) means inputting x into the normalization layer;

[0027] x is a matrix representing the sample vector to be processed;

[0028] ⊙ is based on the product of elements;

[0029] μ and σ are the mean and variance of x, respectively;

[0030] ε represents the layer normalization parameter;

[0031] α and β are the learned scaling factor and bias term, respectively.

[0032] Furthermore, the prediction score of the user for item i at time step t is obtained through the prediction:

[0033]

[0034] where s i,t is the probability that the next item is i given the first t items;

[0035] F t represents the multifaceted preference representation of the user captured by the model at time step t, i.e., the output of the self-attention network layer;

[0036] express The transpose of

[0037] is the combinatorial embedding representation of item i.

[0038] In summary, due to the adoption of the above technical solution, the present invention has the following advantages:

[0039] (1) Embed the multi-faceted features of items into the user's historical interaction sequence to model the user's preferences for multiple attributes. At the same time, the input historical sequence is enhanced to predict similar items as the next items that the user may interact with, thereby improving the accuracy of the model prediction.

[0040] (2) The user interaction history sequence is considered as a sequence with different time intervals, and the relationship between any two items in the sequence is defined as their time interval. At the same time, combining the absolute time relationship and relative position, a time interval-aware self-attention mechanism is introduced to adaptively learn the weights of different items.

[0041] (3) Extensive experiments were conducted on five datasets across three platforms, demonstrating that the proposed method outperforms the state-of-the-art baselines in both metrics. Furthermore, the relationship between the attributes and absolute time of items was studied, demonstrating that embedding multiple aspects of item features can effectively capture user preferences for various aspects of items, and that integrating absolute time and relative position can improve model performance.

[0042] Additional aspects and advantages of the present invention will be set forth in part in the description which follows and, in part, will be obvious from the description which follows, or may be learned by practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of the embodiments with reference to the accompanying drawings, in which:

[0044] Figure 1 It is a structural schematic diagram of the present invention. DETAILED DESCRIPTION

[0045] The following describes embodiments of the present invention in detail. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended only to explain the present invention and are not to be construed as limiting the present invention.

[0046] To clarify the problem, we use U and I to represent the user and item sets respectively. We give a user interaction sequence represents the Lth u | interactive items, |L u | represents the length of the interaction sequence of user u, that is, the number of interactive items; where u∈U, The timestamp sequence for each interaction Indicates user u's first u | timestamps corresponding to the interactions, and |T u |=|L u |; In addition, we define a multi-faceted feature lookup table for items Where k is the number of features. The goal of the task is to predict the next item that the user may interact with. Corresponding timestamp sequence And the multi-faceted feature lookup table of all items as the input of the model, and each time step corresponds to the next item sequence of the interaction as the expected predicted output.

[0047] The present invention proposes the overall architecture of the MFITSRec model as follows: Figure 1 As shown in the figure, it mainly includes embedding layer, absolute time relationship layer, self-attention network layer and prediction layer.

[0048] Embedding layer: Utilizes item attribute information to obtain multi-faceted combined features of items and enhance item representation;

[0049] Absolute time relationship layer: The absolute time relationship between two interactions is expressed as their relative time intervals, forming an absolute time relationship matrix;

[0050] Self-attention network layer: This layer extracts features from the enhanced input sequence, absolute time relationship, and relative position input time interval perception self-attention to obtain a multi-faceted representation of user preferences.

[0051] Prediction layer: Use user preferences to predict user ratings of items, obtain recommendation sequences, and finally push them to target users.

[0052] The absolute time relationship layer preprocesses the original data of the timestamp, and the output absolute time relationship matrix is ​​also used as the input of the embedding layer. The embedding layer embeds all the used and preprocessed data.

[0053] The proposed model combines item information with user history, enabling sequences of limited length to carry useful information about various aspects of an item. When interaction data is sparse, MFITSRec can better capture users' preferences for specific aspects of items, thereby recommending items that meet these preferences to the target user. Furthermore, MFITSRec considers not only the relative position of items in the sequence but also the absolute temporal relationship between any two items, namely the time interval.

[0054] 1. Embedding layer

[0055] Since different users have different number of interactions, we fix the length of each user's interaction sequence to a specific value represents a set of natural numbers; and it is assumed that when the number of user interactions exceeds this specific value, the impact of the first several interactive items on the prediction of the user's next item can be ignored. For user u∈U, the interaction sequence After fixing the length, a new input sequence l=(l1,l2,…,ln ), if |L u |-1 is greater than n, then only the last n interactions in the interaction sequence are selected; if |L u If |-1 is less than n, padding 0 is added to the left of the sequence until the length is n. At the same time, the timestamp sequence corresponding to the interaction sequence Also perform the corresponding transformation and get t=(t1,t2,…,t n ). Different from the input sequence, padding items are added to the left side of the timestamp sequence. The padding items are all the timestamps corresponding to the earliest interaction item l1 in l.

[0056] We build an item embedding matrix where d i is the latent dimension of the item, and retrieves the item embedding of the user input sequence to form the input embedding matrix

[0057]

[0058] E is the sequence of the original data after the fixed length;

[0059] represents the first interaction-related item embedding vector;

[0060] represents the second interaction-related item embedding vector;

[0061] represents the embedding vector of the nth interaction-related item.

[0062] For the items in I, we use their attributes to represent multiple aspects of the item, enhancing its representation. We experimented with multiple attributes and compared their attention weights. However, due to time constraints, we removed some attributes that users didn't particularly focus on and selected the two with the highest attention weights. Specifically, we derived the attention weights m from the self-attention network layer in the trained model, compared each attribute, and selected the two with the highest attention weights.

[0063] By comparison, we found that category and brand are the two attributes with the largest attention weight. Therefore, we defined an item category lookup table and item brand lookup table

[0064]

[0065] in and are the category and brand embeddings of item i, d g and d bare the feature potential dimensions of category and brand respectively. We retrieve the attribute feature embeddings of the corresponding items in the sequence from the item category lookup table and the item brand lookup table respectively, and fuse them with the item embedding in the input embedding matrix, denoted as E I , d represents the latent dimension of the model:

[0066]

[0067] in To embed items With items i Corresponding category attribute feature embedding and brand attribute feature embedding Connect, and d=d i +d g +d b The introduction of item attribute features enables sequences of limited length to carry more useful information and capture users’ preferences for specific aspects of items.

[0068] Since multiple user interactions may occur at the same absolute timestamp, using only absolute timestamps may not accurately model the order in the input sequence. Therefore, we still need to use learnable relative position embedding. For the keys and values ​​in the self-attention network layer, we use and Indicate their relative positions:

[0069]

[0070] The relative positions of keys and values ​​in the self-attention network layer are used in Equations (9) and (11).

[0071] 2. Absolute time relationship layer

[0072] We introduce an absolute time relationship into the model to model the impact of user interactions at different intervals on the prediction of the next item. Since different users interact with each other at different frequencies, the time intervals between their interactions are not on the same order of magnitude, making it difficult to model. Therefore, we express the absolute time relationship between two interactions as their relative time intervals. For a fixed-length time series t = (t1, t2, ..., t n ), the absolute time relationship between item i and item j is:

[0073]

[0074] in is the minimum value of the user's absolute time interval, and Represents a set of natural numbers. We assume that when the interval between two user interactions is too long, the impact on predicting the user's next item is negligible, so we give a maximum time interval t max , so that when the relative time interval is too large, it is limited to t max After the restriction, the absolute time relationship between all interactions constitutes the user's absolute time relationship matrix

[0075]

[0076] The elements on the main diagonal are all 0. Similar to relative position embedding, we use and They represent the absolute time relationship between keys and values ​​in the self-attention network layer:

[0077]

[0078] The elements

[0079] Since the key and value in formula (7) use the same matrix, namely the absolute time relationship matrix R, in order to achieve separate training of keys and values, the keys and values ​​are separated by formula (8). At the same time, two embedding operations are required to obtain two different matrices R K and R V , and then input them into the model for training respectively without interfering with each other.

[0080] 3. Self-Attention Network Layer

[0081] Since the self-attention mechanism can capture the sequential relationship in the sequence, we use the self-attention mechanism to model the user's historical interaction behavior. When calculating the proportional dot product attention, we consider both the relative position in the input sequence and the absolute temporal relationship between each item in the sequence.

[0082] 3.1 Self-attention layer: For a given input sequence E I , a new sequence S=(S1,S2,…,S n ), where each element It is the weighted sum of the linearly transformed input element, relative position embedding, and absolute time relationship embedding:

[0083]

[0084] in is the projection matrix of the value, m ij is the self-attention weight, defined as:

[0085]

[0086]

[0087] in and The projection matrix and scale factor of the query and key respectively Used to scale the dot product attention to avoid the inner product value being too large.

[0088] To avoid information leakage, that is, the output at time step t contains the influence of subsequent items on predicting the next item, we use a mask to prohibit the linking of subsequent keys to the current query at time step t.

[0089] 3.2 Point-by-point feedforward network: Since self-attention is a linear model, in order to make the model nonlinear, a two-layer point-by-point feedforward network is used after the self-attention layer, with the ReLU activation function in the middle:

[0090] F i =FFN(S i )=ReLU(S i W1+b1)W2+b2 (12)

[0091] The weight matrix Bias vector

[0092] 3.3 Stacking Layers: We stack b self-attention layers and feedforward layers to learn more complex object transformations. The stacking pattern is: attention layer - feedforward layer - self-attention layer - feedforward layer, etc. To address the problems of overfitting and vanishing gradients associated with stacking, we perform layer normalization, residual connections, and dropout on each self-attention layer and point-by-point feedforward layer:

[0093] g(x)=x+Dropout(g(LayerNorm(x))) (13)

[0094] Where g(x) represents a self-attention layer or a point-wise feed-forward network layer. Layer normalization is defined as:

[0095]

[0096] where ⊙ is the element-wise product, μ and σ are the mean and variance of x, x represents the sample vector to be processed, ε represents the layer normalization parameter to prevent the denominator from being zero, and α and β are the scale factor and bias term for learning, respectively.

[0097] 4. Prediction Layer

[0098] After b self-attention layers and feedforward network layers, we obtain a multifaceted user preference representation that integrates item attributes and absolute time relationships, and calculate the user's predicted score for item i at time step t:

[0099]

[0100] Among them F t represents the user's multifaceted preference representation captured by the model at time step t, i.e., the output of the self-attention network layer, express The transpose of s i,t Given the first t items, that is, (l1,l2,…,l t ), the probability that the next item is i, is the combined embedding representation of item i. We generate a list of the K items with the highest scores and recommend them to the user.

[0101] The proposed method consists of three main steps. First, the target user's historical interaction sequence is obtained, along with various features of the corresponding items in the sequence and the timestamps of the interactions. This enhances the interaction representation and derives the absolute temporal relationship between each interaction. Second, the absolute temporal relationship and interaction sequence are input into a multi-layer self-attention network sequential recommendation model based on absolute temporal relationships for recommendation learning, thereby obtaining a multifaceted representation of the user's preferences. Finally, this representation is used in conjunction with the item combination embedding to predict the user's score for each item.

[0102] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to the embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the claims and their equivalents.

Claims

1. A sequence recommendation method based on multi-faceted features of items and absolute time relationships, characterized by: The following steps are involved: S1. Obtain the target user's historical interaction sequence, which includes item multi-faceted features and interaction timestamps. Embed the item multi-faceted features into the user's historical interaction sequence through the embedding layer to model the user's preferences for multiple attributes. Enhance the input historical sequence to obtain an enhanced input sequence embedded with the item multi-faceted features. At the same time, use the absolute time relationship layer to derive the absolute time relationship between each interaction. The enhanced input sequence embedded with the item multi-faceted features is: in To embed items With items i The corresponding first attribute feature embedding and the second attribute feature embedding Connect; the first attribute feature is embedded and the second attribute feature embedding It compares the attention size of various features of an item, selects the two attributes with the largest attention weights and deletes the rest; represents the item embedding matrix of the i-th interaction; l i represents the i-th interactive item; The absolute time relationship is expressed as: Where R is the user's absolute time relationship matrix; r nn The element in the nth row and nth column of the matrix R represents the relative time interval between the nth and nth interactions of the user, where n is the set interaction sequence length; S2, inputs the absolute time relationship and input sequence into the multi-layer self-attention network sequence recommendation model based on absolute time relationship for recommendation learning, and obtains the user's multi-faceted preference representation; S3 uses the user's multifaceted preference representation and item combination embedding to predict the score the user gives to each item, and generates a list of the K items with the highest predicted scores and recommends them to the user.

2. The method for sequential recommendation based on multiple item features and absolute time relationships according to claim 1, characterized in that: The length of the interaction sequence is fixed: Determine whether the length of the interaction sequence exceeds the set interaction sequence length value n. If it exceeds this specific value, only the most recent n interactions are selected; if it is less than this specific value, padding items 0 are added to the left of the sequence until the length is n; at the same time, the timestamp sequence corresponding to the interaction sequence is also transformed accordingly, and padding items are added to the left of the timestamp sequence. The padding items are all the timestamps corresponding to the earliest interaction item l1 in the input sequence l.

3. The method for sequential recommendation based on multiple item features and absolute time relationships according to claim 1, characterized in that: The multi-layer self-attention network sequence recommendation model is composed of several self-attention layers and point-by-point feedforward network layers stacked alternately, and each layer of the self-attention layer and the point-by-point feedforward network layer must undergo layer normalization, residual connection and Dropout operations.

4. The method for sequence recommendation based on multiple item features and absolute time relationships according to claim 1, characterized in that: Normalization is defined as: Where LayerNorm(x) means inputting x into the normalization layer; x is a matrix representing the sample vector to be processed; ⊙ is based on the product of elements; μ and σ are the mean and variance of x, respectively; ε represents the layer normalization parameter; α and β are the learned scaling factor and bias term, respectively.

5. The method for sequence recommendation based on multiple item features and absolute time relationships according to claim 1, characterized in that: The prediction score of the user for item i at time step t is obtained through the prediction: where s i,t is the probability that the next item is i given the first t items; F t represents the multifaceted preference representation of the user captured by the model at time step t, i.e., the output of the self-attention network layer; express The transpose of is the combinatorial embedding representation of item i.

Images