A sequence recommendation method and device focusing on user core interests

Abstract

The present invention discloses a sequence recommendation method and device focusing on the core interests of users, including obtaining the interaction sequence between users and items and the timestamp corresponding to each interaction behavior in the interaction sequence; obtaining the embedding matrix of the interaction sequence; performing self-attention calculation on the embedding matrix to obtain the probability distribution of attention values ​​of each query for all keys; obtaining the predefined fixed default probability distribution of each query; obtaining the activity measurement of each query based on the similarity of two probability distributions; calculating the attention value of each key based on the activity measurement to construct a self-attention matrix; and obtaining the project recommendation result of the user according to the self-attention matrix. The present invention adds a time interval to the embedding layer and adds an activity measurement index, so as to adaptively measure the correlation between the project and the core interests of the user, thereby improving the model expression ability and the accuracy of the recommendation results.

Description

Technical Field

[0001] The present invention relates to the field of recommendation systems, and in particular to a sequence recommendation method and device focusing on a user's core interests. Background Art

[0002] Traditional recommendation systems, such as collaborative filtering, model the interaction between users and items in a static way. In contrast, sequential recommendation systems treat user-item interactions as a dynamic sequence and take into account their sequential relevance. The research focus of sequential recommendation is how to concisely capture useful patterns from continuous dynamic behaviors to obtain accurate recommendations.

[0003] The Markov chain-based method is a typical example, which makes a simplifying assumption: the next action is conditioned on one or more recent actions. Its disadvantage is obvious, and it may not be able to capture the complex dynamics in some complex scenarios. Another representative work is to use recursive neural networks for sequential recommendation. Given a user's historical interaction sequence, the recursive neural network-based sequential recommendation attempts to predict the user's next interaction by modeling the sequence dependency. However, limited by its strict unidirectional architecture, the recursive neural network-based sequential recommendation is prone to pseudo-dependencies and is difficult to perform parallel training.

[0004] In recent years, inspired by the Transformer model for machine translation, adopting self-attention mechanism for sequence recommendation has become a research trend. Models based on self-attention mechanism can emphasize the truly relevant and important interactions in the sequence while reducing irrelevant interactions. Therefore, compared with models based on Markov chains and recurrent neural networks, they have higher flexibility and expressiveness.

[0005] Generally, when modeling a user's interaction sequence, the present invention hopes to generate a user's interest representation and make predictions based on it. However, in real-life scenarios, not all interactions between users and items reflect the user's interests. On the one hand, interaction sequences often contain user interest drift caused by accidental clicks. On the other hand, in some cases, users may find that they have no actual interest in the items they interact with. For example, watching a movie they don't like or buying a piece of clothing that doesn't fit. Therefore, when generating a user's interest representation, considering all of the above items may not play a positive role, and may even be counterproductive. The present invention collectively refers to these interactions that do not represent the user's true interests and have no effect on the user's subsequent behavior as noise interactions. In contrast, core interests reflect the user's deep preference for items and dominate their selection of candidate items. Therefore, discovering interactions that represent the user's core interests from the user's interaction sequence is crucial for generating user interest representations and predicting candidate items. Models based on the classic self-attention mechanism achieve a certain degree of emphasis on important items by performing scaled dot product calculations of queries and keys between all items. However, current research still lacks a method to clearly distinguish between core interest-related interactions and noise interactions to directly eliminate the negative impact of the latter. In addition, the simplified assumption made by most previous models is to treat the interaction history as an ordered sequence without considering the time interval between each interaction. This approach will lead to the loss of effective information, because the time interval between interactions is also part of the user's behavior pattern and should be included in the user's interest representation. It can be seen that the existing sequence recommendation methods have many defects. Summary of the invention

[0006] In order to overcome the shortcomings of existing sequence recommendation methods, the present invention provides a sequence recommendation method that focuses on the core interests of users, which can explicitly distinguish core interest-related interactions from noise interactions to directly eliminate the negative impact of the latter. At the same time, the time interval between interactions is considered to retain effective information.

[0007] The technical contents of the present invention include:

[0008] A sequential recommendation method focusing on a user's core interests, the steps of which include:

[0009] Obtaining an interaction sequence between a user and an item and a timestamp corresponding to each interaction behavior in the interaction sequence;

[0010] In combination with the timestamp, obtaining an embedding matrix of the interaction sequence;

[0011] Perform self-attention calculation on the embedding matrix to obtain each query q i For all key attention value probability distributions p, and by setting a scaling exponential function, calculate each query qi A predefined fixed default probability distribution q;

[0012] According to the similarity between the attention value probability distribution p and the predefined fixed default probability distribution q, each query q is obtained i Activity measurement of

[0013] Calculating the attention value of each key based on the activity metric to construct a self-attention matrix;

[0014] According to the self-attention matrix, an item recommendation result for the user is obtained.

[0015] Furthermore, the embedding matrix of the interaction sequence is obtained by the following steps:

[0016] 1) converting the interaction sequence into a detection sequence with a fixed length of l;

[0017] 2) constructing a timestamp sequence based on the timestamp and the detection sequence;

[0018] 3) For each item in the detection sequence, it is mapped to a vector X through a one-dimensional convolution filter i , and each vector X i Superposition, we get the item embedding matrix;

[0019] 4) obtaining a position embedding matrix according to the position of each item in the detection sequence;

[0020] 5) Obtaining a time interval embedding matrix by calculating the time interval between each time stamp in the time stamp sequence;

[0021] 6) Based on the item embedding matrix, the position embedding matrix and the time interval embedding matrix, an embedding matrix of the interaction sequence is obtained.

[0022] Furthermore, the time interval embedding matrix is ​​obtained by the following steps:

[0023] 1) Obtain the shortest time interval among the time intervals between timestamps;

[0024] 2) Divide each time interval by the shortest time interval to obtain a personalized time interval;

[0025] 3) Based on the personalized time interval, construct a time interval sequence with a length of l-1;

[0026] 4) After the rightmost side of the time interval sequence of length l-1 is filled with 0 to a length of l, a time interval embedding matrix is ​​obtained through projection and superposition.

[0027] Furthermore, the method for calculating the similarity includes: using KL divergence for measurement.

[0028] Furthermore, the activity metric Where K represents a key, L K ×d is the dimension of key K, k j is the j-th row vector of key K, and μ is a constant that controls the importance of recent behavior.

[0029] Furthermore, the self-attention matrix is ​​constructed by the following steps:

[0030] 1) Based on each query q i Activity measurement to obtain active queries for each key;

[0031] 2) calculating the self-attention value of the active query;

[0032] 3) For inactive queries, use the predefined fixed default probability distribution q as the self-attention value;

[0033] 4) The self-attention values ​​of active queries and inactive queries are recombined according to their original positions to obtain the self-attention matrix.

[0034] Furthermore, when training the self-attention network that constructs the self-attention matrix, parameter feedback is performed through a two-layer feedforward neural network.

[0035] Further, the project recommendation result of the user is obtained through the following steps:

[0036] 1) The self-attention matrix is ​​processed through layer normalization method, residual connection method and dropout method to obtain user interest representation;

[0037] 2) Based on the user’s interest representation and the item embedding matrix of each item, calculate the user’s preference score for each item;

[0038] 3) Sort the items according to the preference scores, and use the items with the highest scores as the item recommendation results.

[0039] A storage medium stores a computer program, wherein the computer program is configured to execute the above method when running.

[0040] An electronic device includes a memory and a processor, wherein the memory stores a program for executing the above method.

[0041] Compared with the prior art, the present invention has the following beneficial effects:

[0042] 1. This paper proposes a new attention model that can directly and explicitly eliminate the influence of irrelevant items, thereby focusing more attention on items that are truly relevant to user interests. Among them, a novel activity metric is designed to adaptively measure the relevance between items and user core interests.

[0043] 2. The present invention takes the time interval into consideration in the embedding layer, which significantly improves the expressiveness of the model without incurring huge additional computational costs.

[0044] 3. The evaluation results on multiple benchmark datasets show that the proposed method outperforms the existing sequence recommendation models and reaches the state-of-the-art level, and the proposed components play an important role respectively. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 This is a flow chart of the sequence recommendation method focusing on the core interests of the user according to the present invention.

[0046] Figure 2 This is a diagram showing the recommended effect test under different potential dimensions and the effect comparison with other models according to an embodiment of the present invention.

[0047] Figure 3 This is a test diagram of the recommended effects under different sampling factors according to an embodiment of the present invention.

[0048] Figure 4 This is a visualization diagram of the attention matrix of the first head of a user in a multi-head perspective according to an embodiment of the present invention.

[0049] Figure 5 This is a visualization diagram of the attention matrix of the second head of a user in a multi-head perspective in an embodiment of the present invention. DETAILED DESCRIPTION

[0050] The following will be combined with the accompanying drawings to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only specific embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work.

[0051] like Figure 1 As shown, the present invention can be divided into 5 steps in total:

[0052] S1: Obtain the interaction sequence between the user and the item and the timestamp corresponding to each interaction behavior, and truncate or fill the interaction sequence and timestamp sequence to a fixed length.

[0053] The specific construction steps of the data preprocessing module are as follows:

[0054] S1.1: Interaction sequence for user u Where n u is the number of items, then take the sequence As training data, it is converted into a sequence of fixed length l (s1, s2, ..., s l ). If the original sequence length is greater than l, the latest l interaction items are intercepted; if the original sequence length is less than l, the sequence is padded with 0 to the left.

[0055] S1.2: Timestamp sequence for user u Where n u is the number of items, then take the sequence As training data, it is converted into a sequence of fixed length l (t1, t2, ..., t l ). If the original sequence length is greater than l, the latest l interaction items are intercepted; if the original sequence length is less than l, Fill to the left of the sequence.

[0056] S2: Establish the embedding layer, which mainly includes scalar mapping embedding, position embedding and time interval embedding.

[0057] The specific steps for constructing the embedding matrix of each part are as follows:

[0058] S2.1: Scalar mapping. For a sequence (s1, s2, ..., s l ) for each item s i , which is mapped to a d-dimensional vector X through a one-dimensional convolution filter i . where d is the latent dimension. After stacking all items together, we get the embedding matrix of the items:

[0059] S2.2: Position embedding. Since the self-attention model does not contain recursive or convolutional modules, it cannot determine the actual position of the item in the sequence, so a position embedding module is added to identify the specific position of the item in the sequence. A learnable matrix is ​​used here as a position embedding matrix.

[0060] S2.3: Time interval embedding. For the timestamp series (t1, t2, ..., t l ), calculate the time difference between two adjacent timestamps. For each user, the present invention only cares about the relative length of the time interval in the sequence. Therefore, for all time intervals, the present invention divides them by the shortest time interval in the user sequence As a personalized time interval: After obtaining the time interval sequence of length l-1, fill it with 0 on the far right to a length of l, and then obtain the user's time interval embedding matrix through projection and superposition:

[0061] S2.4: The final embedding matrix is ​​the sum of the above three parts:

[0062]

[0063] in

[0064] S3: Establish a self-attention network layer focusing on core interests. A novel evaluation index (activity metric) is designed to adaptively measure the correlation between items and user core interests. According to this evaluation index, the user's interactive items can be divided into two parts: noise interactions and core interest-related interactions, and the attention values ​​of the two parts are obtained in different calculation methods. The two attention values ​​are fused to obtain the user interest representation.

[0065] The specific steps for constructing the self-attention network layer with core interest focus are as follows:

[0066] S3.1: The standard self-attention model form is as follows:

[0067]

[0068] Where Q, K, and V represent query, key, and value respectively, and their respective dimensions are Use q i ,k i ,v i represents the i-th row vector of Q, K, V, then q i The attention value is converted into kernel smoothing in the form of probability:

[0069]

[0070] Where p(k j |q i )=k(q i ,k j ) / ∑ ； k(q i ,k l ), is a probability distribution, representing the attention value of the i-th query for all keys. The self-attention mechanism combines all values ​​according to this probability distribution as the final output. Represents q i and k j The correlation between them.

[0071] S3.2: Query activity measurement. Query activity is a concept proposed by the present invention to represent the correlation between items and user core interests. First, a scaling exponential function is set as the default distribution:

[0072]

[0073] where μ is a constant that controls the importance of recent actions. p(k j |q i ) is the probability distribution actually calculated, q(k j |q i ) is a predefined fixed default probability distribution. If q i If the corresponding items can represent the core interests of the user, the actual calculated probability distribution p will be quite different from the fixed distribution q. Use KL divergence to measure the similarity between distributions q and p:

[0074]

[0075] Removing the last constant term, the present invention defines the activity metric of the i-th query as:

[0076]

[0077] The core interest of users in items drives the attention probability distribution of corresponding queries away from the fixed distribution. If the query corresponding to an item has a larger M(q i ,K), the greater the probability that it corresponds to the user's core interest.

[0078] S3.3: Based on the proposed activity metric, the present invention enables each key to focus only on m active queries, and then obtains the self-attention value focused on the core interest:

[0079]

[0080] in is a sampling matrix of the matrix Q, which contains only m active queries. m is calculated as Where c∈(0,1) is the sampling factor. It is worth mentioning that in the multi-head perspective, this attention extracts different active query-key pairs for each head, thus avoiding serious information loss. For the remaining (L Q -m) queries, without calculation, but directly using the default distribution q(k j |q i ) as its attention value. Finally, the two parts of the attention value are recombined according to the original position to obtain the final attention matrix S.

[0081] S4: A point-by-point feedforward neural network is established, in which the Relu activation function is used to give the model nonlinearity. To address the possible problems of overfitting, gradient vanishing and slow training speed of the model, layer normalization, residual connection and dropout techniques are introduced respectively.

[0082] The steps for establishing a point-by-point feedforward neural network are as follows:

[0083] S4.1: After each attention layer, a two-layer feedforward neural network is used, and Relu is used as the activation function, which can give the model nonlinearity and consider the interaction between different latent dimensions:

[0084] FFN(S)=ReLU(SW (1) +b (1) )W (2) +b (2)

[0085] Where S is the attention matrix obtained in step S3.3, W (1) , W (2) is the weight matrix, and its dimensions are b (1) , b (2) is the bias vector, and its dimensions are

[0086] S4.2: As the self-attention layer and feedforward layer are stacked and the network goes deeper, some problems become more serious, including overfitting, gradient vanishing, and slower training process. This paper introduces layer normalization, residual connection, and dropout technology to solve these problems and obtain the user interest representation S′

[0087] S'=S+Dropout(FFN(LayerNorm(S)))

[0088] S5: Generate a recommendation list for the user based on the finally obtained user interest representation and candidate item set.

[0089] The specific construction steps of its prediction layer are as follows:

[0090] S5.1: After steps S1-4, the present invention adaptively and hierarchically extracts the user interest representation S′ from the previous items. In order to predict the next item, the present invention uses a latent factor model to calculate the user's preference score for item i:

[0091] R i,t =S′X i

[0092] in is the embedding vector of item i obtained in step S2.1.

[0093] S5.2: Sort the candidate items according to the calculated user preference scores for the items, and select the k items with the highest scores to recommend to the user.

[0094] In the embodiment of the present invention, the effectiveness and feasibility of the proposed sequence recommendation system focusing on the core interests of users are verified through experiments, and the performance of the system is verified through three experiments.

[0095] First, consider the influence of the latent dimension d. Figure 2 As shown in FIG. 1 , the NDCG@10 of the latent dimension d from 10 to 100 is obtained while keeping other optimal hyperparameters unchanged. As the latent dimension increases, the recommendation performance improves and gradually approaches the convergence point. And the model of the present invention is always better than other baseline models.

[0096] Secondly, consider the impact of the query sampling factor c. Figure 3 The figure shows the change process of query sampling factor c from 0.2 to 1.0 for the ML-1m and Beauty datasets. The model performance of the ML-1m dataset reaches the optimal point when c=0.8, while the model of the Beauty dataset reaches the optimal point when c=0.5, which means that the sparse dataset contains more noise interactions.

[0097] In addition, the experiment in this embodiment also verifies that under the multi-head perspective, the attention mechanism proposed by the present invention can extract different active query-key pairs for each head. Figure 4 Figure 5 The following is a visualization of the attention score matrix of a user on candidate items under the two-head attention mechanism. It can be clearly seen that different active query-key pairs are extracted on different heads.

[0098] Finally, it should be noted that the described embodiments are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

Claims

1. A sequential recommendation method focusing on a user's core interests, comprising the following steps: Obtaining an interaction sequence between a user and an item and a timestamp corresponding to each interaction behavior in the interaction sequence; In combination with the timestamp, obtaining an embedding matrix of the interaction sequence; Perform self-attention calculation on the embedding matrix to obtain each query q i For all key attention value probability distributions p, and by setting a scaling exponential function, calculate each query q i A predefined fixed default probability distribution q; According to the similarity between the attention value probability distribution p and the predefined fixed default probability distribution q, each query q is obtained i Activity measurement Among them, K represents a key, L K ×d is the dimension of key K, k j is the j-th row vector of key K, μ is a constant that controls the importance of recent behavior; Calculating the attention value of each key based on the activity metric to construct a self-attention matrix; wherein calculating the attention value of each key based on the activity metric to construct the self-attention matrix includes: Based on each query q i Activity measurement, get the m active queries for each key, c∈(0,1) is the sampling factor, and the dimension of the query Q is Get the self-attention values ​​of m active queries V represents the value in the self-attention model, is the sampling matrix of query Q and contains only m active queries; For the remaining (L Q -m) queries q i , using the default distribution q(k j |q i ) as the attention value; Recombine the two attention values ​​according to their original positions to obtain the final self-attention matrix; According to the self-attention matrix, an item recommendation result for the user is obtained.

2. The method according to claim 1, characterized in that The embedding matrix of the interaction sequence is obtained by the following steps: 1) converting the interaction sequence into a detection sequence with a fixed length of l; 2) constructing a timestamp sequence based on the timestamp and the detection sequence; 3) For each item in the detection sequence, it is mapped to a vector X through a one-dimensional convolution filter i , and each vector X i Superposition, we get the item embedding matrix; 4) obtaining a position embedding matrix according to the position of each item in the detection sequence; 5) Obtaining a time interval embedding matrix by calculating the time interval between each time stamp in the time stamp sequence; 6) Based on the item embedding matrix, the position embedding matrix and the time interval embedding matrix, an embedding matrix of the interaction sequence is obtained.

3. The method according to claim 2, characterized in that The time interval embedding matrix is ​​obtained by the following steps: 1) Obtain the shortest time interval among the time intervals between timestamps; 2) Divide each time interval by the shortest time interval to obtain a personalized time interval; 3) Based on the personalized time interval, construct a time interval sequence with a length of l-1; 4) After the rightmost side of the time interval sequence of length l-1 is filled with 0 to a length of l, a time interval embedding matrix is ​​obtained through projection and superposition.

4. The method according to claim 1, characterized in that The method of calculating the similarity includes: using KL divergence for measurement.

5. The method according to claim 1, characterized in that When training the self-attention network that constructs the self-attention matrix, parameter feedback is performed through a two-layer feedforward neural network.

6. The method according to claim 1, characterized in that The project recommendation result of the user is obtained by following the steps below: 1) The self-attention matrix is ​​processed through layer normalization method, residual connection method and dropout method to obtain user interest representation; 2) Based on the user’s interest representation and the item embedding matrix of each item, calculate the user’s preference score for each item; 3) Sort the items according to the preference scores, and use the items with the highest scores as the item recommendation results.

7. A storage medium storing a computer program, wherein: The computer program is configured to execute the method according to any one of claims 1 to 6 when executed.

8. An electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to perform the method according to any one of claims 1 to 6.

Images