A Session Recommendation Method Based on Adaptive Dual Correction of User Interests

Abstract

The present invention discloses a conversation recommendation method based on dual correction of user interests with adaptability, belonging to the technical field of conversation recommendation. The method of the present invention mainly includes the following steps: initializing the high-dimensional spatial semantics and position representations of each item in the user click sequence, splicing to obtain an item representation matrix, and then generating an item representation matrix containing temporal and context information; initializing the current user interest representation by using a first adaptive correction mechanism, and then outputting the user interest representation at the next moment by using a user interest representation distillation module containing a second adaptive correction mechanism; furthermore, predicting the probability that all items in the item dictionary will be clicked at the next moment. The method of the present invention adopts an adaptive correction mechanism, effectively solves the problem that the last item click cannot accurately represent the current user interest, reduces the content of noise information in the user interest representation at the next moment, and has higher prediction accuracy.

Description

Technical Field

[0001] The present invention belongs to the technical field of conversation recommendation, and in particular relates to a conversation recommendation method based on adaptive user interest dual correction. Background Art

[0002] The goal of the recommendation system is to achieve an item recommendation mechanism that can meet the personalized needs of users. It is widely used in e-commerce platforms such as JD.com and Amazon. The conversational recommendation technology predicts the item that the user will click next time by analyzing only anonymous user behavior (i.e. user click sequence), thus achieving an item recommendation mechanism under the premise of protecting user privacy.

[0003] Using attention networks to learn user interest representations and then predict the user's clicks at the next moment is the mainstream method of existing session recommendation technologies. For example, the existing technology "Neural Attentive Session-based Recommendation" uses attention networks to capture the user's main interests to generate user interest representations. Experimental results show that the attention mechanism can reduce the impact of interest drift caused by random user clicks and short-term curiosity on prediction results.

[0004] Although the existing technology has achieved better item representation by using the attention network and improved the prediction accuracy of the item recommendation mechanism, there are still two major limitations in the use of the attention network in the existing technology: (1) The last item click cannot accurately represent the user's current interest. The existing technology often uses it as the query vector of the attention network, which will lead to unreasonable distribution of attention weights. (2) Not every item in the sequence is strongly related to the user's current interest. When the attention network is applied to a long sequence with interest drift characteristics, the noise information that is weakly related to the user's current interest will accumulate or even drown out the effective information that is strongly related to the user's current interest. Summary of the invention

[0005] The purpose of the present invention is to overcome the defects of the prior art and provide a conversation recommendation method based on adaptive user interest dual correction.

[0006] The technical problem proposed by the present invention is solved in this way:

[0007] A conversation recommendation method based on adaptive user interest dual correction includes the following steps:

[0008] Step 1. Initialize the high-dimensional space semantic representation and high-dimensional space position representation of each item in the user click sequence, and concatenate the high-dimensional space representation of the item and the item representation matrix of the user click sequence;

[0009] Step 2. Input the item representation matrix of the user click sequence into the self-attention network and the feed-forward neural network in sequence to generate an item representation matrix containing temporal and context information;

[0010] Step 3. Stack several distillation units in sequence to construct a user interest representation distillation module; the distillation unit consists of an additive attention network and a second adaptive rectification mechanism constructed by the adaptive residual method; use the first adaptive rectification mechanism to initialize the current user interest representation, and input it together with the item representation matrix of the user click sequence into the user interest representation distillation module containing the second adaptive rectification mechanism to obtain the user interest representation at the next moment;

[0011] Step 4. Predict all items in the item dictionary based on the user interest representation at the next moment to obtain the probability values of all items in the item dictionary, and take the item corresponding to the maximum probability as the item recommended at the next moment in the conversation.

[0012] Furthermore, the specific process of Step 1 is as follows:

[0013] Step 1-1. The item dictionary is V = {v1, v2,..., v |V|}, where v j represents the j-th item in the item dictionary, 1 ≤ j ≤ |V|, and |V| is the size of the item dictionary; the high-dimensional space semantic representation of the item dictionary is X = {x1, x2,..., x |V|}, where represents the high-dimensional space semantic representation of the j-th item in the item dictionary, represents a d-dimensional vector;

[0014] The high-dimensional space position representation of the conversation sequence is P = {p1, p2,..., p L}, where represents the high-dimensional space position representation of the item clicked by the user at the k-th moment in the conversation sequence, 1 ≤ k ≤ L, and L is the number of items clicked by the user in the conversation sequence;

[0015] Intercept the user click sequence with a duration of t from the conversation sequence as S t = {s1, s2,..., s t}, where s i represents the serial number of the item clicked by the user at the i-th moment in the item dictionary V, 1 ≤ i ≤ t, and t < L.

[0016] Step 1-2. Denote the high-dimensional space semantic representation and the high-dimensional space position representation of the item corresponding to s i as and p i ∈ P respectively; denote the high-dimensional space semantic representation and the high-dimensional space position representation pi Perform splicing to obtain s in the user click sequence i The corresponding high-dimensional space representation c of the item i And the item representation matrix C of the user click sequence, that is:

[0017]

[0018] C = {c1, c2,..., c t}

[0019] Among them, Concat represents splicing.

[0020] Furthermore, the specific process of step 2 is as follows:

[0021] Step 2-1. Input the item representation matrix C of the user click sequence into the self-attention network to obtain the first item representation matrix

[0022]

[0023] Among them, soffmax is the activation function, the superscript T represents transpose, and the weight parameter matrix W Q , W K , W V ∈R 2d×2d , represents a 2d×2d-dimensional matrix;

[0024] Step 2-2. Input the first item representation matrix of the user click sequence into the feed-forward neural network with a residual mechanism to obtain the second item representation matrix A of the user click sequence:

[0025]

[0026] Among them, ReLU is the activation function, and the weight parameter matrices W1, W2∈R 2d×2d , and the offset parameter b1, represents a 2d-dimensional vector.

[0027] Furthermore, the specific process of step 3 is as follows:

[0028] Step 3-1. Use the adaptive feature combination method to construct the first adaptive correction mechanism;

[0029] Based on the high-dimensional space representation c of the item corresponding to the last click in the user click sequence t , use the adaptive feature combination method to initialize the current user interest representation, and the specific process is expressed as:

[0030]

[0031] Among them, represents the initial value of the current user interest representation, and the weight parameter matrix W3,

[0032] Step 3-2. Construct a user interest representation distillation unit, which includes an additive attention network and a second adaptive correction mechanism constructed by the adaptive residual method;

[0033] Among them, the additive attention network is expressed as:

[0034] α i = W T σ(W5A i + W6u p + b3)

[0035]

[0036] Among them, u p is the current user interest representation, A i is the high-dimensional space representation of the item that the user clicks on at the i-th moment in matrix A, σ is the sigmoid activation function, and α i is the correlation score of u p with respect to A i , cor represents the output result of the additive attention network, and the weight parameter matrix W5, the offset parameter b3 ∈ R 2d ;

[0037] Based on the output result of the additive attention network, the current user interest representation is updated using the adaptive residual method, and the second adaptive correction mechanism constructed by the adaptive residual method is expressed as:

[0038] u' p = W7cor + W8u p

[0039] Among them, u' p is the updated current user interest representation, which is used as the knowledge distillation result of the distillation unit, and the weight parameter matrix

[0040] Denote the above distillation unit processing process as u' p = DU(A, u p ), where DU represents the function corresponding to the distillation unit;

[0041] Step 3-3. Stack l distillation units, where l ≥ 1, to form a user interest representation distillation module. Input and matrix A into the user interest representation distillation module, which is represented as:

[0042]

[0043]

[0044] ……

[0045]

[0046]

[0047] Among them, represents the distillation results of the 1st distillation unit to the lth distillation unit, and is used as the user interest representation for the next moment.

[0048] Furthermore, the specific process of step 4 is as follows:

[0049] Based on the user interest representation for the next moment predict all items in the item dictionary to obtain the probability values of all items in the item dictionary:

[0050]

[0051] Among them, is the probability value of the jth item in the item dictionary, L2Norm is the L2 normalization function, and w k is the normalization weight;

[0052] Select the item corresponding to the maximum probability as the item recommended for the next moment in the session.

[0053] The beneficial effects of the present invention are:

[0054] The method of the present invention uses the first adaptive correction mechanism to effectively solve the problem that the last item click cannot accurately represent the user's interest; uses the second adaptive correction mechanism to effectively reduce the content of noise information in the user interest representation for the next moment, making the prediction method based on the user interest representation for the next moment have higher accuracy. Specific embodiments

[0055] The present invention will be further described below in conjunction with embodiments.

[0056] This embodiment provides a session recommendation method based on dual correction of user interest with adaptability, including the following steps:

[0057] Step 1. Initialize the high-dimensional spatial semantic representation and high-dimensional spatial position representation of each item in the user click sequence, and splice them to obtain the high-dimensional spatial representation of the item and the item representation matrix of the user click sequence;

[0058] The specific process of Step 1 is as follows:

[0059] Step 1-1. The item dictionary is V = {v1, v2,..., v |V|}, where v j represents the j-th item in the item dictionary, 1 ≤ j ≤ |V|, and |V| is the size of the item dictionary; the high-dimensional spatial semantic representation of the item dictionary is X = {x1, x2,..., x |V|}, where x j ∈R d represents the high-dimensional spatial semantic representation of the j-th item in the item dictionary, indicating a d-dimensional vector;

[0060] The high-dimensional spatial position representation of the session sequence is P = {p1, p2,..., p L}, where represents the high-dimensional spatial position representation of the item clicked by the user at the k-th moment in the session sequence, 1 ≤ k ≤ L, and L is the number of times the user clicks on an item in the session sequence;

[0061] The user click sequence with a duration of t is intercepted from the session sequence as S t = {s1, s2,..., s t}, where s i represents the serial number of the item clicked by the user at the i-th moment in the item dictionary V, 1 ≤ i ≤ t, and t < L.

[0062] Step 1-2. Denote the high-dimensional spatial semantic representation and high-dimensional spatial position representation of the item corresponding to s i as and p i ∈P respectively; splice the high-dimensional spatial semantic representation and the high-dimensional spatial position representation p i to obtain the high-dimensional spatial representation c i of the item corresponding to s i in the user click sequence and the item representation matrix C of the user click sequence, that is:

[0063]

[0064] C = {c1, c2,..., c i}

[0065] where Concat represents splicing.

[0066] Step 2. Input the item representation matrix C of the user click sequence into the self-attention network and the feed-forward neural network in sequence to generate item representations containing temporal and context information;

[0067] The specific process of Step 2 is as follows:

[0068] Step 2-1. Input the item representation matrix C of the user click sequence into the self-attention network to obtain the first item representation matrix of the user click sequence containing temporal and context information

[0069]

[0070] where softmax is the activation function, the superscript T represents transpose, and the weight parameter matrices W Q , W K , W V ∈R 2d×2d , represents a 2d×2d-dimensional matrix; is an adjustment factor that uses the model dimension to avoid the matrix product from becoming too large.

[0071] Step 2-2. Input the first item representation matrix of the user click sequence into the feed-forward neural network with a residual mechanism to provide additional non-linear changes for the self-attention network that only contains linear changes, thereby obtaining the second item representation matrix A of the user click sequence with richer information:

[0072]

[0073] where ReLU is the activation function, the weight parameter matrix W1, the offset parameter b1, represents a 2d-dimensional vector; in the above formula, the offset parameters b1 and b2 are learnable parameters.

[0074] Step 3. Stack a number of distillation units in sequence to construct a user interest representation distillation module; the distillation unit includes two parts: an additive attention network and a second adaptive rectification mechanism constructed by an adaptive residual method;

[0075] Initialize the current user interest representation using the first adaptive rectification mechanism, and input it together with the second item representation matrix A of the user click sequence into the user interest representation distillation module containing the second adaptive rectification mechanism, thereby obtaining the user interest representation at the next moment;

[0076] The specific process of Step 3 is as follows:

[0077] Step 3-1. Construct the first adaptive rectification mechanism using the adaptive feature combination method;

[0078] Based on the high-dimensional space representation of the item corresponding to the last click in the user click sequence, use the adaptive feature combination method to initialize the current user interest representation, incorporating additional sequence average information to address the issue that the last item click cannot accurately represent the user's current interest. The specific process is as follows:

[0079]

[0080] Among them, represents the initial value of the current user interest representation, the weight parameter matrix W3,

[0081] Step 3-2. Construct a user interest representation distillation unit, which consists of an additive attention network and a second adaptive rectification mechanism constructed using the adaptive residual method;

[0082] Among them, the additive attention network is expressed as:

[0083] α i =W T σ(W5A i +W6u p +b3)

[0084]

[0085] Among them, u p is the current user interest representation, A i is the high-dimensional space representation of the item clicked by the user at the i-th moment in matrix A, σ is the sigmoid activation function, α i is the correlation score of u p with respect to A i cor represents the output result of the additive attention network, the weight parameter matrix W5, the offset parameter

[0086] Based on the output result of the additive attention network, use the adaptive residual method to update the current user interest representation, thereby weakening the impact of interest drift caused by situations such as random user clicks and short-term curiosity on the prediction result.

[0087] The second adaptive rectification mechanism constructed using the adaptive residual method is expressed as:

[0088] u′ p =W7cor+W8u p

[0089] where, u' p is the updated current user interest representation, which is the knowledge distillation result of the distillation unit, and the weight parameter matrix W7,

[0090] Denote the above distillation unit processing process as u' p = DU(A, u p ), where DU represents the function corresponding to the distillation unit;

[0091] Step 3-3. Stack l distillation units, l ≥ 1, to form a user interest representation distillation module. Input and matrix A into the user interest representation distillation module, so as to weaken the influence of user interest drift in the user click sequence on the prediction result, and realize the transformation and learning from the current user interest representation to the user interest representation at the next moment.

[0092] The user interest representation distillation module is expressed as:

[0093]

[0094]

[0095] ……

[0096]

[0097]

[0098] where, represents the distillation results of the 1st distillation unit to the lth distillation unit, and is used as the user interest representation at the next moment.

[0099] Step 4. Based on the user interest representation at the next moment, predict the probability that all items in the item dictionary will be clicked at the next moment, and take the item corresponding to the maximum probability as the item recommended at the next moment in the session.

[0100] The specific process of Step 4 is as follows:

[0101] Based on the user interest representation at the next moment, predict all items in the item dictionary to obtain the probability values of all items in the item dictionary:

[0102]

[0103] where, is the probability value of the jth item in the item dictionary, L2Norm is the L2 normalization function, and w k is the normalization weight;

[0104] Take the item corresponding to the maximum probability as the item recommended for the next moment in the session.

Claims

1. A session recommendation method based on dual correction of user interests with adaptability, characterized in that, It includes the following steps: Step 1. Initialize the high-dimensional spatial semantic representation and high-dimensional spatial position representation of each item in the user click sequence, and splice them to obtain the high-dimensional spatial representation of the item and the item representation matrix of the user click sequence; Step 2. Input the item representation matrix of the user click sequence into the self-attention network and the feed-forward neural network in sequence to generate an item representation matrix containing temporal and context information; Step 3. Stack a number of distillation units in sequence to construct a user interest representation distillation module; the distillation unit includes an additive attention network and a second adaptive correction mechanism constructed by an adaptive residual method; initialize the current user interest representation using the first adaptive correction mechanism, and input it together with the item representation matrix of the user click sequence into the user interest representation distillation module containing the second adaptive correction mechanism to obtain the user interest representation at the next moment; The specific process of Step 3 is as follows: Step 3-1. Construct the first adaptive correction mechanism using the adaptive feature combination method; Based on the high-dimensional space representation c of the item corresponding to the last click in the user click sequence t , the current user interest representation is initialized using an adaptive feature combination method, and the specific process is expressed as: Among them, represents the initial value of the current user interest representation, and the weight parameter matrix c i is the high-dimensional space representation of the item corresponding to s in the user click sequence i where s i represents the serial number of the item clicked by the user at the i-th moment in the item dictionary V, 1 ≤ i ≤ t, and t is the duration; Step 3-2. Construct a user interest representation distillation unit, which includes an additive attention network and a second adaptive correction mechanism constructed by an adaptive residual method; Among them, the additive attention network is expressed as: α i = W T σ(W5A i + W6u p + b3) Among them, u p is the current user interest representation, A i is the high-dimensional space representation of the item clicked by the user at the i-th moment in matrix A, σ is the sigmoid activation function, α i is the relevance score of u p with respect to A i , cor represents the output result of the additive attention network, the weight parameter matrix bias parameter Based on the output result of the additive attention network, update the current user interest representation using the adaptive residual method, and the second adaptive correction mechanism constructed by the adaptive residual method is expressed as: u′ p = W7cor + W8u p where u′ p is the updated current user interest representation, which is the knowledge distillation result of the distillation unit, and the weight parameter matrix Denote the above distillation unit treatment process as u′ p = DU(A, u p ), where DU represents the function corresponding to the distillation unit; Step 3-3. Stack l distillation units, where l≥1, to form a user interest representation distillation module; input and matrix A into the user interest representation distillation module, and the user interest representation distillation module is expressed as: …… Among them, represents the distillation results of the 1st to the lth distillation units, and is used as the user interest representation for the next moment; Step 4. Predict all items in the item dictionary based on the user interest representation at the next moment to obtain the probability values of all items in the item dictionary, and take the item corresponding to the maximum probability as the item recommended at the next moment in the conversation.

2. The session recommendation method based on dual correction of user interests based on adaptability according to claim 1, wherein, The specific process of Step 1 is as follows: Step 1-1. The item dictionary is V = {v1, v2,..., v |V|}, where v j represents the j-th item in the item dictionary, 1 ≤ j ≤ |V|, and |V| is the size of the item dictionary; the high-dimensional space semantic representation of the item dictionary is X = {x1, x2,..., x |V|}, where represents the high-dimensional space semantic representation of the j-th item in the item dictionary, represents a d-dimensional vector; The high-dimensional spatial position of the session sequence is represented as P = {p1, p2,..., p L}, where represents the high-dimensional spatial position representation of the item clicked by the user at the k-th moment in the session sequence, 1 ≤ k ≤ L, and L is the number of times the user clicks on an item in the session sequence; Intercept the user click sequence with a duration of t from the session sequence as S t ={s1, s2,..., s t}, where s i represents the serial number of the item clicked by the user at the i-th moment in the item dictionary V, 1 ≤ i ≤ t, t < L; Step 1-2. Let s i The high-dimensional spatial semantic representation and the high-dimensional spatial position representation of the corresponding item be denoted as and p i ∈ P; Perform splicing on the high-dimensional space semantic representation and the high-dimensional space position representation pi to obtain the item high-dimensional space representation c i corresponding to s in the user click sequence i and the item representation matrix C of the user click sequence, that is: C = {c1, c2,..., c i} Among them, Concat represents splicing.

3. The session recommendation method based on dual correction of user interests by self - adaptation according to claim 2, wherein The specific process of Step 2 is as follows: Step 2-1. Input the item representation matrix C of the user click sequence into the self-attention network to obtain the first item representation matrix where softmax is the activation function, the superscript T represents transpose, and the weight parameter matrix represents a 2d×2d dimensional matrix; Step 2-2. Input the first item representation matrix of the user click sequence into a feed-forward neural network with a residual mechanism to obtain the second item representation matrix A of the user click sequence: where ReLU is the activation function, and the weight parameter matrix bias parameter represents a 2d-dimensional vector.

4. The session recommendation method based on dual correction of user interests by adaptation according to claim 3, characterized in that The specific process of Step 4 is as follows: Based on the user interest representation at the next moment Predict all items in the item dictionary to obtain the probability values of all items in the item dictionary: wherein, is the probability value of the j-th item in the item dictionary, L2Norm is the L2 normalization function, and w k is the normalization weight; Take the item corresponding to the maximum probability as the item recommended at the next moment in the conversation.

Images