A weighted multi-interest category enhanced recall method for recommender systems

Abstract

The present invention provides a recommendation system recall method with weighted multi-interest category enhancement, including an item candidate pool, user-item historical interaction data, and a backend server. Its working method is: based on the historical interaction data between users and items, by modeling the user's historical item sequence and item category sequence, the user's weighted multi-interest vector is learned, and items are recommended to the user from the item candidate pool based on the user's weighted multi-interest. The present invention applies the item category features and the user's weighted multi-interest modeling technology to the recall stage of the recommendation system, which greatly improves the accuracy and recall rate of the recall results of the recommendation system.

Description

Technical Field

[0001] The present invention relates to the technical field of recommendation system recall, and in particular to a recommendation system recall method with weighted multi-interest category enhancement. Background Art

[0002] With the popularization of Internet technology, people are exposed to massive amounts of data every day through terminal devices such as smartphones. For users, only part of the data is of interest, and redundant information will affect the user experience. The recommendation system is designed to help users filter out useless information and intelligently recommend information to users. The recommendation system usually takes the user's historical behavior log as input. The historical behavior log includes the user's purchase, evaluation, and browsing behavior logs. The user's portrait is obtained through behavior log modeling. After obtaining the user portrait, the recommendation algorithm can recommend information of interest to the user. The application of the recommendation system provides many conveniences to the majority of users. An accurate and efficient recommendation system can tap into the user's potential consumption tendencies and increase the user's stickiness.

[0003] Recommendation systems generally include recall, rough sorting, fine sorting, and re-ranking. The task of the recall stage is to quickly filter out some data from the entire information set for use in subsequent stages. The performance of the recall stage has a direct impact on the subsequent rough sorting, fine sorting, and re-ranking modules, so the recall stage is very important in the recommendation system.

[0004] The recall module in the existing recommendation system still has the following problems: 1) Most existing recall methods only model a single interest when modeling users, but in reality, users' interests are often diverse; 2) The existing recall module cannot reasonably use the category features that are strongly related to items due to model design reasons. The above two problems will lead to inaccurate modeling of users and poor recommendation results. Summary of the invention

[0005] Purpose of the invention: The technical problem to be solved by the present invention is to provide a recommendation system recall method based on weighted multi-interest category enhancement in view of the shortcomings of the existing user interest modeling technology in the recall stage of the recommendation system. This method improves the accuracy and recall rate of the recommendation system recall stage by using the user's historical item sequence and item category sequence to model the user's weighted multi-interest vector.

[0006] The present invention discloses a recommendation system recall method with weighted multi-interest category enhancement, comprising the following steps:

[0007] Step 1: Extract the most recent n historical interaction item sequences S of user u from the entire data set u and the category sequence Q corresponding to the item u ;

[0008] Step 2: vectorize the user's historical interaction item sequence and item category sequence;

[0009] Step 3: Send the two vectorized results into the self-attention network to obtain the overall information of the sequence;

[0010] Step 4: Send the overall sequence information into the multi-interest extraction network to extract the multi-interest vector of user u;

[0011] Step 5: Send the user's multi-interest vector to the weighted multi-interest evolution layer to learn the weight of the multi-interest vector at the next moment;

[0012] Step 6: Combine user u's multiple interest vectors and interest weights to calculate the scores with other items, and make recommendations based on the size of the item scores.

[0013] In step 1, the data in the entire data set is a common user and item interaction data set, including user number u, item number i, item category number c and item interaction time t. According to the user number u, the historical interaction item sequence is aggregated. If the sequence length exceeds n, the most recent n are intercepted. If it is less than n, 0 is added to the left. According to the user number u, the historical interaction item sequence is aggregated. If the sequence length exceeds n, the most recent n are intercepted. If it is less than n, 0 is added to the left until the length is n, and the historical interaction item sequence S is obtained. u and its corresponding item category sequence Q u The value of n can be 20 or 50, which is set according to the actual data set.

[0014] Step 2 includes: vectorizing the item number as e, where The dimension of e is d, which is usually 64. The historical interaction item sequence S of user u is u The vectorized representation is in represents the second item in the history of user u’s interaction. Similarly, we can get E u The meaning of the other e in . Similarly, the vectorization of the item category sequence is obtained as δ represents the operation of taking the item category.

[0015] Step 3 includes the following steps:

[0016] Step 3-1, define the learnable parameter W Q , W K , W V , C u Respectively with W K and W VMultiply them to get the intermediate parameters K and V, and vectorize the historical interaction item sequence E u With W Q Multiply to get the intermediate parameter Q, where Q, K,

[0017] Q=E u ·W Q ,

[0018] K=C u ·w K ,

[0019] V=C u ·w v ;

[0020] Step 3-2, use the following formula to calculate the attention weight R1 between the vectorized representation of the historical interaction item sequence and the vectorized representation of the item category sequence:

[0021]

[0022] Among them, softmax is a normalization operation for array data. For example, for the array [x1, x2, ..., x m ], for x i The softmax operation is:

[0023]

[0024] T represents the transpose of the matrix, d k is the dimension of the last dimension of K;

[0025] Step 3-3, use the following formula to calculate the result H of the association between the historical interaction item sequence and the item category sequence:

[0026] H=R1.V+E u ,

[0027]

[0028] Step 4 includes the following steps:

[0029] Step 4-1, the multi-interest extraction network is used to extract k interests of user u from H, and define W1 and W2 as trainable parameters. Calculated The physical meaning of A1 is the k interests corresponding to each item in the historical interactive item sequence; the multi-interest extraction network first performs nonlinear compression on the original input H, converting it from Compress to The expression is as follows:

[0030]

[0031] A1=softmax(tanh(HW1)W2)

[0032] Where tanh is the activation function, e x is an exponential function, e is the base of natural logarithm, approximately equal to 2.718281828;

[0033] Step 4-2: Calculate the vector representation V of the user's k interests u :

[0034]

[0035] V u It represents the k most common interests of users;

[0036] Step 5 includes the following steps:

[0037] Step 5-1: Model the differentiation of each user’s interest based on V u , H, E u and C u Get the weight A2 of the user's multiple interests in the process of purchasing items:

[0038]

[0039]

[0040] Step 5-2: Use the multi-head self-attention mechanism to learn the user's multi-interest change process. The last result is taken to represent the k interest importance scores A3 of the user at the next moment. In the formula, ":" is the operation of taking all the data in the array, where multi_attention is a multi-head attention mechanism:

[0041] A3=multi_attention(A2,A2,A2)[:,-1],

[0042]

[0043] Step 6 includes the following steps:

[0044] Step 6-1, training phase, first find the target object e t Most similar interests u As the current stage of user interest:

[0045] M u =V u [argmax(softmax(V u e t )A3),:]

[0046] Where argmax is the subscript of the maximum element in the array, and the “:” in the formula is the operation of taking all the data in the array;

[0047] Maximize user u and target item e t The probability of interaction between user u and other target items is calculated using the sampling softmax method, through the objective function shown below To train the model:

[0048]

[0049] Sample is a random sampling operation, e i is the result of a random sampling operation;

[0050] Step 6-2: Set the user's interest V u And the interest weight score A3 is applied in the test phase. Each interest vector can independently retrieve the top N related items in the item pool by using the nearest neighbor library Faiss, and obtain the recommended item set S(u, I) of user u through the following calculation expression:

[0051]

[0052] After obtaining the specific score corresponding to each interest, it is weighted with the interest weight score and then sorted uniformly. The top N items with the highest scores are recommended to the user, where N is generally 20 or 50.

[0053] Beneficial effects: The present invention utilizes the user's historical items and their category sequence characteristics, first designs a network that reasonably mines the item category sequence characteristics, designs and models the user's weighted multi-interest vector structure, and provides an accurate and effective recall method. Compared with the traditional recall method, the biggest feature of the present invention is that it introduces the strongly correlated category features of the items when modeling the historical item sequence, and at the same time models and learns the user's weighted multi-interest vector. According to the experimental results, this method can greatly improve the accuracy and recall rate of the recall stage. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] The present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments, and the above and / or other advantages of the present invention will become more clear.

[0055] Figure 1 It is a schematic diagram of the overall architecture of the recommendation system.

[0056] Figure 2 It is a schematic diagram of the user's historical purchase sequence.

[0057] Figure 3This is the overall flow chart of the recall method of the recommendation system based on multi-interest category enhancement. DETAILED DESCRIPTION

[0058] Figure 1 The recall stage is at the front link of the recommendation system, and the recall results have an important impact on the subsequent modules of the recommendation system.

[0059] Figure 3 In one embodiment, the present invention provides an overall flow chart of a weighted multi-interest category enhanced recommendation system recall method, which includes the following contents:

[0060] Step 1: Extract the most recent n historical interaction item sequences S of user u from the entire data set u and the category sequence Q corresponding to the item u ,like Figure 2 As shown, it is the user's historical purchase sequence;

[0061] Step 2: vectorize the user's historical interaction item sequence and the corresponding category sequence respectively;

[0062] Step 3: Send the vectorization result to the self-attention network to obtain the overall sequence information after the two sequences are vectorized;

[0063] Step 4: Send the overall sequence information into the multi-interest extraction network to extract the multi-interest vector of the user at the next moment;

[0064] Step 5: Send the user's multi-interest vector at the next moment to the weighted multi-interest evolution layer to learn the weight of the multi-interest vector at the next moment;

[0065] Step 6: Combine the multiple interest vectors and interest weights of user u to calculate the scores with other items, and output a certain number of items according to the score sorting.

[0066] In step 1, the data is a common user-item interaction dataset, which contains user ID u, item ID i, item category ID c, and item interaction time t. The historical interaction item sequence is aggregated according to the user ID u. If the sequence length exceeds n, the most recent n items are intercepted. If it is less than n, zeros are added on the left to a length of n, where n can be 20 or 50, to obtain the item sequence S u and its corresponding item category sequence Q u .

[0067] In step 2, vectorization refers to the dense vectorization representation of the item number. The dimension of e is d, and d is generally 64. The historical item sequence of user u is vectorized as in represents the second item in the history of user u’s interaction. Similarly, we can get E u The meanings expressed by the other e in . Similarly, we can get the vectorized representation of the historical item category sequence δ() is the operation of taking the item category.

[0068] Step 3 includes the following steps:

[0069] Step 3-1, define learnable parameters C u Respectively with W K and W V Multiply them to get K and V, and vectorize the item sequence E u With W Q Multiply them to get Q, where Q, K,

[0070] Q=E u ·W Q

[0071] K=C u ·W K

[0072] V=C u ·W V

[0073] Step 3-2, multiply Q and K to get R1, T represents the transpose of the matrix, d k is the last dimension of K, R1 is the attention weight between the vectorized representation of the modeled item sequence and the vectorized representation of the item category sequence, where softmax is the normalization operation for the array data:

[0074]

[0075] Step 3-3, after obtaining the attention weights between the item sequence and the item category sequence, the result of the association between the item sequence and the item category sequence is obtained through the following formula:

[0076]

[0077] Step 4 includes the following steps:

[0078] Step 4-1, the multi-interest extraction network aims to extract k interests of user u from H, and defines W1 and W2 as trainable parameters. Calculated The physical meaning of A1 is the k interests corresponding to each item in the item sequence. The essence of the first step of the multi-interest extraction network is to perform nonlinear compression on the original input H, converting it from Compress to Where tanh is the activation function, the expression is as follows, e x is an exponential function, e is the base of the natural logarithm, approximately equal to 2.718281828:

[0079]

[0080] A1=softmax(tanh(HW1)W2)

[0081] Step 4-2, multiplying A1 by input H can further obtain the user's k interests V u The k most common interests of a user:

[0082]

[0083] Step 5 includes the following steps:

[0084] Step 5-1: The above steps obtain the vectorized representation of the user's k interests. Model the differentiation of each user's interest, based on V u , H, E u and C u Get the weight of multiple interests of users in the process of purchasing items

[0085]

[0086] Step 5-2: Use the multi-head self-attention mechanism to learn the user's multi-interest change process, and take the last result to represent the user's k interest importance scores Among them, multi_attention is the multi-head attention mechanism operation, and the ":" in the formula is the operation of taking all the data in the array:

[0087] A3=multi_attention(A2,A2,A2)[:,-1]

[0088] Step 6 includes the following steps:

[0089] Step 6-1, training phase, find out the t Most similar interests u As the user's interest at the current stage, argmax is the operation of taking the maximum value of the array element subscript, and the formula ":" is the operation of taking all the data in the array:

[0090] M u =V u [argmax(softmax(V u e t )A3), :]

[0091] Maximize user u and target item e t The probability of user u interacting with other target items is calculated using the sampling method. The model is trained using the following objective function, where the Sample operation is a random sampling operation:

[0092]

[0093] Step 6-2, test phase, the user's interest V u The interest weight score A3 is applied in the testing phase. Each interest vector can independently retrieve the first N related items in the project pool by using the nearest neighbor library Faiss, and then get the specific score and weight it with the interest weight score and then sort it uniformly. The first N items in the score ranking will be recommended to the user, N is generally 20 or 50, and the calculation expression is as follows:

[0094]

[0095] by Figure 2 As an example, assume that the dimensions of the three multi-interest vectors of user u are 3 and the recall number N is 2. u =[[0.12,0.14,0.43],[0.34,0.64,0.23],[0.62,0.64,0.13]], the corresponding interest weight A3 =[0.3,0.3,0.4], among the corresponding 5 items, lipstick =[0.22,0.24,0.63], computer =[0.66,0.31,0.23], mouse =[0.88,0.24,0.73], shoes =[0.22,0.04,0.73] and mobile phone =[0.68,0.32,0.33]. The scores of the five items for each interest of user u are multiplied by the weight coefficient A3, and the two items with the highest overall scores are mobile phones and computers, as shown in Table 1 (scores of the five items for user u interest 1), Table 2 (scores of the five items for user u interest 2), and Table 3 (scores of the five items for user u interest 3):

[0096] Table 1

[0097] Item Name Initial score Increase weight Lipstick 0.2007716 0.06023148 computer 0.179966 0.0539898 mouse 0.22686791 0.068060373 shoe 0.20380588 0.061141764 cell phone 0.18858861 0.056576583

[0098] Table 2

[0099] Item Name Score Increase weight Lipstick 0.18390781 0.055172343 computer 0.20373795 0.061121385 mouse 0.23552913 0.070658739 shoe 0.16557671 0.049673013 cell phone 0.21124838 0.063374514

[0100] Table 3

[0101] Item Name Score Increase weight Lipstick 0.16481941 0.065927764 computer 0.21495966 0.085983864 mouse 0.25140127 0.100560508 shoe 0.14691444 0.058765776 cell phone 0.22190522 0.088762088

[0102] In order to verify the effectiveness of this method, this method is compared with other excellent models on the recommendation system public dataset Amazonbooks. The experimental results are shown in Table 4. The experimental results show that this method can greatly improve the accuracy and recall rate of the recall stage. The compared models are all the best performing methods in the recall field in recent years. The year information is marked in Table 4.

[0103] Table 4

[0104]

[0105] The present invention provides a method for recalling a recommendation system with weighted multi-interest category enhancement. There are many methods and ways to implement the technical solution. The above is only a preferred implementation of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the protection scope of the present invention. All components not specified in this embodiment can be implemented by existing technologies.

Claims

1. A weighted multi-interest category enhanced recommendation system recall method, characterized in that: The following steps are involved: Step 1: Extract the most recent n historical interaction item sequences S of user u from the entire data set u and the corresponding item category sequence Q u ; Step 2: vectorize the user's historical interaction item sequence and item category sequence; Step 3: Send the vectorized result of step 2 to the self-attention network to obtain the overall information of the sequence; Step 4: Send the overall information of the sequence into the multi-interest extraction network to extract the multi-interest vector of user u; Step 5: Send the multi-interest vectors to the weighted multi-interest evolution layer to learn the weights of the multi-interest vectors at the next moment; Step 6: Combine user u's multiple interest vectors and weights to calculate the scores with other items, and make recommendations based on the item scores. In step 1, the entire data set includes user number u, item number i, item category number c and item interaction time t. The historical interaction item sequence is aggregated according to the user number u. If the sequence length exceeds n, the most recent n are intercepted. If it is less than n, 0 is added to the left until the length is n, and the historical interaction item sequence S is obtained. u and its corresponding item category sequence Q u ; Step 2 includes: vectorizing the item number as e, where The dimension of e is d; the historical interaction item sequence S of user u u The vectorized representation is in Represents the second item in the history interaction of user u; the item category sequence is vectorized as δ represents the operation of taking the item category; Step 3-1, define learnable parameters C u Respectively with W K and W V Multiply them to get the intermediate parameters K and V, and vectorize the historical interaction item sequence E u With W Q Multiply to get the intermediate parameter Q, where Q=E u ·W Q , K=C u ·W K , V=C u ·W V ; Step 3-2, use the following formula to calculate the attention weight R1 between the vectorized representation of the historical interaction item sequence and the vectorized representation of the item category sequence: Among them, softmax is the normalization operation for array data; T represents the transpose operation of the matrix, d k is the dimension of the last dimension of K; Step 3-3, use the following formula to calculate the result H of the association between the historical interaction item sequence and the item category sequence: H=R1 V+E u , Step 4 includes the following steps: Step 4-1, the multi-interest extraction network is used to extract k interests of user u from H, and define W1 and W2 as trainable parameters. Calculated A1 is the k interests corresponding to each item in the historical interactive item sequence. The multi-interest extraction network first performs nonlinear compression on the original input H to convert it from Compress to The expression is as follows, A1=softmax(tanh(HW1)W2) Where tanh is the activation function, e x is an exponential function, and e is the base of the natural logarithm; Step 4-2: Calculate the vector representation V of the user's k interests u : Step 5 includes the following steps: Step 5-1: Model the differentiation of each user’s interest based on V u , H, E u and C u Get the weight A2 of the user's multiple interests in the process of purchasing items: Step 5-2, use the multi-head self-attention mechanism to learn the user's multi-interest change process, take the last result to represent the user's k interest importance scores A3 at the next moment, ":" is to take all the data in the array: Among them, multi_attention is a multi-head attention mechanism; Step 6 includes the following steps: Step 6-1, training phase, first find the target object e t Most similar interests u As the current stage of user interest: M u =V u [argmax(softmax(V u e t )A3),:] Where argmax is the subscript of the maximum element in the array. In the formula: is the operation of taking all the data in the array; Maximize user u and target item e t The probability of interaction between user u and other target items is calculated using the sampling softmax method, through the objective function shown below To train the model: Sample is a random sampling operation, e i is the result of a random sampling operation; Step 6-2, obtain the recommended item set S(u, I) of user u through the following calculation expression: After obtaining the specific score corresponding to each interest, it is weighted with the interest weight score and then sorted uniformly, and the top N items in the score are recommended to the user.

Images