An interpretable efficient sequence recommendation method with auxiliary information

Abstract

The application discloses an interpretable efficient sequence recommendation method introducing auxiliary information. The application explicitly models user interest as a probability distribution sequence of next interaction items; in addition to taking the item transition matrix as the input of the autoencoder, the application introduces item auxiliary information in the hidden layer and combines the classification task to extract better item feature representation, and then obtains the probability distribution matrix representing the overall user interest. When performing incremental recommendation, the updated probability distribution of the next interaction item is obtained by integrating the new item transition information, and the updated personalized recommendation is obtained by weighting the next interaction item probability distribution sequence according to the updated user interaction sequence, attention and time decay weight. The modeling process of the application is relatively transparent and has certain interpretability; the new user-item interaction is integrated without changing the model parameters, and instant recommendation is realized at the incremental level of single interaction.

Description

Technical Field

[0001] This invention relates to the field of recommender system technology, and more specifically, to an interpretable and efficient sequence recommendation method that incorporates auxiliary information. Background Technology

[0002] Recommender systems model user preferences based on historical behavioral data to recommend items or content that users may be interested in. Traditional recommender systems often represent user behavior data as a static user-item interaction matrix. However, with the rapid proliferation of the internet and mobile devices, user behavior data has become richer and more dynamic, and static user-item interaction matrices are often insufficient to meet the requirements of dynamism. User interests may change over time, and recommender systems need to be able to dynamically update and capture these changes in a timely manner. Furthermore, recommender systems not only need to provide high-quality predictions but also need to provide explanations for the prediction results to gain user trust. Existing sequence recommendation methods do not yet meet practical needs in terms of accuracy, rapid incremental updates, and interpretability. More sophisticated recommender systems should have the ability to dynamically update and rapidly capture changes in user interests and provide explanations for the recommendation results. Summary of the Invention

[0003] To address the shortcomings of existing technologies, the present invention aims to provide an interpretable and efficient sequence recommendation method that incorporates auxiliary information. This method is applicable to dynamic recommendation scenarios, integrating new user-item interactions without altering model parameters. This enables real-time recommendations at the incremental level of a single interaction, and explicitly models user interests as a probability distribution sequence of items for the next interaction. The modeling process is relatively transparent and possesses a certain degree of interpretability.

[0004] This invention proposes an interpretable and efficient sequence recommendation method that incorporates auxiliary information, explicitly modeling user interests as a probability distribution sequence of items for the next interaction. The method includes an overall preference generator based on an item transition graph and a personalized interest modeler using parameterless attention and time decay modules. The overall preference generator is based on a multi-information autoencoder, which, in addition to using the item transition matrix as input, introduces auxiliary item information in the hidden layer and combines it with a classification task to extract better item feature representations. During incremental recommendation, the updated probability distribution of items for the next interaction is obtained by integrating the new item transition information. Then, the updated personalized recommendation is obtained by weighting the probability distribution sequence of items representing user interests with attention and time decay weights based on the updated user interaction sequence.

[0005] The technical solution of the present invention is described in detail below.

[0006] An interpretable and efficient sequence recommendation method incorporating auxiliary information models user interests as a probability distribution sequence of items for the next interaction, and combines overall user interests with the user's personalized interests to achieve updated personalized recommendations; including the following steps.

[0007] (1) Preprocess the original user-item interaction data and construct the item transfer matrix;

[0008] (2) Input the item transfer matrix into the at most information autoencoder for low-rank approximation to obtain the embedded representation of the item transfer matrix after the encoder compression. Then input the embedded representation of the item transfer matrix into the decoder to obtain the reconstructed item transfer matrix. Construct the reconstruction loss based on the difference between the input matrix and the reconstructed matrix.

[0009] Item auxiliary information is introduced into the hidden layer of the autoencoder. The category information of the item is used as the true value. The embedded representation of the item transition matrix is ​​processed by softmax to obtain the predicted value in the classification task. A multi-label classification loss is constructed to optimize the latent space representation of the item transition matrix.

[0010] The reconstruction loss and multi-label classification loss are integrated into a total loss for learning. The hyperparameters are adjusted, and the autoencoder is trained iteratively until convergence. This allows the decoder to recover the original data as much as possible, and to make items with the same category label have similar feature representations during the feature extraction process of the encoder, thus obtaining a well-trained overall preference generator model.

[0011] (3) After training the overall preference generator, the reconstructed item transition matrix is ​​processed by softmax in rows to obtain the probability distribution of the next interactive item, which is the probability distribution matrix representing the overall user interest.

[0012] (4) The probability distribution matrix representing the overall user interest is weighted by time decay according to the interaction sequence of each user to obtain the probability distribution sequence of the next interactive item of each user after time decay weighting; the probability distribution sequence of the next interactive item of each user after time weighting is weighted by attention mechanism, and the distribution probability of each user's future interaction is obtained by personalizing user interests.

[0013] (5) For a new interaction data generated by the user, construct the updated item transfer matrix according to the method in step (1) and update the user's interaction sequence; input the updated item transfer matrix into the trained overall preference generator model to obtain the updated probability distribution matrix representing the overall user interest;

[0014] (6) Following the method in step (4), the updated probability distribution matrix representing the overall user interest is subjected to time decay weighting and attention weighting based on the updated user interaction sequence to personalize user interests, thereby obtaining the updated distribution probability of the user's future interactions and realizing the updated personalized recommendation.

[0015] In this invention, in step (1), the original user-item interaction data is preprocessed by discarding users and items with fewer than 5 observations; the item transfer matrix is ​​constructed based on the interaction history of all users, and each row in the item transfer matrix records which item the user will interact with next after interacting with the corresponding source item, which is based on the statistical data of all users.

[0016] In this invention, in step (2), the multi-information autoencoder adopts a stacked autoencoder consisting of multiple autoencoders stacked together.

[0017] In this invention, in step (2), the Frobenius norm is used as the reconstruction loss and the cross-entropy is used as the multi-label classification loss.

[0018] In this invention, in step (3), the probability distribution matrix P representing the overall user interests is as follows:

[0019]

[0020] In the P matrix, the value in the i-th row and j-th column represents the probability that the user interacts with item j immediately after interacting with item i. This represents the reconstructed item transfer matrix.

[0021] In this invention, in step (4), for user u, there exists an interaction sequence S. u From S u S′ is obtained by taking the nearest 'a' interactions. u ∈R a According to S′ u P is obtained by taking the corresponding row from the probability distribution matrix P representing the overall user interests. u ∈R a×n , where n is the number of items;

[0022] First, regarding P u Perform time decay:

[0023] f(t) = exp(βt / T) max -1)

[0024] Where β is a hyperparameter controlling the attenuation intensity, and T max It is the maximum time of the current interaction sequence;

[0025] Then, through the self-attention mechanism, P...u Each row in the array can contain this interaction sequence S′ u All information in:

[0026]

[0027] Use the item that user u last interacted with. As a query, P′ u Using the key and value, the final probability distribution is obtained through an attention mechanism:

[0028]

[0029] In this invention, when steps (5) and (6) are executed multiple times, continuous prediction of the item the user will interact with next is achieved.

[0030] Compared with existing technologies, the advantages of this invention are as follows: This invention is based on a multi-information autoencoder, which models user interests according to various types of information to improve recommendation accuracy. It also integrates new user-item interactions without changing the model parameters, achieving instant recommendations at the incremental level of a single interaction. This extends static recommendations to dynamic recommendations, and significantly improves both accuracy and efficiency compared to existing methods. Furthermore, it explicitly models user interests as a probability distribution sequence of items for the next interaction. The modeling process is relatively transparent and therefore has a certain degree of interpretability, which helps to increase users' trust in the recommendation system. Attached Figure Description

[0031] Figure 1 This is a schematic diagram of the algorithm flow of the present invention.

[0032] Figure 2 This is a schematic diagram of the algorithm model of the present invention.

[0033] Figure 3 This is a diagram of the overall preference generator model architecture of the present invention. Detailed Implementation

[0034] The method of the present invention will be further described below with reference to the accompanying drawings and the description of the invention.

[0035] This invention provides an interpretable and efficient sequence recommendation method that incorporates auxiliary information. The specific steps of this method are as follows:

[0036] Preprocess the original user-item interaction data and construct an item transfer matrix;

[0037] The item transfer matrix is ​​input into the overall preference generator based on a multi-information autoencoder. After compression by the encoder, the embedded representation of the item transfer matrix is ​​obtained. The embedded representation of the item transfer matrix is ​​then input into the decoder to reconstruct the item transfer matrix.

[0038] By introducing item auxiliary information and combining it with a multi-label classification task during the training of the overall preference generator, the embedding representation of the item transition matrix is ​​optimized using the item category information as the true value.

[0039] The item transition matrix reconstruction loss and multi-label classification loss are integrated into a total loss for learning. The model is trained iteratively until convergence and the overall preference generator model is saved to obtain the probability distribution of the next interactive item representing the overall user interest.

[0040] Design a user personalized interest modeler to capture dynamic personalized interests in user interaction sequences through parameterless time decay and attention mechanisms, and obtain the probability distribution of user future interactions.

[0041] When a new interaction occurs, the item transition matrix is ​​incrementally modified and input into the loaded overall preference generator model to obtain the reconstructed result of the updated item transition matrix. Based on the updated user interaction sequence, the distribution probability of the user's future interactions is obtained after parameterless time decay and attention weighting.

[0042] The method of the present invention will be described in detail below.

[0043] This section begins by stating the problem. To further clarify the research problem, it is described and defined in detail. Each user-item interaction can be represented by a tuple (u (τ) i (τ) ), where τ = 1, 2, ... represents the order in which the interactions occur. Assume there are a total of N interactions, which are represented as... The problem solved by this invention can be defined as follows: Given a user u, a recommendation system needs to predict which item user u will interact with in the (N+1)th interaction. The specific steps are as follows:

[0044] Step 1: Preprocess the original user-item interaction data and construct an item transfer graph / matrix based on the interaction history of all users. Each row in the item transfer matrix records which item the user will interact with next after interacting with the corresponding source item. This is based on statistical data of all users.

[0045] Step 2: Input the item transition matrix into the overall preference generator based on a multi-information autoencoder, where the item transition matrix T (N) The element in row i′ and column i″ of the storage item pair is defined as:

[0046]

[0047] Because of T (N)Typically very sparse, an autoencoder is used to perform a low-rank approximation of the item transition matrix. To extract features better, a stacked autoencoder, consisting of multiple autoencoders stacked together, is employed. Its encoder representation is as follows:

[0048] h u =T CN)

[0049] h i =f i (W i ·h i-1 +b i For i = 1, 2, ..., l, after passing through the last encoder layer, the embedded representation of the item transfer matrix after encoder compression is obtained:

[0050]

[0051] Step 3: Input the embedded representation of the item transition matrix into the decoder to reconstruct the item transition matrix:

[0052]

[0053] h′ i =g i (W′ i ·h′ i-1 +b′ i ), for i = 1, 2, ..., l

[0054] The decoder is symmetrical to the encoder. After the last decoder layer, the reconstructed item transition matrix is ​​obtained:

[0055]

[0056] The Frobenius norm is used as a measure of the difference between the input matrix and the reconstructed matrix:

[0057]

[0058] Step 4: Introduce auxiliary item information. During training, combine this with a multi-label classification task, using the item category information as the true value, and optimize the latent space representation of the item transition matrix. After applying softmax processing to the latent space representation (embedded representation) of the item transition matrix obtained in Step 2, the predicted value for the classification task is obtained.

[0059]

[0060] For this multi-label classification task, the cross-entropy loss function is designed as follows:

[0061]

[0062]

[0063] in This represents the true value of the i-th sample in the j-th category. This represents the predicted value of the i-th sample in the j-th category.

[0064] Step 5: Loss function of the overall preference generator The loss consists of the transition matrix reconstruction loss and the multi-label classification task loss, which is minimized during training through backpropagation. This allows the decoder to recover the original data as much as possible, while ensuring that items with the same category label have similar feature representations during the encoder's feature extraction process.

[0065]

[0066] Where α is a hyperparameter that controls the importance of reconstruction loss and classification loss;

[0067] Step 6: Adjust the hyperparameters based on the results of Step 5, train iteratively until convergence, and save the overall preference generator model;

[0068] Step 7: After training the overall preference generator, perform softmax processing on the reconstructed item transition matrix row by row to obtain the probability distribution of the next interactive item:

[0069]

[0070] In the P matrix, the value in the i-th row and j-th column represents the probability that a user interacts with item j immediately after interacting with item i, thus obtaining the matrix P representing the overall user interests.

[0071] Step 8: Design a user personalized interest modeler to capture dynamic personalized interests in user interaction sequences through time decay and attention mechanisms. For user u, there exists an interaction sequence S. u From S u S′ is obtained by taking the nearest 'a' interactions. u ∈R a According to S′ u Take the corresponding row from P to get P u ∈R a×n First, regarding P... u The time decay process, similar to the concept of position embedding, aims to emphasize the importance of closer interactions.

[0072] f(t) = exp(βt / T) max -1)

[0073] Where β is a hyperparameter controlling the attenuation intensity, and T max It is the maximum time of the current interaction sequence;

[0074] Step 9: Using self-attention mechanism to make P u Each row in the array can contain this interaction sequence S′ u All information in:

[0075]

[0076] Use the item that user u last interacted with. As a query, P′ u Using the key and value, the final probability distribution is obtained through an attention mechanism:

[0077]

[0078] Step 10: For a newly generated interaction data from a user, namely user u (whose interaction sequence is S) u In the (N+1)th interaction, interact with item i. First, construct the updated item transition matrix T according to the method in step 1. (N+1) And update user u's interaction sequence S u This allows us to obtain the latest overall preferences and the latest interaction status of user u.

[0079] Step 11: Load the overall preference generator model from Step 6 and take the updated item transition matrix as input to obtain the updated probability distribution matrix P representing the overall user interests. Integrate the new user-item interaction data without changing the model parameters.

[0080] Step 12: Update the user interaction sequence S using the methods described in steps 8 and 9. u By applying time decay and attention weighting, we obtain the updated probability distribution p of the user's future interactions. u Steps 10, 11, and 12 can be executed multiple times as needed to achieve continuous prediction of the items the user will interact with next.

[0081] In step 2, the item transfer matrix is ​​input into the overall preference generator based on a multi-information autoencoder. The encoder forces the model to learn the most important features by encoding the input into low-dimensional latent variables. In this process, the embedded representation of the item transfer matrix after compression by the encoder is obtained. In step 3, the decoder restores the latent variables obtained by dimensionality reduction to the initial input dimension, that is, the embedded representation of the item transfer matrix is ​​input into the decoder to obtain the reconstructed item transfer matrix and construct the reconstruction loss. An autoencoder learns how to "copy" the input to the output, thus enabling it to encode the input into a low-dimensional representation. The reconstructed item transition matrix is ​​a low-rank approximation of the original. Since the item transition matrix can represent both overall interest (which item to interact with next after interacting with the current item) and item attributes (representing items using items), step 4 introduces auxiliary information about the items (item category) to obtain a better embedding representation. The result from step 2 is then processed using softmax to obtain the predicted value for the classification task, and a multi-label classification loss is constructed. Because items of the same category should have more similar embedding representations, step 5 of this invention combines the multi-label classification task during training, considering both the reconstruction loss and the multi-label classification task loss to optimize the learning process of the embedding representation. This combines the reconstruction loss from step 3 with the loss from step 4. Multi-label classification loss is integrated into a total loss for learning; step 6 adjusts hyperparameters based on the results of step 5, iteratively trains the overall preference generator model until convergence, and saves the overall preference generator model; step 7, after the overall preference generator is trained, obtains the reconstructed item transition matrix, i.e., the probability distribution of the next interactive item (overall user preference); since using only the overall preference generator does not consider the user's personalized interests, the recommendation list generated by the model is not optimal, therefore, step 8 of this invention designs a personalized interest modeler that includes parameterless time decay and attention modules, and applies time decay weighting to the results of step 7 according to the interaction sequence of each user to obtain the probability distribution sequence of the next interactive item for each user after time decay weighting; step 9 further weights the results of step 8 through a parameterless attention mechanism to obtain the distribution probability of each user's future interaction. When modeling personalized interests, the time decay and attention mechanism are designed to be parameterless, which means that the model does not need time-consuming parameter learning, which helps to shorten the model's running time. Because user interests are explicitly modeled as a probability distribution sequence of items for the next interaction and a parameter-free time decay and attention mechanism is used, the overall preference prediction, attention weights, and time decay weights corresponding to each user interaction are all intermediate results of the model. These can be used to analyze how the model makes the final prediction based on these intermediate results, thus giving the model a certain degree of transparency and interpretability. In this invention, the item transition matrix input to the multi-information autoencoder is based on statistical data from all users (i.e., the overall user base), and this invention supports incremental recommendation.Step 10: For a new interaction data point generated by the user, first construct the updated item transition matrix using the method in Step 1, and update the user's interaction sequence; Step 11: Load the overall preference generator model from Step 6 and use the updated item transition matrix as input. During this process, the updated probability distribution matrix representing the overall user interest can be obtained without changing the model parameters of the overall preference generator.

[0082] Step 12 applies parameter-free time decay and attention weighting to the updated user interaction sequence using the methods in steps 8 and 9, obtaining the updated probability distribution of the user's future interactions. The incremental recommendation process in steps 10, 11, and 12 can be executed multiple times as needed to achieve continuous prediction of the items the user will interact with next.

[0083] In a specific embodiment, the original user-item interaction data is first preprocessed, discarding users and items with fewer than 5 observations. This invention employs the same data partitioning method as JODIE and CoPE, splitting the dataset chronologically into a training set (the first 80% of interactions), a validation set (the next 10% of data), and a test set (the last 10% of data). In the training dataset, an item transition graph / matrix is ​​constructed based on the interaction history of all users. Each row in the item transition matrix records which item the user will interact with next after interacting with the corresponding source item, based on statistical data from all users.

[0084] This invention conducts experiments on the classic recommender system datasets ML-100K and ML-1M. The experiments use MRR and Recall@10 metrics to evaluate the performance of the proposed method (AutoXplain) on future link prediction tasks. Table 1 shows a comparison of the results of AutoXplain with and without auxiliary information on the ML-100K and ML-1M datasets, and five existing state-of-the-art interaction graph modeling methods.

[0085] Table 1

[0086]

[0087] On all datasets, regardless of whether auxiliary information is considered, the proposed method outperforms all baseline methods. On all datasets, AutoXplain without auxiliary information achieves a minimum improvement of 24.49%–37.50% in MRR and 18.92%–38.82% in Recall@10; however, the performance of AutoXplain is further improved after introducing auxiliary information. Compared to not considering auxiliary information, AutoXplain with auxiliary information achieves an improvement of 27.27%–27.87% in MRR and 27.97%–32.58% in Recall@10. To evaluate the efficiency of the proposed method, it is compared with JODIE, CoPE, and the parameterless FreeGEM datasets on the ML-100K and ML-1M datasets, and the results are shown in Table 2.

[0088] Table 2

[0089]

[0090] Compared to JODIE, the method of this invention, AutoXplain, is at least 307.5 times faster; compared to CoPE, AutoXplain is at least 589.0 times faster; and compared to the equally parameterless FreeGem, AutoXplain is also at least 1.9 times faster. The above experiments demonstrate the superior efficiency of the method of this invention.

Claims

1. An interpretable and efficient sequence recommendation method that incorporates auxiliary information, characterized in that, The user's interests are modeled as a probability distribution sequence of items to be interacted with next. This is combined with overall user interests and the user's individual interests to achieve updated personalized recommendations. This includes the following steps: (1) Preprocess the original user-item interaction data and construct the item transfer matrix; (2) Input the item transfer matrix into the at most information autoencoder for low-rank approximation to obtain the embedded representation of the item transfer matrix after the encoder compression. Then input the embedded representation of the item transfer matrix into the decoder to obtain the reconstructed item transfer matrix. Construct the reconstruction loss based on the difference between the input matrix and the reconstructed matrix. Item auxiliary information is introduced into the hidden layer of the autoencoder. The category information of the item is used as the true value. The embedded representation of the item transition matrix is ​​processed by softmax to obtain the predicted value in the classification task. A multi-label classification loss is constructed to optimize the latent space representation of the item transition matrix. The reconstruction loss and multi-label classification loss are integrated into a total loss for learning. The hyperparameters are adjusted, and the autoencoder is trained iteratively until convergence. This allows the decoder to recover the original data as much as possible, and to make items with the same category label have similar feature representations during the feature extraction process of the encoder, thus obtaining a well-trained overall preference generator model. (3) After training the overall preference generator, the reconstructed item transition matrix is ​​processed by softmax in rows to obtain the probability distribution of the next interactive item, which is the probability distribution matrix representing the overall user interest. (4) The probability distribution matrix representing the overall user interest is weighted by time decay according to the interaction sequence of each user to obtain the probability distribution sequence of the next interactive item of each user after time decay weighting; the probability distribution sequence of the next interactive item of each user after time weighting is weighted by attention mechanism, and the distribution probability of each user's future interaction is obtained by personalizing user interests. (5) For a new interaction data generated by the user, construct the updated item transition matrix according to the method in step (1) and update the user's interaction sequence; input the updated item transition matrix into the trained overall preference generator model to obtain the updated probability distribution matrix representing the overall user interest; (6) Following the method in step (4), the updated probability distribution matrix representing overall user interests is subjected to time decay weighting and attention weighting based on the updated user interaction sequence to personalize user interests, thereby obtaining the updated distribution probability of future user interactions and realizing the updated personalized recommendation; where: In step (3), the probability distribution matrix representing the overall user interests It is expressed as follows: in, The first in the matrix Line 1 The values ​​in the column represent the user's interaction with the item. Immediately interact with the item The probability of interaction. This represents the reconstructed item transition matrix; In step (4), for the user There exists an interaction sequence ,from Take the nearest one The interaction was obtained ;according to From the probability distribution matrix representing the overall user interests Take the corresponding row to get ,in For the quantity of items; First of all Perform time decay: in It is a hyperparameter that controls the attenuation intensity. It is the maximum time of the current interaction sequence; Then through the self-attention mechanism... Each line in the array can contain this interaction sequence. All information in: User The last item to interact with As a query Using the key and value, the final probability distribution is obtained through an attention mechanism: 。 2. The interpretable and efficient sequence recommendation method incorporating auxiliary information according to claim 1, characterized in that, In step (1), the original user-item interaction data is preprocessed by discarding users and items with fewer than 5 observations; the item transfer matrix is ​​constructed based on the interaction history of all users. Each row in the item transfer matrix records which item the user will interact with next after interacting with the corresponding source item, and is based on the statistical data of all users.

3. The interpretable and efficient sequence recommendation method incorporating auxiliary information according to claim 1, characterized in that, In step (2), the multi-information autoencoder uses a stacked autoencoder with multiple autoencoders stacked together.

4. The interpretable and efficient sequence recommendation method incorporating auxiliary information according to claim 1, characterized in that, In step (2), the Frobenius norm is used as the reconstruction loss and the cross-entropy is used as the multi-label classification loss.

5. The interpretable and efficient sequence recommendation method incorporating auxiliary information according to claim 1, characterized in that, When steps (5) and (6) are executed multiple times, continuous prediction of the item the user will interact with next is achieved.

Images