Recommendation method based on multi-behavior knowledge distillation
Through the recommendation method based on multi-behavior knowledge distillation, users' purchasing behavior and comment behavior are separately modeled, and interactive data is processed using graph convolution network and BERT model. Combined with the counterfactual distillation module, the problem of difficult correlation between different behaviors in the existing technology is solved, achieving more accurate user preference portrayal and recommendation accuracy improvement.
Patent Information
- Application Number
- CN202510456753.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-12
- Publication Date
- 2025-07-18
AI Technical Summary
Existing multi-behavior recommendation methods are difficult to effectively capture the potential relationship between different behaviors, resulting in models tending toward high-frequency ratings and making it difficult to accurately characterize users' true preferences.
The recommendation method based on multi-behavior knowledge distillation is adopted, and the user's purchasing behavior and comment behavior are separately modeled, and interactive data is processed using graph convolution network and BERT model, and knowledge distillation learning is carried out in combination with the counterfactual distillation module to construct the total loss function to optimize the model.
Improve the accuracy and accuracy of recommendations, learn user and item representation through multi-behavior modeling and knowledge distillation, alleviate the problem of scoring bias and improve the accuracy of recommendations.
Smart Images
Figure CN120338927A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to a recommendation method based on multi-behavior knowledge distillation, belonging to the field of recommendation systems. Background Art
[0002] Multi-behavior recommendation is a challenging task. It not only goes beyond the paradigm of collaborative filtering that only relies on interactions to capture collaborative signals, but also deeply explores the multi-modal user preferences contained in complex behaviors. However, although existing multi-behavior recommendation methods have made significant progress, they still face challenges in effectively modeling the correlation between different behaviors. Specifically, traditional methods usually model different behaviors independently and are difficult to fully capture the potential relationships between them. In addition, the historical rating distributions of different behaviors are uneven, resulting in the model tending to high-frequency ratings and thus making it difficult to accurately depict the true preferences of users. Summary of the Invention
[0003] In order to overcome the deficiencies of the prior art, the present invention proposes a recommendation method based on multi-behavior knowledge distillation, aiming to comprehensively utilize comment behaviors and purchase behaviors to learn the characteristics of users and items when dealing with the recommendation task of multi-behavior interaction data, so as to quickly and accurately learn the feature representations of users and items, thereby improving the accuracy and precision of recommendations.
[0004] To achieve the above object, the present invention adopts the following technical solutions:
[0005] A recommendation method based on multi-behavior knowledge distillation according to the present invention is characterized by including the following steps:
[0006] Step 1, construct a user set U and an item set V; for any user in U and any item in V, the rating is denoted as , and ∈ , R represents the rating set. For any user in U and any item in V, the interaction is denoted as , D represents the interaction set; for any user in U and any item in V, the comment is denoted as ;
[0007] Map the ID features of user i and item j to the dense vectors of user and the dense vector of item respectively;
[0008] For user Score for project j Map to a dense vector ;
[0009] Input the comment of user on project j into the pre-trained BERT-Whiting model for processing to generate a comment embedding , thereby calculating the average value of the comment embeddings of user on all projects and taking it as the comment feature of user i , and calculating the average value of the comment embeddings of all users on project j and taking it as the comment feature of project j ;
[0010] Step 2: Construct a purchase behavior and comment behavior prediction module, and based on the dense vector and as well as the comment feature and , correspondingly obtain the predicted score of user i on project j in the purchase behavior and the predicted score of user i on project j in the comment behavior :
[0011] Step 3: Construct a counterfactual calculation module, and based on the dense vector and as well as the comment feature and , correspondingly obtain the predicted counterfactual score of user i on project j in the purchase behavior and the predicted counterfactual score of user i on project j in the comment behavior ;
[0012] Step 4: Use Equation (13) to construct the total loss function :
[0013] (13)
[0014] In Equation (13), represents the scoring prediction loss constructed based on and as well as , represents the distillation loss constructed based on and as well as and ;
[0015] Step 5: Use the gradient descent method to train the multi-behavior knowledge distillation model composed of the purchase behavior and comment behavior prediction module and the counterfactual calculation module, and calculate the total loss function When the number of training iterations reaches the set number or the total loss function error is less than the set threshold, the training stops, thereby obtaining an optimal multi-behavior knowledge distillation model for processing user ratings and reviews, as well as user and item features, and outputting the rating of each user for each item.
[0016] The feature of the recommendation method based on multi-behavior knowledge distillation according to the present invention also lies in that the step 2 includes the following steps:
[0017] Step 2.1. In the purchase behavior modeling, the dense vectors of user i and item j and are input into the graph convolutional network, so as to obtain the ID feature representation of user i at the l-th layer and the ID feature representation of item j at the l-th layer by using equations (1) and (2):
[0018] (1)
[0019] (2)
[0020] In equations (1) and (2), represents the ID feature representation of item j at the -th layer; represents the ID feature representation of user i at the -th layer; when , let = , = ; and respectively represent the neighbor sets of user i and item j; and respectively represent the number of elements in the neighbor sets of user i and item j;
[0021] The ID feature representation of user i at the last L-th layer and the ID feature representation of item j at the last L-th layer are input into the multi-layer perceptron MLP, so as to obtain the predicted rating of user i for item j in the purchase behavior by using equation (3):
[0022] (3)
[0023] In equation (3), represents the element-wise multiplication of vectors;
[0024] Step 2.2. In the review behavior modeling, and Input into the graph convolutional network, so as to obtain the review feature representations of user \(i\) and item \(j\) at the \(l\)-th layer by using Equations (4) and (5). and :
[0025] (5)
[0026] (6)
[0027] In Equations (4) and (5), represents the review feature of item \(j\) at the -th layer; represents the review feature of user \(i\) at the -th layer; when , let = , = ;
[0028] Input the review feature representation of user \(i\) at the last \(L\)-th layer and the review feature representation of item \(j\) at the last \(L\)-th layer into the MLP, so as to obtain the predicted score of user \(i\) for item \(j\) under the review behavior by Equation (6) :
[0029] (6).
[0030] Furthermore, step 2 includes the following steps:
[0031] Step 3.1: In the graph convolutional network at the \((l - 1)\)-th layer, by respectively fixing the ID feature and the review feature of user \(i\), the ID feature of user \(i\) in the counterfactual world at the \((l - 1)\)-th layer and the review feature of item \(j\) at the \((l - 1)\)-th layer are correspondingly obtained, so as to input and into the graph convolutional network, and use Equations (7) and (8) to obtain the ID feature of user \(i\) in the counterfactual world at the \(l\)-th layer and the review feature of item \(j\) at the \(l\)-th layer:
[0032] (7)
[0033] (8)
[0034] Step 3.2: The ID feature of user \(i\) at the \(L\)-th layer in the purchase behaviorand the ID feature of item j in the L-th layer in the counterfactual world is input into the MLP, so as to obtain the counterfactual score of user i's prediction of item j in the purchase behavior by using Equation (9) :
[0035] (9)
[0036] The review feature of user i in the L-th layer in the review behavior and the review feature of item j in the L-th layer in the counterfactual world are input into the MLP, so as to obtain the counterfactual score of user i's prediction of item j in the review behavior by using Equation (10) :
[0037] (10).
[0038] Furthermore, in step 3, the distillation loss is constructed by using Equation (11) :
[0039] (11)
[0040] In Equation (11), represents the softmax function; KL represents the Kullback-Leibler divergence; represents the number of elements in;
[0041] The score prediction loss is constructed by using Equation (12) :
[0042] (12).
[0043] An electronic device according to the present invention includes a memory and a processor, characterized in that the memory is used to store a program for supporting the processor to execute the recommendation method, and the processor is configured to execute the program stored in the memory.
[0044] A computer-readable storage medium according to the present invention, characterized in that a computer program is stored on the computer-readable storage medium, and when the computer program is run by a processor, the steps of the recommendation method are executed.
[0045] Compared with the prior art, the beneficial effects of the present invention are as follows:
[0046] 1. The present invention separately models the purchase behavior and review behavior of users to mine user preferences from different behaviors, thereby improving the accuracy of recommendation prediction.
[0047] 2. The present invention uses knowledge distillation to promote mutual learning among multiple behaviors, thereby improving the accuracy of individual behaviors while enhancing the accuracy of other behaviors.
[0048] 3. While performing knowledge distillation, the present invention uses counterfactual theory to remove counterfactual outputs from different behaviors, enabling the knowledge distillation process to unbiasedly share knowledge among multiple behaviors, capture the true preferences of users, and improve the accuracy of recommendations. BRIEF DESCRIPTION OF THE DRAWINGS
[0049] Figure 1 It is a schematic diagram of the recommendation method based on multi-behavior knowledge distillation according to the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS
[0050] In this embodiment, a recommendation method based on multi-behavior knowledge distillation uses a graph neural network model and combines the technology of counterfactual distillation to unbiasedly extract and share knowledge using multi-behavior interaction data. First, multiple parallel networks are used to model interaction behavior, review behavior, mixed behavior, and rating information. Then, a counterfactual distillation module is used to promote the unbiased extraction and transfer of knowledge between different networks during the embedding learning stage. Finally, the training model is optimized using the mean squared error loss. Specifically, as Figure 1 shown, it is carried out according to the following steps:
[0051] Step 1. Construct a user set U and an item set V; let any user in U rate any item in V as ∈ , where R represents the rating set. Let any user in U interact with any item in V as ; let any user in U review any item
[0052] in V as
[0053] The features of users and items consist of ID features, rating features, and review features. The ID features of user i and item j are respectively mapped to the dense vector of user and the dense vector of item
[0054] The rating of user for item j is mapped to the dense vector ;
[0055] Input the user's comment on project j into the pre-trained BERT-Whiting model for processing to generate comment embeddings , thereby calculating the average of the comment embeddings of the user's comments on all projects and using it as the comment feature of user i , and calculating the average of the comment embeddings of all users' comments on project j and using it as the comment feature of project j ;
[0056] Step 2: Build a purchase behavior and comment behavior prediction module, and based on the dense vectors and as well as the comment features and , respectively obtain the predicted score of user i on project j in the purchase behavior and the predicted score of user i on project j in the comment behavior :
[0057] Step 2.1: To accurately capture the user interests and project features in multiple behavior patterns, three parallel views are constructed to separately model the purchase and comment behaviors. In addition, the score directly reflects the overall satisfaction of the user with the project. Therefore, by treating the score as the edge of each view, the score is used to evaluate the overall preference of the user for the project. In the purchase behavior modeling, the dense vectors of user i and project j and are input into the graph convolutional network, and thereby the ID feature representation of user i at the l-th layer and the ID feature representation of project j at the l-th layer are obtained using equations (1) and (2):
[0058] (1)
[0059] (2)
[0060] In equations (1) and (2), represents the ID feature representation of project j at the -th layer; represents the ID feature representation of user i at the -th layer; when , let = , = ; and respectively represent the neighbor sets of user i and project j; and respectively represent the number of elements in the neighbor sets of user i and item j;
[0061] To simulate the rating behavior between users and items, the present invention uses the Hadamard product to calculate their matching features. The ID feature representation of user i at the last L-th layer and the ID feature representation of item j at the last L-th layer are input into the multi-layer perceptron MLP, so as to obtain the predicted rating of user i for item j in the purchase behavior by using Equation (3) :
[0062] (3)
[0063] In Equation (3), represents the element-wise multiplication of vectors, that is, the Hadamard product;
[0064] Step 2.2, in the comment behavior modeling, and are input into the graph convolutional network, so as to obtain the comment feature representations of user i and item j at the l-th layer by using Equation (4) and Equation (5) and :
[0065] (4)
[0066] (5)
[0067] In Equation (4) and Equation (5), represents the comment feature of item j at the -th layer; represents the comment feature of user i at the -th layer; when , let = , = ;
[0068] The comment feature representation of user i at the last L-th layer and the comment feature representation of item j at the last L-th layer are input into the MLP, so as to obtain the predicted rating of user i for item j in the comment behavior by Equation (6) :
[0069] (6)
[0070] Step 3.1. The present invention calculates the direct impact of ratings on the final prediction result by constructing a counterfactual world. That is, in the (l - 1)-th layer graph convolutional network, by fixing the ID feature of user i and the review feature respectively, the ID feature of user i in the (l - 1)-th layer in the counterfactual world and the review feature of item j in the (l - 1)-th layer are obtained correspondingly. Then, and are input into the graph convolutional network, and the ID feature of user i in the l-th layer and the review feature of item j in the l-th layer in the counterfactual world are obtained by using Equations (7) and (8):
[0071] (7)
[0072] (8)
[0073] Step 3.2. The ID feature of user i in the L-th layer in the purchase behavior and the ID feature of item j in the L-th layer in the counterfactual world are input into the MLP, and the counterfactual rating of user i's prediction of item j in the purchase behavior is obtained by using Equation (9):
[0074] (9)
[0075] The review feature of user i in the L-th layer in the review behavior and the review feature of item j in the L-th layer in the counterfactual world are input into the MLP, and the counterfactual rating of user i's prediction of item j in the review behavior is obtained by using Equation (10):
[0076] (10)
[0077] Step 3. To evaluate the impact of ratings on predictions, we construct a counterfactual world, that is, by fixing the features irrelevant to ratings to calculate the prediction result in the counterfactual world. In the counterfactual calculation module, based on the dense vectors and as well as the review features and , the counterfactual rating of user i's prediction of item j in the purchase behavior and the counterfactual rating of user i's prediction of item j in the review behavior are obtained correspondingly;
[0078] Step 4. For traditional online distillation tasks, the optimization objective of the distillation module is to minimize the rating prediction loss and the distillation loss. To address the bias contamination problem during distillation, the present invention eliminates the bias by subtracting the counterfactual prediction from the traditional prediction and subtracting the counterfactual prediction from the traditional prediction . and subtracting the counterfactual prediction from the traditional prediction to eliminate the bias.
[0079] Construct the distillation loss using Equation (11) ;
[0080] (11)
[0081] In Equation (11), represents the softmax function; KL represents the Kullback-Leibler divergence; represents the number of elements in
[0082] Construct the rating prediction loss using Equation (12) ;
[0083] (12)
[0084] Construct the total loss function using Equation (13) :
[0085] (13)
[0086] In Equation (13), represents the rating prediction loss constructed based on and and , represents the distillation loss constructed based on and and and ;
[0087] Step 5. Use the gradient descent method to train the multi-behavior knowledge distillation model composed of the purchase behavior and review behavior prediction module and the counterfactual calculation module, and calculate the total loss function . When the number of training iterations reaches the set number or the error of the total loss function is less than the set threshold, the training stops, thereby obtaining the optimal multi-behavior knowledge distillation model for processing the ratings and reviews of users, as well as the features of users and items, and outputting the ratings of each user for each item.
[0088] In this embodiment, an electronic device includes a memory and a processor. The memory is used to store a program that supports the processor to execute the above method, and the processor is configured to execute the program stored in the memory.
[0089] In this embodiment, a computer-readable storage medium stores a computer program, and when the computer program is run by a processor, it executes the steps of the above method.
Claims
1. A recommendation method based on multi-behavior knowledge distillation, characterized in that Including the following steps: Step 1, constructing a user set U and an item set V; Let any user in U For any item in V The score is denoted as and ∈ , R represents the set of scores. Let any user in U For any item in V The interaction is denoted as , D represents the set of interactions; Let any user in U For any item in V The comment is denoted as ; Map the ID features of user i and item j to the dense vectors of user and the dense vectors of item respectively; ; ; Map the user 's score for project j to a dense vector ; Input the user Comments on project j into the pre-trained BERT-Whiting model for processing to generate comment embeddings , thereby calculating the user average of the comment embeddings for all projects and using it as the comment feature of user i , and calculating the average of the comment embeddings of all users for project j and using it as the comment feature of project j ; Step 2: Construct a purchase behavior and review behavior prediction module, and based on the dense vector and as well as the review features and , correspondingly obtain the predicted score of user i for item j in the purchase behavior and the predicted score Step 3: Construct a counterfactual calculation module and, based on the dense vectors and as well as the comment features and , respectively obtain the counterfactual score of the prediction of user i for item j in the purchase behavior and the counterfactual score of the prediction of user i for item j in the comment behavior ; Step 4. Construct the total loss function using Equation (13). : (13) In formula (13), represents the scoring prediction loss constructed based on and as well as ; represents the distillation loss constructed based on and as well as and Step 5: Use the gradient descent method to train the multi-behavior knowledge distillation model composed of the purchase behavior and review behavior prediction module and the counterfactual calculation module, and calculate the total loss function , when the number of training iterations reaches the set number or the error of the total loss function is less than the set threshold, the training stops, so as to obtain the optimal multi-behavior knowledge distillation model, which is used to process the ratings and reviews of users, as well as the features of users and items, and output the ratings of each user for each item.
2. The recommendation method based on multi-behavior knowledge distillation according to claim 1, wherein The said Step 2 includes the following steps: Step 2.
1. In the purchase behavior modeling, input the dense vectors of user i and item j and into the graph convolutional network, so as to obtain the ID feature representation of user i at the l-th layer and the ID feature representation of item j at the l-th layer by using equations (1) and (2): and : (1) (2) In formulas (1) and (2), represents the ID feature representation of item j at the th layer; represents the ID feature representation of user i at the th layer; when , let = , = ; and respectively represent the neighbor sets of user i and item j; and respectively represent the number of elements in the neighbor sets of user i and item j; The ID feature representation of user i at the last L-th layer and the ID feature representation of item j at the last L-th layer are input into the multi-layer perceptron MLP, so as to obtain the predicted score of user i for item j in the purchase behavior by using Equation (3) : (3) In Equation (3), represents the element-wise multiplication of vectors; Step 2.
2. In the comment behavior modeling, input and into the graph convolutional network, so as to obtain the comment feature representations of user i and item j at the l-th layer by using equations (4) and (5). and : (5) (6) In Formula (4) and Formula (5), represents the comment feature of item j at the layer; represents the comment feature of user i at the layer; when , let = , = ; The comment feature representation of user i at the last L-th layer and the comment feature representation of item j at the last L-th layer are input into the MLP, so that the predicted score of user i for item j under the comment behavior is obtained in Equation (6) : (6)。 3. The recommendation method based on multi-behavior knowledge distillation according to claim 2, wherein The said Step 2 includes the following steps: Step 3.
1. In the l-1 layer graph convolutional network, by fixing the ID feature of user i and the comment feature of user i respectively, the ID feature of user i at the l-1 layer in the counterfactual world and the comment feature of item j at the l-1 layer are obtained accordingly, so as to input and into the graph convolutional network, and use Equations (7) and (8) to obtain the ID feature of user i at the l layer and the comment feature of item j at the l layer in the counterfactual world: and the comment feature , the ID feature of user i at the l-1 layer in the counterfactual world and the comment feature of item j at the l-1 layer are obtained accordingly, so as to input and the comment feature of item j at the l-1 layer , so as to and are input into the graph convolutional network, and the ID feature of user i at the l layer and the comment feature of item j at the l layer in the counterfactual world are obtained by using Equations (7) and (8): and the comment feature of item j at the l layer : (7) (8) Step 3.2: Input the ID feature of user i at the L-th layer in the purchase behavior and the ID feature of item j at the L-th layer in the counterfactual world into the MLP, so as to obtain the predicted counterfactual score of user i for item j in the purchase behavior by using Equation (9) : (9) The comment feature of user i at the L-th layer in the comment behavior and the comment feature of item j at the L-th layer in the counterfactual world are input into the MLP, so as to obtain the counterfactual score predicted by user i for item j in the comment behavior using Equation (10) : (10)。 4. The recommendation method based on multi-behavior knowledge distillation according to claim 3, wherein In step 3, the distillation loss is constructed using Equation (11). : (11) In formula (11), represents the softmax function; KL represents the Kullback-Leibler divergence; represents the number of elements in; Construct the scoring prediction loss using Equation (12) : (12)。 5. An electronic device, comprising a memory and a processor, characterized in that The memory is used to store a program that supports the processor to execute any one of the recommendation methods described in claims 1-4, and the processor is configured to execute the program stored in the memory.
6. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is run by the processor, it executes the steps of any one of the recommendation methods described in claims 1-4.