Multi-behavior user intention unwrapping representation learning method based on information bottleneck

Abstract

The invention discloses a multi-behavior user intention unwrapping representation learning method based on information bottleneck, which effectively separates pseudo-correlation intentions in multi-behavior interaction by using an information bottleneck principle so as to improve the performance of a recommendation system. Specifically, the method comprises the following steps: firstly, introducing a time-sensitive pseudo-correlation coefficient, dynamically measuring a pseudo-correlation intention proportion of a user in an auxiliary behavior, and using the coefficient to guide a subsequent multi-intention learning task; secondly, pre-training the multi-behavior interaction data of the user and the article by adopting a graph convolutional neural network so as to learn embedded representation under high-order connectivity; then, embedding and mapping the auxiliary behaviors into a target behavior space by utilizing an orthogonal projection technology, so as to separate a real related intention and a pseudo related intention; and finally, after real related intention embedding and pseudo related intention embedding of the user are obtained, multi-intention learning is carried out based on an information bottleneck principle, under the guidance of a pseudo related coefficient, a pseudo related intention in an entanglement auxiliary behavior is deentangled in a targeted manner, and meanwhile, the real related intention is reserved and transmitted to a target behavior as far as possible, so that the user experience is improved. Therefore, the modeling capability and recommendation effect of the recommendation system on the target behavior of the user are improved.

Description

2. Technical Field

[0001] The present invention relates to technical fields such as information bottleneck, multi-behavior recommendation system, graph neural network, disentangled representation learning, etc. Specifically, it designs a multi-behavior user intention disentanglement representation learning method based on information bottleneck. 3. Background Art

[0003] Driven by the rapid development of the Internet and social media, the generation and dissemination speed of information has reached an unprecedented level. Although the Internet has greatly facilitated people's daily lives, it has also brought the problem of information overload, making it difficult for users to quickly identify and obtain truly valuable information when facing a vast amount of content. In addition, merchants also face the challenge of effectively attracting users and precisely meeting their needs in a highly competitive market environment. To solve this problem, recommendation systems have emerged. By analyzing users' historical behaviors, interaction records, and interest preferences, it predicts the content or products that users may be interested in, thus playing an important role in the information screening process. Nowadays, recommendation systems have been widely applied in multiple fields such as e-commerce, online video platforms, and social media, not only helping users efficiently obtain personalized information, alleviating the trouble brought by information overload, but also enhancing the user experience and the market competitiveness of the platform.

[0004] Traditional recommendation methods usually only rely on users' purchase records for interest modeling, while multi-behavior recommendation systems construct a more comprehensive user interest profile by integrating multiple interaction behaviors such as browsing, adding to cart, liking, and collecting. In contrast, recommendation methods based on a single behavior are easily restricted by the problem of data sparsity. Especially in the cold start scenarios of new users or new products, fewer purchase records may lead to limited recommendation effects. The core of multi-behavior recommendation systems lies in fully exploring the user intentions contained in different interaction behaviors. For example, browsing behavior may indicate initial interest, while adding to cart may be a prelude to the purchase decision. By modeling the association between these auxiliary behaviors and the target behavior (such as purchase), the recommendation system can more accurately predict users' needs and improve the recommendation quality. In addition, multi-behavior recommendation can also alleviate the cold start problem of long-tail products, enabling both popular and non-popular content to be reasonably exposed, thereby optimizing the user experience and enhancing the commercial value of the platform. Nowadays, this technology has been widely applied in fields such as e-commerce, short videos, and social media, helping users obtain personalized content more efficiently while improving user retention and revenue of the platform.

[0005] 4. Summary of the Invention (A detailed elaboration of the technical solution of the present invention. If there are drawings, it should be described in combination with flowcharts, principle block diagrams, circuit diagrams, timing diagrams, structure diagrams, etc.)

[0006] I. Technical Problems to be Solved

[0007] In an actual multi-behavior recommendation scenario, the user's auxiliary behaviors (such as browsing and adding to cart) and target behaviors (such as purchasing) may correspond to different intentions. Ideally, the intention of the auxiliary behavior should be consistent with that of the target behavior, that is, the user's interest in the auxiliary behavior can truly reflect their ultimate purchase preference. However, in the complex user interaction process, the auxiliary behavior often contains intentions unrelated to purchasing. For example, the user browses certain products (such as electronic products) due to system recommendations, accidental clicks, or short-term interests, but their actual purchase demand may focus on clothing products. Therefore, the clothing products interacted by the user under the auxiliary behavior reflect the user's actual purchase intention and represent the truly relevant intention. In contrast, the interaction between the user and other types of products under the auxiliary behavior may constitute pseudo-relevant intentions because they do not reflect the actual purchase intention.

[0008] However, when the current mainstream multi-behavior recommendation models model the multi-type interaction relationship between users and items, they usually default that all auxiliary behaviors can provide useful information for modeling the target behavior, and fail to effectively distinguish the differences between the intentions of auxiliary behaviors and target behaviors. This data coupling may lead to pseudo-relevant problems, causing interference to the recommendation system when learning user preferences, thus affecting the recommendation effect. For example, multi-behavior recommendation models based on the attention mechanism or heterogeneous graph neural networks will mix and encode the truly relevant intention features and pseudo-relevant intention features in the embedding space, which may ultimately lead to intention semantic confusion and recommendation bias.

[0009] To address the above problems, the present invention proposes a multi-behavior user intention disentanglement representation learning method based on the information bottleneck. Based on the existing project relationship knowledge graph, a time factor is introduced to measure the association strength between users and items. By projecting the embedding representations of users, items, and interaction relationships onto the time plane corresponding to the user's historical interaction items, the association strength between entity embeddings is dynamically weighted. Finally, it is applied to multi-behavior learning to improve the model's ability to represent user behavior intentions and recommendation accuracy.

[0010] Therefore, in order to effectively distinguish the pseudo-relevant intentions in the auxiliary behavior while retaining and transmitting the truly relevant intentions to the target behavior. Considering that the key idea of the information bottleneck principle is to remove the noise in the input data and retain the information most relevant to the downstream task to obtain a high-quality representation, which is consistent with our main goal. For this reason, the present invention proposes a multi-behavior user intention disentanglement representation learning method based on the information bottleneck. This framework uses a customized information bottleneck principle to handle the pseudo-relevant intentions contained in multi-behaviors.

[0011] II. Technical Solutions

[0012] Step 1: Obtain an existing publicly available real user multi-behavior interaction dataset, which should contain various user interaction behaviors, such as product purchase, adding to cart, favoriting, and browsing, etc. Preprocess the downloaded data, filter out users with low interaction frequency with the system, and re-number the filtered users and items to ensure data consistency. Finally, divide the dataset using a time-based leave-one-out evaluation strategy, taking the most recent interaction record of each user as the test set and the remaining historical interaction data as the training set. The divided training set and test set will be used for subsequent training and validating the model performance.

[0013] Step 2: Based on the user multi-behavior interaction data obtained in Step 1, use a time-sensitive pseudo-correlation coefficient to calculate the overlap rate of interaction items under auxiliary behavior and target behavior according to the user's historical interaction records, so as to dynamically evaluate the proportion of pseudo-correlated intentions of the user in the auxiliary behavior.

[0014] Step 3: Import the preprocessed user multi-behavior interaction dataset and use graph convolutional technology (the widely used benchmark model LightGCN) for pre-training to learn the user's historical interaction patterns under different behaviors. By capturing the high-order connectivity between users and items, generate user and item embedding representations from a multi-behavior perspective.

[0015] Step 4: Use orthogonal projection to project the auxiliary behavior embedding obtained through pre-training into the target behavior embedding, so as to decompose the intention embedding that is truly relevant to the user's target intention and the intention embedding that is irrelevant to the target intention.

[0016] Step 5: Design a multi-intention learning task based on information bottleneck and use the pseudo-correlation coefficient obtained in Step 2 to guide the model to learn user intentions in a personalized manner. Specifically, this task includes two intention learning methods: intention disentanglement representation learning method and target intention enhancement representation learning method. The intention disentanglement representation learning method untangles the falsely correlated intentions by minimizing the mutual information between the pseudo-correlated intention embedding and the auxiliary behavior embedding, so as to retain and enhance the truly relevant intentions. The target intention enhancement representation learning method transfers the truly relevant intentions retained in the auxiliary behavior to the target behavior by maximizing the mutual information between the truly relevant intention embedding and the target behavior embedding, thereby improving the model's ability to learn the user's target intention.

[0017] Step 6: Calculate the loss value of the model according to the multi-intention learning task constructed in Step 5, assign appropriate weights to the losses of each task, and calculate the gradients through backpropagation.

[0018] Step 7: Optimize the model parameters using gradients. By disentangling the pseudo-related intentions in the auxiliary behavior and transmitting the truly relevant intentions to the target behavior, the accuracy of the recommendation is improved. Finally, it is tested on the test set, and based on the evaluation metrics (HR, NDCG), the top-K item recommendations are generated for the user.

[0019] III. Beneficial Effects

[0020] Compared with the existing technologies, the present invention has the following advantages:

[0021] 1. By introducing the orthogonal projection technology, the model can accurately identify and extract the truly relevant intentions and pseudo-related intentions, thereby improving the accuracy and effectiveness of the model in modeling the user behavior intentions.

[0022] 2. According to the user's historical interaction data and its interaction time, dynamically weighing the user's pseudo-correlation coefficient, the model can more accurately learn and model the user's long-term behavior preferences and short-term behavior changes, improving the effect of disentangled representation.

[0023] 3. The present invention introduces a multi-behavior user intention disentangled representation learning method based on information bottleneck, which can effectively identify and model the pseudo-related intentions in the auxiliary behavior, retain and transmit the truly relevant intentions to the target behavior, and avoid the recommendation bias caused by pseudo-related interference.

[0024] 5. Specific implementation manners (proving that the present technical solution can obtain the foregoing beneficial effects through different specific experimental methods and the experimental data obtained thereby, or elaborating the foregoing technical solution can solve the technical problems and obtain the technical effects through principles) Description of the Drawings:

[0025] Figure 1 Overall flowchart of the recommendation method

[0026] Figure 2 Model framework diagram of the present invention

[0027] To make the objectives, technical solutions, and advantages of the present invention clearer, the following further elaborates the present invention in detail with reference to specific examples and the accompanying drawings.

[0028] The present invention conducts item recommendations based on the user's multi-behavior interaction data, and the overall process is as Figure 1 shown. Specifically, it includes the following steps:

[0029] Step 1: Download publicly available recommendation datasets on the Internet, such as Tmall, Beibei, and Retailrocket, which contain various user behavior interactions. The datasets mainly cover the historical records of four behaviors of users: click, add to cart, favorite, and purchase. Secondly, preprocess these data, filter out users with fewer interactions, and re-number the users and items in the data. Furthermore, adopt the time-based leave-one-out evaluation method to divide the data into a training set and a test set. Among them, the test set only contains the last interaction of each user, and the remaining interaction data is used as the training set. Finally, the processed dataset generates five txt files: four files respectively correspond to the historical interaction records of the four behaviors (click.txt, cart.txt, favorite.txt, buy.txt), and a test set file (test.txt). Each line in each behavior file includes the user ID and the item ID interacted by the user under the corresponding behavior, while each line in the test set file contains the user ID and the item ID of the user's last purchase.

[0030] Step 2: Based on the multi-behavior interaction data of users collected in Step 1, use the time-sensitive pseudo-correlation coefficient to calculate the overlap rate of interaction items of users under auxiliary behaviors and target behaviors, so as to indirectly measure the pseudo-correlation intention ratio of each user between different behaviors. For the time-sensitive pseudo-correlation coefficient s u,a The calculation formula is as follows:

[0031]

[0032] where, + u represents the timestamp of the last interaction of user u under the target behavior K, and + u,i represents the interaction timestamp between user u and item i. and respectively represent the sets of items interacted by user u under the auxiliary behavior a and the target behavior K. By introducing the time factor T(·), the model can capture the behavior intentions of users at different time periods.

[0033] Step 3: In order to learn user and item representations from the multi-behavior interaction graph, we introduce the widely used graph encoder LightGCN as the model backbone. Taking user u as an example, under a specific behavior k, the message passing process of LightGCN can be defined as:

[0034]

[0035] where, is the embedding vector of user u's behavior k at the l-th layer. and They respectively represent the first-hop neighbors of user u and item i under behavior k. After L layers of message passing, the final representation of user u under behavior k is obtained by averaging the embedding vectors of all layers:

[0036]

[0037] Using the same method as the user behavior embedding, we can obtain the behavior embedding of item i

[0038] Step 4. Based on the multi-behavior embedding representation of the user obtained in Step 3, in order to distinguish the true and false relevant intentions in the auxiliary behavior a, we decompose the auxiliary behavior embedding of user u into two parts:

[0039]

[0040] where, and respectively represent the embedding of the true relevant intention and the embedding of the false relevant intention in the auxiliary behavior a. Their formal definitions are as follows. By projecting onto the hyperplane containing the embedding, we obtain the component in the same direction as The specific calculation formula of is:

[0041]

[0042] Another auxiliary behavior component is obtained by projecting onto the hyperplane with as the normal vector. The calculation formula is as follows:

[0043]

[0044] Based on this decomposition, the user auxiliary behavior embedding matrix E a can be naturally decomposed into the true relevant intention embedding matrix and the pseudo-relevant intention embedding matrix The specific definitions are as follows:

[0045]

[0046] Step 5: The goal of multi-intent learning is to disentangle the pseudo-related intents in the auxiliary behavior and further transfer the truly related intents from the auxiliary behavior to the target behavior, so as to enhance the model's representation learning of the user's target intent. We introduce the information bottleneck principle, aiming to remove irrelevant information from the input data and retain the information most relevant to the downstream task. Therefore, given the user embedding matrix E of the target behavior K and the auxiliary behavior embedding matrix E a , as well as the truly related intent embedding matrix a decomposed from E and the pseudo-related intent embedding matrix Based on the information bottleneck principle, the goal of multi-intent learning is designed as follows:

[0047]

[0048] where I(·) represents the mutual information between two random variables, and β is the bottleneck coefficient that balances information compression and target relevance. Due to the dynamic and multi-faceted nature of user intents, the intensity of pseudo-related intents in each auxiliary behavior of each user usually varies, making a fixed β ineffective and rigid for the task of disentangling pseudo-related intents. To overcome this limitation, we replace the fixed β with the pseudo-related coefficient s u,a calculated through Step 2. This coefficient is uniquely determined by each user u and each of their auxiliary behaviors a, quantifying the intensity of pseudo-related intents for each user-behavior pair, enabling the model to customize the disentanglement strategy individually and achieve reliable results in different behavioral contexts. The optimization goal of multi-intent learning based on the information bottleneck is defined as follows:

[0049]

[0050] According to the above multi-intent learning goal, information bottleneck multi-intent learning includes two methods: intent disentanglement representation learning method and target intent enhancement learning representation method. The specific technical details are as follows:

[0051] First, since the mutual information of high-dimensional and discrete data cannot be directly calculated, we introduce the Hilbert-Schmidt independence criterion to approximate the mutual information by measuring statistical dependence aiming to disentangle pseudo-related intents. Therefore, the mutual information estimation under the auxiliary behavior a can be expressed as:

[0052]

[0053] where, K a and are the kernel matrices of E a and respectively, is a centering matrix, where \(I\) is the identity matrix, \(1\) represents a column vector of all ones, and \(Tr(·)\) represents the trace operation of a matrix. To capture the statistical dependencies between variables, we adopt the widely used Gaussian kernel function to calculate the kernel function \(K\). a and Therefore, the objective function of the disentangled representation method of intentions under the auxiliary behavior \(a\) can be expressed as:

[0054]

[0055] After disentangling the pseudo - correlated intentions existing in the auxiliary behavior, the truly relevant intentions are fully retained and enhanced. We further design a method for enhancing target intentions, which transfers the truly relevant intentions from the auxiliary behavior to the target behavior.

[0056] Specifically, we first use InfoNCE to estimate and optimize the mutual information Therefore, to transform the true associated intentions into the target behavior, we regard the representations from the same user as positive samples to promote consistency, and the representations between different users as negative samples to strengthen the divergence. The optimization function of the target intention enhancement method under the auxiliary behavior \(a\) can be defined as:

[0057]

[0058] where \((·)\) is the cosine similarity, and \(\tau\) is the temperature hyperparameter in softmax.

[0059] Step 6: To disentangle the pseudo - correlated intentions in each auxiliary behavior and transfer the semantic information consistent with the target intention to the user's preference representation, we apply the information bottleneck independent constraint to each auxiliary behavior sub - graph, thus obtaining the final multi - intention learning optimization objective as follows: Figure 1

[0060]

[0061] Step 7: In each round of training, optimize the model gradients and parameters according to the multi - intention learning objective obtained in Step 6, so as to disentangle the pseudo - correlated intentions in the auxiliary behavior and transfer the truly relevant intentions to the target behavior to improve the recommendation accuracy. And select the optimal model parameters according to the evaluation metrics HR (Hit Ratio), NDCG (Normalized Discounted Cumulative Gain) to generate top - K item recommendations for users.

[0062] It should be noted that although the embodiments described above of the present invention are illustrative, they are not limitations on the present invention. Therefore, the present invention is not limited to the above specific embodiments. Without departing from the principle of the present invention, any other embodiments obtained by those skilled in the art under the inspiration of the present invention are considered to be within the protection scope of the present invention.

Claims

1. A multi-behavior user intention disentanglement representation learning method based on information bottleneck, characterized by including The following steps: Step 1: Obtain a user multi-behavior interaction dataset, which should contain various user interaction behaviors, such as product purchase, adding to cart, collection, and browsing, etc. Preprocess the downloaded data, screen out users with low interaction frequencies with the system, and re-number the screened users and items to ensure data consistency. Finally, divide the dataset using a time-based leave-one-out evaluation strategy, taking the most recent interaction record of each user as the test set and the remaining historical interaction data as the training set. The divided training set and test set will be used for subsequent training and validating the model performance. Step 2: Based on the user multi-behavior interaction data obtained in Step 1, adopt a time-sensitive pseudo-correlation coefficient to calculate the overlap rate of interaction items under auxiliary behavior and target behavior according to the user's historical interaction records, so as to dynamically evaluate the proportion of pseudo-correlated intentions of the user in the auxiliary behavior. Step 3: Import the preprocessed user multi-behavior interaction dataset and use a graph convolutional neural network for pre-training to learn the user's historical interaction patterns under different behaviors. Step 4: Use orthogonal projection to project the auxiliary behavior embedding obtained through pre-training into the target behavior embedding, so as to decompose the intention embedding that is truly relevant to the user's target intention and the intention embedding that is irrelevant to the target intention. Step 5: Adopt a multi-intention learning task based on information bottleneck and use the pseudo-correlation coefficient obtained in Step 2 to guide the model to learn the user's intention in a personalized manner. Specifically, this task includes two intention learning methods: intention disentanglement representation learning method and target intention enhancement representation learning method. The intention disentanglement representation learning method unties the falsely correlated intentions by minimizing the mutual information between the pseudo-correlated intention embedding and the auxiliary behavior embedding, so as to retain and enhance the truly relevant intentions. The target intention enhancement representation learning method transfers the truly relevant intentions retained in the auxiliary behavior to the target behavior by maximizing the mutual information between the truly relevant intention embedding and the target behavior embedding, thereby improving the model's ability to learn the user's target intention. Step 6: Calculate the loss value of the model according to the multi-intention learning task constructed in Step 5, allocate appropriate weights to the losses of each task, and calculate the gradient through backpropagation. Step 7: Use the gradient to optimize the model parameters, improve the recommendation accuracy by disentangling the pseudo-correlated intentions in the auxiliary behavior and transferring the truly relevant intentions to the target behavior. Finally, generate Top-K item recommendations for users based on the scores of evaluation metrics (HR, NDCG).

2. The multi-behavior user intention disentanglement representation learning method based on information bottleneck according to claim 1, characterized in that, in In step 2, the time-sensitive pseudo-correlation coefficient s u,a The calculation formula is as follows: Among them, + u represents the timestamp of the most recent interaction of user u under the target behavior K, while + u,i represents the timestamp of the interaction between user u and item i. and respectively represent the set of items interacted by user u under the auxiliary behavior a and the target behavior K.

3. The multi-behavior user intention disentanglement representation learning method based on information bottleneck according to claim 1, characterized in that, In Step 3, under a specific user u and their behavior k, the message passing process is defined as follows: Among them, is the embedding vector of the behavior k of user u at the l-th layer. and respectively represent the first-hop neighbors of user u and item i under behavior k.

4. A method for learning the disentangled representation of multi-behavior user intentions based on the information bottleneck, characterized in that, In Step 3, after L layers of message passing, the final representation of user u under behavior k is as follows:

5. The multi-behavior user intention disentanglement representation learning method based on information bottleneck according to claim 1, characterized in that In step 4, the specific calculation formula for the true relevant intention embedding is as follows:

6. A method for learning the disentangled representation of multi-behavior user intentions based on the information bottleneck, characterized in that In step 4, the pseudo-related intent embedding The specific calculation formula is as follows:

7. A multi-behavior user intention disentanglement representation learning method based on information bottleneck according to claim 1, characterized in that In Step 5, the optimization objective of the multi-intention learning based on information bottleneck is defined as follows: where I(·) represents the mutual information of two random variables, and β is the bottleneck coefficient that balances information compression and target relevance. s u,a is the pseudo-correlation coefficient calculated in Step 2 and is uniquely determined by each user u and each of their auxiliary behaviors a.

8. A method for learning the disentangled representation of multi-behavior user intentions based on information bottleneck according to claim 1, characterized in that In Step 5, the mutual information estimation value calculated by the Hilbert-Schmidt (HSIC) independence criterion is expressed as: Among them, K a and are the kernel matrices of E a and respectively, is a centering matrix, where I is the identity matrix, 1 represents a column vector of all 1s, and Tr(·) represents the trace operation of a matrix.

9. A method for multi-behavior user intention disentanglement representation learning based on information bottleneck according to claim 1, characterized in that In Step 5, the objective function of the intention disentanglement representation method is defined as:

10. A method for multi-behavior user intention disentanglement representation learning based on information bottleneck according to claim 1, characterized in that In Step 5, the optimization function of the target intention enhancement method is expressed as: where (·) is the cosine similarity and τ is the temperature hyperparameter in softmax.

11. A method for multi-behavior user intention disentanglement representation learning based on information bottleneck according to claim 1, characterized in that In step 6, the final multi-intent learning optimization objective is as follows:

Images