A debiasing method and recommendation system based on personalized causal decomposition
By employing a personalized causal decomposition method to remove bias, and utilizing neural networks to extract user interest and conformity representations, combined with contrastive learning and causal inference to optimize the recommendation model, the problem of popularity bias in recommendation systems is solved, resulting in fairer and more diverse recommendation outcomes and improved user satisfaction.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SUZHOU YUNTU HEALTH TECH CO LTD
- Filing Date
- 2023-10-16
- Publication Date
- 2026-07-21
AI Technical Summary
Existing recommendation models fail to effectively remove popularity bias, resulting in a lack of diversity in recommendation results, which affects user satisfaction, and they also fail to fully consider the two-way impact of user-item interaction.
By grouping users into interest attributes and conformity attributes, and classifying items into popular and unpopular items, a bias-reduction method based on personalized causal decomposition is adopted. Neural networks are used to extract user interest and conformity representations, and the recommendation results are optimized by combining contrastive learning, causal inference, and domain-adaptive enhancement models with the BCE loss function.
It effectively eliminates popularity bias in recommendation systems, improves personalized and diverse recommendation results, and enhances user satisfaction.
Smart Images
Figure CN117312675B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of recommender systems, specifically relating to a bias removal method and recommender system based on personalized causal decomposition. Background Technology
[0002] In modern Internet applications, recommendation systems are widely used in e-commerce, social networks, search engines, and entertainment to provide users with personalized and diverse recommendations. Most existing recommendation models are based on users' historical interaction records. However, due to users' herd mentality, most of their interaction records are popular items. Popularity bias in recommendation systems affects the diversity of recommended items, resulting in recommendations that are mainly limited to popular products and cannot reflect users' personal interests. These problems are further aggravated by the Matthew effect, ultimately leading to a lack of diversity in recommendation results and affecting user satisfaction. To solve these problems, a recommendation model that can remove recommendation bias is needed. Currently, some technical models have attempted to solve this problem, but some existing bias removal models have not considered the following two issues: (1) Popularity bias is brought about by the interaction between users and items. Many models only solve this problem from the perspective of users or items, without taking both into account. (2) The granularity of existing bias removal models is not fine enough. They do not fully consider that different users will have different popularity biases for items with different popularity. Summary of the Invention
[0003] The purpose of this invention is to provide a more equitable debiasing method and recommendation system based on personalized causal decomposition that can remove popularity bias.
[0004] The technical solution of this invention is: a bias removal method based on personalized causal decomposition, which first groups users into interest attributes. and conformity attribute Group items into popular items and unpopular items Then, by cross-referencing user groups and item groups, we obtain users' interest in and conformity levels for items with different popularity levels; finally, we use a bias-reducing model to learn users' interest in items with different popularity levels. , and conformity , ; Use last , The interest-based prediction recommendation results remove popularity bias (where pop represents popularity, unpop represents unpopularity, int represents interest, conf represents conformity, U represents user, I represents item, and D represents degree). This indicates the user's level of interest in popular items. This indicates the user's level of interest in unpopular items. It indicates the degree to which users conform to popular items. This indicates the degree to which users conform to unpopular items.
[0005] The specific algorithm for the bias correction model is as follows:
[0006] Step 1: Classify the dataset and determine the popularity of an item based on the number of interactions. If the number of interactions is higher than the average, the item is considered a popular item; otherwise, it is considered an unpopular item.
[0007] Step 2: Transform the popular and unpopular items categorized in Step 1 into representation vectors suitable for matrix operations using an encoding embedding layer. It is also transformed into a representation vector that can be used for matrix operations through the encoding embedding layer;
[0008] Step 3: Define two neural networks, an interest network (Int) and a conformity network (Conf), to extract user interests and conformity respectively, denoted as . and By combining the item and user representations obtained in step 2 and then passing them through the two neural networks, the user's interest representation and conformity representation for items of different popularity can be obtained, denoted as ( ), ( )and( ), ( );in This represents a neural network for extracting user interest. This represents a neural network that extracts user conformity; (·; θ) represents the input of this neural network, where · represents the data input when the neural network needs to extract interest or conformity, θ represents the internal parameters of the neural network, i represents interest, and c represents conformity. This indicates a user's interest in popular items. This indicates a user's interest in unpopular items. User conformity is a characteristic of popular items. User conformity regarding unpopular items;
[0009] Step 4: Utilize the contrastive learning task to enhance user interest and decompose conformity to obtain the first loss function L. similar ,
[0010] ;
[0011] γ is a hyperparameter, e is the representation, pop represents popularity, unpop represents unpopularity, int represents interest, conf represents conformity, and * represents an element in the set. In this step... Therefore, * indicates popular or unpopular items in the set;
[0012] Step 5: Utilize causal reasoning constraints to enhance the decomposition of user interest and conformity, thereby obtaining the second loss function L. causal ,
[0013] ;
[0014] Where 'e' represents the status quo, 'pop' represents popularity, 'unpop' represents unpopularity, and 'int' represents interest.
[0015] Step 6: Further separate the two neural networks described in Step 3 using domain adaptation to obtain the third loss function L. domain ,
[0016] ,
[0017] BCE (Binary CrossEntropy) represents the cross-entropy loss function, d represents the domain, d1=1 represents the popular domain, d0=0 represents the unpopular domain, and * represents an element in the set. In this step... This indicates the user's interests or herd mentality;
[0018] Step 7: Minimize the gap between the actual value and the predicted result of the user interaction using the BCE loss function to obtain the fourth loss function L. recommend ,
[0019] ,
[0020] ,
[0021] |B| represents the training batch, Y i This represents the actual value of user interaction with items. This represents a predicted score indicating the user's preference for an item. Indicates the use of only interest-based networks Predict user preferences for items and their predicted scores. Indicates the use of only conformity networks Predict user preferences for items and their predicted scores. Indicates the use of interest-based networks and conformity networks The two networks jointly predict the user's item preference score; this step allows them to learn the user's true preference score, thus narrowing the gap between the true value and the predicted value.
[0022] Step 8: Combine the four loss functions above into a weighted loss function Loss, so that the model can minimize the impact of bias and make the recommendation results more equitable.
[0023] Furthermore, it also includes a testing and evaluation step for the biased model. The testing and evaluation step is as follows: split the three public datasets into 20% portions as test sets, use the biased model to make predictions on the test sets, compare the predicted values with the true values, and calculate the accuracy.
[0024] A recommendation system based on any of the above bias-reduction methods using personalized causal decomposition includes: a data processing module, used to determine the interaction between users and items based on user ratings of items, and to determine the popularity of items based on the number of interactions; and a model training module, defining a neural network to represent user interest, user conformity, and item features, and optimizing four loss functions using the above bias-reduction model algorithm: recommendation loss L... recommend Causal constraint loss L causal Comparison of learning loss L similar Domain adaptive loss L domain Update network parameters; the model evaluation module uses the user matching network to calculate the final recommendation score.
[0025] The advantages of this invention are:
[0026] The introduction of a bias-reduction model based on causal reasoning effectively eliminates bias in recommender systems through three steps: data processing, model training, and model evaluation. This reduces popularity bias, improves personalized and diverse recommendation performance, and further enhances user satisfaction. Attached Figure Description
[0027] The present invention will be further described below with reference to the accompanying drawings and embodiments.
[0028] Figure 1 This is a diagram illustrating the user-item interaction process and its causal relationship with the recommendation results, as presented in this invention.
[0029] Figure 2 This is a schematic diagram of the overall model framework of the present invention.
[0030] Figure 3 This is a computer calculation diagram of the algorithm for an embodiment of the present invention.
[0031] Figure 4 This is a comparison chart of calculation results for embodiments of the present invention.
[0032] Where: U: set of users, I: set of items, Y: set of sorting scores. Implementation
[0033] Example: A cause-effect graph is used to represent the user-item interaction process and its causal relationship with the recommendation results, such as... Figure 1 As shown. In this embodiment, users are categorized by interest attributes. and conformity attribute Items are categorized into popular items. and unpopular items By cross-referencing these two factors, we can obtain users' interest in and conformity levels for items of varying popularity. Ultimately, the model learns users' interest in items of different popularity levels. , and conformity , Based on the cause-effect graph, user attributes are divided into user interests, denoted as... User herd mentality, denoted as ; categorize items into popular items, denoted as Unpopular items are denoted as Therefore, by cross-combining different items, the user's level of interest in popular items can be obtained. Users' level of interest in unpopular items The degree of herd mentality among users towards popular items The degree of herd mentality among users towards unpopular items .
[0034] Use only , The interest prediction recommendation results are used to remove popularity bias.
[0035] Based on causal analysis, this embodiment designs a bias-correction model and trains it on a real-world dataset, such as... Figure 2 As shown, the specific steps are as follows:
[0036] Step 1: In the data processing stage, the dataset is first processed. The popularity of an item is determined by the number of interactions. If the number of interactions is higher than the average, the item is considered a popular item; otherwise, the item is considered unpopular.
[0037] Step 2: Pass the previously categorized popular and unpopular items through an embedding layer, transforming them into representation vectors suitable for matrix operations. Similarly, for users... It also underwent a similar transformation.
[0038] Step 3: Define two neural networks to extract user interests and conformity respectively, denoted as . and By combining the item and user representations obtained in step 2, and then passing them through two neural networks, we can obtain the user's interest representation and conformity representation for items with different popularity levels, denoted as (…). ), ( ), ( ), ( ).
[0039] Step 4: Enhance the decomposition of user interest and conformity using a contrastive learning task. Since user interest should not change with the popularity of items, it is natural to derive the same user's interest representation for popular items. ) and interest representations for unpopular items ( The two interest representations should be similar, and these two interest representations ( , ) and conformity representation ( , These representations are dissimilar. Therefore, a loss function can be derived by calculating the similarity of these representations. By training and iterating the parameters of the neural network defined in step 3, the network can learn the following information:
[0040] .
[0041] Step 5: Enhance the decomposition of user interest and conformity using causal reasoning constraints. Based on the causal graph, the influencing factor Y for the ranking scores of popular and unpopular items can be derived, as shown in the following formula:
[0042]
[0043]
[0044] For bias-correcting recommendation systems, it is necessary to make ( Furthermore, because users are significantly more likely to conform to popular items than to non-popular items, that is ( Therefore, according to the inequality, we can obtain ( This indicates that less popular products better reflect user interest compared to popular products. To ensure the inequality derived from causal reasoning holds, a second loss function is derived:
[0045] .
[0046] Step 6: Further separate the two neural networks proposed in Step 3 using domain adaptation. In the user interest network, since interest does not change with popularity, the representations of items with different popularity levels should be mixed up. However, in the conformity network, the representations of items with different popularity levels should be separated. Use a domain classifier. Separate user representations of items with different popularity levels. A gradient inversion layer is added during the interest representation separation process to improve the representation of user interest in items with different popularity levels. The representation becomes more mixed, and no further operations are taken during the separation of conformity representation, so that users' conformity representation of items with different popularity ( Normal separation. The final result is four separated domains, denoted as ( ), ( ), ( ), ( The calculation process is shown in the following formula:
[0047]
[0048]
[0049] GRL stands for Gradient Reversal Layer, which is a general-purpose neural network. Representation domain classifier The internal parameter of , d represents the domain. Definition Represents the popular domain, defines The field representing unpopular items will contain the information related to popular items that were separated above. , ) Pull towards Similarly, ( , ) Pull towards Therefore, we obtain the third loss function. This makes the interest network... Unaffected by information about the popularity of items, but a herd network The influence of items with different levels of popularity is still retained, thus making the two neural networks more separate.
[0050] .
[0051] Step 7: Ranking Model: The BCE loss function is used to minimize the gap between the true value of user interaction and the predicted result. The purpose is to allow the two neural networks in Step 3 to learn the association weights between users and different items from the user's historical interactions, thereby enhancing the accuracy of the recommendation results.
[0052] ,
[0053] ,
[0054] |B| represents the training batch, Y i This represents the actual value of user interaction with items. This represents a predicted score indicating the user's preference for an item. Indicates the use of only interest-based networks Predict user preferences for items and their predicted scores. Indicates the use of only conformity networks Predict user preferences for items and their predicted scores. Indicates the use of interest-based networks and conformity networks The two networks jointly predict the user's item preference score; this step allows them to learn the user's true preference score, thus narrowing the gap between the true value and the predicted value.
[0055] like Figure 2 As shown, the results of steps 4, 5, 6, and 7 are merged to obtain step 8.
[0056] Step 8: The final loss function is a weighted sum of the four loss functions above, as shown in the formula below. Through joint optimization of these parts, the model minimizes the impact of bias, resulting in fairer, more diverse, and personalized recommendations.
[0057] ,
[0058] , respectively represent L causal L similar L domain The weights of these three loss functions in the overall loss.
[0059] Testing and Evaluation: Two 20% portions of the data from three public datasets were split into test sets. The model trained in step 2 was used to make predictions on these test sets, and the predicted values were compared with the actual values to calculate the accuracy. Experiments demonstrate that this bias-correcting model achieves excellent results on multiple public datasets and in real-world application scenarios. Compared with traditional recommendation algorithms, this model significantly improves recommendation accuracy and effectiveness, and effectively eliminates the repetition bias in recommendation results. This technical solution has broad application prospects in the field of recommendation systems.
[0060] To verify the effectiveness of the method, it was tested on a real-world dataset of user and object interactions. MovieLens 100K (ML100K) was used as the base dataset, and the statistical data of the dataset is shown in Table 1. Detailed implementation steps are as follows:
[0061] Users Items Interactions Sparsity 943 1682 100000 93.70%
[0062] Table 1
[0063] Step 1: Group the 100,000 interaction data in the dataset by item ID to obtain 1,682 data groups. Each data group is represented by its item ID. Count the number of interaction data in each data group, add up the data in each group and calculate the average. Finally, determine whether the number of data in each group is greater than the average. Mark the item IDs that are greater than the average as popular items and the item IDs that are less than or equal to the average as unpopular items.
[0064] Step 2: Split the 100,000 interaction data points in the dataset into two groups in an 8:2 ratio. 80,000 data points are used as the model training dataset to help the biased model learn user interests, while the remaining 20,000 data points are used as the test dataset to evaluate the biased model's performance. After splitting, these data points are transformed into representation vectors that can be used for matrix operations through an embedding layer.
[0065] Step 3: Input the 80,000 transformed representation vectors from Step 2 into the model algorithm, such as... Figure 3 The algorithm is shown below, where: Algorithm 1: Algorithm 1, Require: Input, Ensure: Output, and the rest is code.
[0066] To optimize the model, this embodiment employs the Adam optimizer, which exhibits strong robustness across various deep learning tasks. A grid search technique is used to adjust the model parameters to produce optimal performance values. In this embodiment, the learning rate is set to 0.0001, and the training batch epoch is set to 100. After execution, a bias-reduced model conforming to the characteristics of the ML100K dataset is obtained, representing the implementation result of this algorithm on the ML100K dataset.
[0067] Step 4: To obtain more accurate test results, an unbiased dataset needs to be validated. Therefore, the 20,000 data points extracted in Step 2 are randomly sampled with an equal probability of 40%, meaning the sampling probability is the reciprocal of the number of interactions, to obtain the final test dataset.
[0068] Step 5: Use the test dataset obtained in Step 4 to evaluate and validate the bias-reduced model obtained on the ML100K dataset in Step 3. Take the top 10 (topK=10) and top 20 (topK=20) recommendations from the model, respectively, and compare them with the actual results in the test dataset to calculate metrics such as Recall, HR, and NDCG. Figure 4 As shown. Figure 4In this context, TopK represents the top K recommendation results, NDCG is the evaluation metric, and Normalized Discounted cumulative gain can be translated as normalized discounted cumulative gain. ML-1M, Douban, and Jester are all datasets.
[0069] Step 6: Run other models of the same type using the ML100K dataset to obtain the same evaluation metrics as in Step 5, such as... Figure 4 As shown in the figure. Finally, compared with the results of the PCDR model in step 5, the superiority of this algorithm model is demonstrated.
[0070] Of course, the above embodiments are only for illustrating the technical concept and features of the present invention, and their purpose is to enable those skilled in the art to understand the content of the present invention and implement it accordingly. They should not be used to limit the scope of protection of the present invention. All modifications made according to the spirit and essence of the main technical solution of the present invention should be covered within the scope of protection of the present invention.
Claims
1. A bias-reduction method based on personalized causal decomposition, characterized in that, First, group users according to their interest attributes. and conformity attribute Group items into popular items and unpopular items Then, by cross-referencing user groups and item groups, we obtain users' interest in and conformity levels for items with different popularity levels; finally, we use a bias-reducing model to learn users' interest in items with different popularity levels. , and conformity , Where pop represents popularity, unpop represents obscurity, int represents interest, conf represents conformity, U represents user, I represents item, and D represents degree. This indicates the user's level of interest in popular items. This indicates the user's level of interest in unpopular items. It indicates the degree to which users conform to popular items. This indicates the degree to which users conform to unpopular items; the last item used... , The interest-based prediction and recommendation results are used to remove popularity bias; the specific algorithm for the bias removal model is as follows: Step 1: Classify the dataset and determine the popularity of an item based on the number of interactions between users and the item. If the number of interactions is higher than the average, the item is considered a popular item; otherwise, the item is considered an unpopular item. Step 2: Transform the popular and unpopular items categorized in Step 1 into representation vectors that can be used for matrix operations through an encoding layer. This process is then applied to the user... It is also transformed into a representation vector that can be used for matrix operations through the encoding layer; Step 3: Define two neural networks, an interest network and a conformity network, to extract user interest and conformity respectively, denoted as . and ;in This represents a neural network for extracting user interest. This represents the neural network that extracts user conformity; (·; θ) represents the input of this neural network, where · represents the data input when the neural network needs to extract interest or conformity, θ represents the internal parameters of the neural network, i represents interest, and c represents conformity; By combining the item and user representations obtained in step 2 and then passing them through the two neural networks, we can obtain the user's interest representation and conformity representation for items with different popularity levels, denoted as […]. , , , In this context, 'e' represents popularity, 'pop' represents popularity, 'unpop' represents unpopularity, 'int' represents interest, and 'conf' represents conformity. This indicates a user's interest in popular items. This indicates a user's interest in unpopular items. User conformity representation for popular items; User conformity representation for unpopular items; Step 4: Utilize the contrastive learning task to enhance user interest and decompose conformity to obtain the first loss function L. similar , , γ is a hyperparameter, e is a representation, pop represents popularity, unpop represents unpopularity, int represents interest, conf represents conformity, and * represents an element in the set. In this step... Therefore, * indicates popular or unpopular items in the set; Step 5: Utilize causal reasoning constraints to enhance the decomposition of user interest and conformity, thereby obtaining the second loss function L. causal , ， Where 'e' represents the status quo, 'pop' represents popularity, 'unpop' represents unpopularity, and 'int' represents interest. Step 6: Further separate the two neural networks described in Step 3 using domain adaptation to obtain the third loss function L. domain , , BCE represents the cross-entropy loss function, d represents the domain, d1=1 represents the popular domain, d0=0 represents the unpopular domain, and * represents an element in the set. In this step... This indicates the user's interests or herd mentality; Step 7: Minimize the gap between the actual value and the predicted result of the user interaction using the BCE loss function to obtain the fourth loss function L. recommend , , , |B| represents the training batch, Y i This represents the actual value of user interaction with items. This represents a predicted score indicating the user's preference for an item. Indicates the use of only interest-based networks Predict user preferences for items and their predicted scores. Indicates the use of only conformity networks Predict user preferences for items and their predicted scores. Indicates the use of interest-based networks and conformity networks Jointly predict user preferences for items; Step 8: Combine the four loss functions above into a weighted loss function Loss, so that the model can minimize the impact of bias and make the recommendation results more equitable.
2. The bias removal method based on personalized causal decomposition according to claim 1, characterized in that, It also includes a testing and evaluation step for the biased model, which involves: splitting the three public datasets into 20% portions as test sets, using the biased model to make predictions on the test sets, comparing the predicted values with the true values, and calculating the accuracy.
3. A recommendation system based on a bias-reduction method according to claim 1 or 2, characterized in that, include: The data processing module is used to determine the popularity of an item based on the number of interactions between the user and the item. If the number of interactions is higher than the average, the item is considered popular; otherwise, it is considered unpopular. The module also converts the user and item data into representation vectors that can be used for matrix operations through an encoding layer. The model training module defines an interest network Int and a conformity network Conf, which are used to extract user interest and conformity respectively, and optimizes four loss functions using the bias-reduction model algorithm described in claim 1: recommendation loss L recommend Causal constraint loss L causal Comparison of learning loss L similar Domain adaptive loss L domain Update network parameters; The model evaluation module calculates the final recommendation score using only the user's interest in items of different popularity, as output by the interest network.