Knowledge distillation depolarization recommendation method and system based on causal theory

By introducing causal theory in knowledge distillation, the causal paths of observed and unobserved data are constructed, the causal effect of selection deviation is quantified, and the unbiased soft label is generated, the problem of selection deviation in knowledge distillation is solved, and the performance of the recommendation system is significantly improved.

CN119808836AActive Publication Date: 2025-04-11BEIJING YINGSHI INTELLIGENT TECHNOLOGY CO LTD
View PDF 8 Cites 0 Cited by

Patent Information

Application Number
CN202510292925.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2025-03-13
Publication Date
2025-04-11
Estimated Expiration
2045-03-13

AI Technical Summary

Technical Problem

When generating soft labels, the existing knowledge distillation method fails to effectively solve the problem of selection bias in the recommendation system, resulting in the student model being unable to correct the bias during the learning process, which may intensify the bias and affect performance.

Method used

Using the knowledge distillation and debiased recommendation method based on causal theory, the causal paths of observed data and unobserved data are generated by constructing two teacher models and one student model, the causal paths of observed data and unobserved data are quantified, the causal effect of selection deviation is supplemented, the causal paths of unobserved data are generated, and the unbiased soft labels are generated, and the debiasing knowledge distillation is performed through the Debiasing-KD method.

Benefits of technology

It effectively reduces the selection deviation during knowledge distillation, and the generated unbiased soft labels can more accurately reflect user preferences, improving the learning effect of the student model and the performance of the recommendation system.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119808836A_ABST
    Figure CN119808836A_ABST
Patent Text Reader

Abstract

The invention relates to the technical field of knowledge distillation, in particular to a knowledge distillation depolarization recommendation method and system based on a causal theory. The method comprises the steps that a knowledge distillation depolarization recommendation model is constructed, two teacher models and a student model are constructed, and a causal path of observation data is generated according to a user score data set; supplementing the causal path of the unobserved data based on the causal effect of the quantitative selection deviation; respectively predicting the causal path of the observation data and the causal path of the non-observation data by using the two teacher models, and generating unbiased soft labels; performing knowledge distillation on the student model based on the unbiased soft label to obtain a knowledge distillation debiased recommendation model; and carrying out user prediction by utilizing the knowledge distillation depolarization recommendation model. From the perspective of causal inference, causal reasons generated by selection deviation in the knowledge distillation process are analyzed, and the problem of selection deviation is solved by supplementing causal paths of unobserved data.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of knowledge distillation, and in particular to a knowledge distillation debiasing recommendation method and system based on causal theory. Background Art

[0002] In recommendation systems, preferences are usually learned through interactive feedback between users and recommended items. However, there is an important problem with such preference learning process, namely, selection bias in the user interaction process. Currently, most recommendation systems are built only based on biased data, which cannot provide completely accurate recommendations.

[0003] In recent years, researchers have proposed a variety of methods to solve the selection bias problem in recommendation systems. Among these methods, the technology based on knowledge distillation has shown relatively advanced performance and great potential in alleviating the bias problem in recommendation systems. The traditional knowledge distillation method first pre-trains a large teacher model, and then uses the soft labels distilled from the teacher model to supervise the learning of the small student model. Since these soft labels encode the knowledge learned by the large teacher model, the student model can obtain additional user preference information from it and outperform the case of learning directly from the original training data. However, there are still some problems. The data collected by the recommendation system usually has serious selection bias, and the learning process of the large teacher model mainly relies on these biased user feedback data. Therefore, the teacher model will inevitably inherit this bias and pass it to the soft labels. When the student model uses these biased soft labels for learning, it not only fails to correct its original bias, but may further aggravate this bias, thereby affecting its performance. The main problem is that the teacher model only generates soft labels from the causal path of the observed data, but ignores the soft labels on the causal path of the unobserved data. This approach results in the generated soft labels only reflecting user preferences on some specific causal paths, thus carrying selection bias. Therefore, how to enable a large teacher model to generate unbiased soft labels to assist the student model in learning has become a key issue in reducing selection bias in the knowledge distillation process.

[0004] Based on the above problems, it is urgent to propose a new knowledge distillation debiasing method, namely Debiasing-KnowledgeDistillation (Debiasing-KD), to provide more unbiased soft labels to guide the student model to perform debiased learning. Summary of the invention

[0005] In order to solve the selection bias problem caused by the lack of historical interaction data of users in the knowledge distillation recommendation system, the present invention provides a knowledge distillation debiasing recommendation method and system based on causal theory.

[0006] In a first aspect, the present invention provides a knowledge distillation debiasing recommendation method based on causal theory, which adopts the following technical solution: A knowledge distillation debiasing recommendation method based on causal theory, including: Get the user rating dataset; Construct a knowledge distillation debiased recommendation model, in which two teacher models and one student model are constructed to generate the causal path of the observed data based on the user rating dataset; the causal path of the unobserved data is supplemented based on the causal effect of the quantitative selection bias; and the two teacher models are used to make predictions on the causal path of the observed data and the causal path of the unobserved data to generate unbiased soft labels; Perform knowledge distillation on the student model based on unbiased soft labels to obtain a knowledge distillation debiased recommendation model; Use knowledge distillation to debias the recommendation model for user prediction.

[0007] Furthermore, the obtaining of user observed rating data includes obtaining observed user rating data on the product, wherein: is a set of M users, is a collection of N items; Represented as the true feedback label matrix collected from the user interaction history, where each item Indicates user For items The real rating.

[0008] Furthermore, the causal path of the observed data is generated based on the user rating data set, including inputting the user observed rating data into a display feedback teacher model, the display feedback teacher model makes predictions based on the observed data and generates soft labels, and the path for generating soft labels serves as the causal path of the observed data.

[0009] Furthermore, the causal effect based on quantitative selection bias supplements the causal path of unobserved data, including Biased causal effect formula and unobserved data in the case of The causal effect formula in the case of , by combining the biased causal effect formula and the causal effect formula, the unbiased causal effect formula is obtained, which is expressed as: Among them, users ;thing ; Unbiased soft label .

[0010] Furthermore, the Debiasing-KD method is used to perform debiased knowledge distillation on the causal paths of observed data and the causal paths of unobserved data, respectively, to generate unbiased soft labels, including debiased knowledge distillation based on the Debiasing-KD method, wherein the causal paths obtained by using the explicit feedback teacher model to act on the observed user feedback data; and the causal paths obtained by using the implicit feedback teacher model to act on the unobserved user feedback data, and finally generating unbiased soft labels.

[0011] Furthermore, the method performs knowledge distillation on the student model based on the unbiased soft labels to obtain a knowledge distillation debiased recommendation model, including using the student model to learn with the unbiased soft labels as the fitting target, based on the knowledge learned by the teacher model encoded by the unbiased soft labels, and based on the loss function, making the student model learn the teacher model knowledge in the unbiased soft labels during the learning process. The student model training loss function is expressed as: in Loss function for fitting the student model predictions to the unbiased soft labels.

[0012] The second aspect is a knowledge distillation debiasing recommendation system based on causal theory, including: The data acquisition module is configured to acquire a user rating dataset; The model building module is configured to build a knowledge distillation debiased recommendation model, wherein two teacher models and one student model are built to generate the causal path of the observed data according to the user rating data set; the causal path of the unobserved data is supplemented based on the causal effect of the quantitative selection bias; the two teacher models are used to predict the causal path of the observed data and the causal path of the unobserved data respectively to generate unbiased soft labels; The knowledge distillation module is configured to perform knowledge distillation on the student model based on unbiased soft labels to obtain a knowledge distillation debiased recommendation model; The prediction module is configured to use the knowledge distillation debiased recommendation model to perform user prediction.

[0013] In a third aspect, the present invention provides a computer-readable storage medium storing a plurality of instructions, wherein the instructions are suitable for being loaded and executed by a processor of a terminal device, a knowledge distillation debiasing recommendation method based on causal theory.

[0014] In a fourth aspect, the present invention provides a terminal device including a processor and a computer-readable storage medium, wherein the processor is used to implement various instructions; the computer-readable storage medium is used to store multiple instructions, wherein the instructions are suitable for being loaded by the processor and executing the knowledge distillation debiasing recommendation method based on causal theory.

[0015] In summary, the present invention has the following beneficial technical effects: From the perspective of causal inference, this paper analyzes the causal reasons for selection bias in the knowledge distillation process, and solves the selection bias problem by supplementing the causal path of unobserved data. At the same time, it quantifies the causal effect of selection bias, gives the causal effect formulas on two paths respectively, and further gives the unbiased causal effect formula on the comprehensive path. The present invention innovatively introduces two teacher models to generate soft labels from the causal path of observed data and the causal path of unobserved data respectively, and integrates these two soft labels to generate unbiased soft labels, thereby guiding the student model to perform debiased learning.

[0016] Through experimental analysis, the implicit feedback teacher model has been shown to significantly improve performance, which fully demonstrates the innovation of the implicit feedback teacher model (IFTM) in the Debiasing-KD debiasing method. By supplementing the implicit feedback causal path in the knowledge distillation causal graph, this method can better fit user preferences and generate more unbiased distillation soft labels. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 This is a causal diagram of the selection bias problem in knowledge distillation in Example 1 of the present invention. DETAILED DESCRIPTION

[0018] The present invention is further described in detail below in conjunction with the accompanying drawings.

[0019] Example 1 Reference Figure 1 , a knowledge distillation debiasing recommendation method based on causal theory in this embodiment includes: Get the user rating dataset; Construct a knowledge distillation debiased recommendation model, in which two teacher models and one student model are constructed to generate the causal path of the observed data based on the user rating dataset; the causal path of the unobserved data is supplemented based on the causal effect of the quantitative selection bias; and the two teacher models are used to make predictions on the causal path of the observed data and the causal path of the unobserved data to generate unbiased soft labels; Perform knowledge distillation on the student model based on unbiased soft labels to obtain a knowledge distillation debiased recommendation model; Use knowledge distillation to debias the recommendation model for user prediction.

[0020] Specifically: S1. Obtain user observation rating data, The data is obtained from the existing public datasets Coat Shopping, Yahoo, and KuaiRec. The data is in the form of users' ratings of items. For example, (1, 2, 4) means that user 1 rated item 2 with 4 points.

[0021] Among them, is a set of M users, is a collection of N items. Represented as the true feedback label matrix collected from the user interaction history, where each item Indicates user For items In the recommendation system, since users usually only give feedback on some items, most of them is missing, which leads to the selection bias problem. Represents the soft labels output by the large teacher model, which contains the knowledge learned by the large teacher model. Indicates user feedback Is it observed? If so, ,otherwise .

[0022] S2. Generate the causal path of observed data based on the user rating data set, input the user's observed data into the display feedback teacher model, and the display feedback teacher model makes predictions based on the observed data to generate soft labels. This path to generate soft labels is called the causal path of observed data. (Observed data -> Display feedback teacher model -> Soft label) like Figure 1 (a) shows the causal relationship in the knowledge distillation process of the traditional recommendation system, which contains two causal paths that affect the generation of soft labels. In the ideal case where all user feedback can be observed, according to the user and items Generate unbiased soft labels based on the features , and guides the causal process of student model learning. However, in the actual knowledge distillation process, since users only have feedback on some items, the causal path of generating soft labels changes from the ideal state to , that is, based only on observed user feedback data Generate soft labels while ignoring unobserved user feedback data , which leads to the problem of selection bias in soft labels.

[0023] According to causal analysis, due to the lack of unobserved user feedback data The corresponding causal path leads to the generation of biased soft labels, and further transfers the selection bias to the student model during the distillation process. Therefore, the key to solving the selection bias problem is to recover the bias based on unobserved user feedback. Generate the causal path of soft labels. Specifically, Figure 1 As shown in (b), in the existing path In addition, we need to add another path This ensures that all possible user-item interactions are taken into account, thereby mitigating the impact of selection bias.

[0024] S3. Supplementing the causal path of unobserved data based on the causal effect of quantified selection bias, Among them, the supplementary causal path It is crucial to eliminate selection bias in knowledge distillation. In this section, we use the language of causal inference to infer a formula for the unbiased causal effect that knowledge distillation aims to estimate. First, we give The biased causal effect formula in the case of Finally, the causal effect formulas in these two cases are combined to derive the unbiased causal effect formula.

[0025] let Indicates that the conditional distribution A random variable under which the variable Intervention to a specific value ,variable right The causal effect of The change in value, The value changes, therefore, for users , along the path hour, right The causal effect formula can be expressed as: in represents the baseline situation for comparison. The above formula can be expressed as Conditions and Baseline Conditions The difference below.

[0026] Similarly, for users , along the path hour, right The causal effect formula can be expressed as: The above formula can be expressed as Conditions and Baseline Conditions The difference below.

[0027] Traditional knowledge distillation only A causal path, therefore, eliminating selection bias can be achieved by increasing the causal path To achieve this, the total causal effect can be expressed as: This formula represents the unbiased causal effect when combining two paths, and can be used to generate unbiased soft labels to guide student model learning.

[0028] S4. Use two teacher models to make predictions on the causal path of observed data and the causal path of unobserved data respectively to generate unbiased soft labels.

[0029] In order to achieve debiased knowledge distillation, this paper proposes the Debiasing-KD method. Specifically, this paper introduces two teacher models: the Explicit Feedback Teacher Model (EFTM) and the Implicit Feedback Teacher Model (IFTM), which act on the causal path respectively. and , thereby generating comprehensive unbiased soft labels.

[0030] The display feedback teacher model directly observes user feedback It is easy to generate soft labels. The learning process can be expressed as: (1) in, are the soft labels learned by the teacher model.

[0031] However, for implicit feedback teacher models, directly learning the unobserved data It is impractical to fill in the real labels. Therefore, this paper generates pseudo labels by estimating the post-click conversion rate of items that users have not observed. To replace the real user feedback. In the learning process of the model, the loss function is used to measure the pseudo label Post-click conversion rate predicted by the model The difference between them can improve the accuracy of pseudo labels, which can be expressed as: In the above formula, although ideally the pseudo-label Should be shared with real users However, due to the lack of true labels for these unobserved data, this paper proposes a new idea, which is to use the post-click conversion rate predicted by the display feedback teacher model. With pseudo labels The reason for this is that is obtained by comparing it with the true label Compared with the training, and with the learning of the display feedback teacher model, will continue to approach the true label and become more accurate. In the process of learning Compare and narrow the gap, It will gradually become more accurate.

[0032] Finally, the soft labels generated by the feedback teacher model are displayed and soft labels generated by the implicit feedback teacher model Combine to obtain unbiased soft labels , thereby guiding students’ model learning.

[0033] in, It is a weight parameter used to adjust the contribution ratio of explicit feedback soft labels and implicit feedback soft labels in the final unbiased soft labels.

[0034] As a further implementation method, the Debiasing-Knowledge Distillation method is unbiased analysis. In this section, we analyze the unbiasedness of the Debiasing-KD method. By introducing an explicit feedback teacher model and an implicit feedback teacher model, the Debiasing-KD method aims to generate comprehensive unbiased soft labels. , thereby guiding the student model to learn and reduce the influence of selection bias. In order to verify the unbiasedness of this method, we will conduct a detailed discussion from a theoretical perspective. This paper proposes the following Lemma 1: Lemma 1. For every user , this paper supplements the causal path of unobserved user data , thus generating an unbiased soft label that integrates the two paths Therefore, based on Perform knowledge distillation based on total causal effect Performing knowledge distillation is approximately equivalent.

[0035] For the above Lemma 1, we prove: Proof. Consider any two items and , and assume that the user , the probability of giving feedback or not giving feedback to these two items is the same, that is, , so we have: because is on the path and path The comprehensive unbiased causal effect on , so the distilled soft labels generated by the Debiasing-KD method So far, Lemma 1 has been proved, that is, the unbiasedness of the Debiasing-KD method has been proved.

[0036] S5. Perform knowledge distillation on the student model based on unbiased soft labels to obtain a knowledge distillation debiased recommendation model.

[0037] Among them, the unbiased soft labels are obtained by weighted combination of the soft labels output by the two teacher models; and the unbiased soft labels are passed to the student model; The student model uses unbiased soft labels as the fitting target during the learning process. Since the unbiased soft labels encode the knowledge learned by the large teacher model, the student model can learn the rich knowledge in the unbiased soft labels during the learning process. The student model training loss function is as follows: in Loss function for fitting the student model predictions to the unbiased soft labels.

[0038] In this embodiment, the Debiasing-KD method can theoretically be integrated into various debiasing models. In this section, we apply the Debiasing-KD method to the dual robust (DR) debiasing model for an instantiation study.

[0039] Traditional DR models mainly rely on observed user feedback data during the learning process. Although some existing methods use weighted loss functions to balance data distribution to further reduce bias, traditional DR models have fewer parameters, weaker learning capabilities, and limited user feedback data, which makes it difficult for DR models to achieve ideal debiasing effects.

[0040] To this end, this paper applies the Debiasing-KD method to the DR debiasing model to further reduce the bias. Specifically, first, two large teacher models with more parameters and stronger learning ability are introduced, namely the explicit feedback teacher model and the implicit feedback teacher model, and the two large teacher models are pre-trained using the method in Section 4.3. Then, unbiased soft labels are generated. Finally, based on Perform knowledge distillation to train the DR model. The training loss of the DR model can be expressed as: in, is the training loss between the DR model prediction value and the true label, is the distillation loss. β is a hyperparameter used to balance the contribution between the two losses.

[0041] Experimental verification: Yahoo: This dataset is a dataset of user ratings of songs collected in the music recommendation scenario, which includes a biased subset and an unbiased subset. The biased subset contains 300,000 rating records of 15,400 users on 1,000 songs. These data are collected from the normal historical interactions of users. The unbiased subset is collected from the interactions of the first 5,400 users among these 15,400 users. Each user is required to randomly select 10 songs from 1,000 songs for rating. This random strategy ensures the unbiasedness of the subset. In addition, this paper binarizes the ratings with a threshold of 3, that is, user ratings less than 3 are binarized to 0, otherwise to 1.

[0042] Coat Shopping: This dataset is a user rating dataset collected from the Amazon platform, which contains a biased subset and an unbiased subset. The biased subset contains the rating records of 290 users on 300 coats, which are collected from the normal historical interactions of users. The unbiased subset is obtained by randomly selecting 16 coats from these 290 users for rating. The binarization method of the dataset is the same as that of the Yahoo dataset.

[0043] KuaiRec: This dataset is derived from the recommendation log of the Kuaishou platform, which records 4,676,570 viewing records of 3,237 videos by 1,411 users. This paper uses a threshold of 2 to binarize the dataset, that is, values ​​less than 2 are set to 0, otherwise they are set to 1.

[0044] Four evaluation metrics are used to measure the debiasing performance: mean square error (MSE), area under the curve (AUC), recall (Recall@N), and normalized discounted cumulative gain (NDCG@N). On the Yahoo dataset and Coat Shopping dataset, we set N to 5 and 10; on the KuaiRec dataset, we set N to 20 and 50. All experiments are implemented on the PyTorch framework.

[0045] In the experiment, we selected graph convolutional networks (GCN) and neural collaborative filtering (NCF) models with relatively rich parameters for the teacher model to fully utilize their powerful representation capabilities and prediction accuracy; while for the student model, a relatively simple matrix factorization (MF) model was used as the backbone model, aiming to achieve efficient knowledge transfer through knowledge distillation. This experiment follows previous studies. All models are trained using the Adam optimizer. The learning rate is adjusted in the range of {0.005, 0.01, 0.05, 0.1}, the weight decay is adjusted in the range of {1e-5, 5e-5, 1e-4, 5e-4, 1e-3, 5e-3, 1e-2}, the hyperparameter α is fixed to 0.5, the hyperparameter β ranges from {0.1, 0.2, 0.3, …,1.0}, the batch size on the Coat Shopping dataset is {128, 256, 512, 1024, 2048}, and the batch size on the Yahoo and KuaiRec datasets is {1024, 2048, 4096, 8192, 16384}.

[0046] In this section, we verify the effectiveness of the Debiasing-KD debiasing method on 8 baseline models through comparative experiments.

[0047] Comparative experimental analysis: Tables 1 and 2 show the experimental results of the comparison with other baseline models on three datasets. The model after applying the Debiasing-KD debiasing method outperforms the baseline model in all cases. Specifically, compared with the latest baseline model BEDR-JL, on the Yahoo dataset, after applying the Debiasing-KD debiasing method, MSE, AUC, Recall@5, NDCG@5 and NDCG@10 are improved by 1.4%, 0.4%, 0.8%, 0.83% and 0.44% respectively; on the Coat Shopping dataset, they are improved by 0.9%, 1.2%, 3.6%, 3.4% and 1.8% respectively; on the KuaiRec dataset, MSE, AUC, NDCG@20, NDCG@50, Recall@20 and Recall@50 are improved by 3.17%, 2.6%, 1.48%, 1.18%, 4.08% and 2.19% respectively.

[0048] This shows that the Debiasing-KD method effectively solves the selection bias problem in the knowledge distillation process by supplementing the causal path of unobserved feedback. This method can learn unbiased soft labels through the teacher model and effectively guide the student model to reduce bias, thereby significantly improving the accuracy and robustness of the recommendation system.

[0049] Table 1 Comparative experimental results on Yahoo and Coat Shopping datasets Table 2 Comparative experimental results on the KuaiRec dataset

[0050] Example 2 This embodiment provides a knowledge distillation debiasing recommendation system based on causal theory, including: The data acquisition module is configured as follows: A computer-readable storage medium stores a plurality of instructions, wherein the instructions are suitable for being loaded and executed by a processor of a terminal device, a knowledge distillation debiasing recommendation method based on causal theory.

[0051] A terminal device includes a processor and a computer-readable storage medium, wherein the processor is used to implement various instructions; the computer-readable storage medium is used to store multiple instructions, wherein the instructions are suitable for being loaded by the processor and executing the knowledge distillation debiasing recommendation method based on causal theory.

[0052] The above are all preferred embodiments of the present invention, and are not intended to limit the protection scope of the present invention. Therefore, any equivalent changes made based on the structure, shape, and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A knowledge distillation debiasing recommendation method based on causal theory, characterized in that: include: Get the user rating dataset; Construct a knowledge distillation debiased recommendation model, in which two teacher models and one student model are constructed to generate the causal path of the observed data based on the user rating dataset; the causal path of the unobserved data is supplemented based on the causal effect of the quantitative selection bias; and the two teacher models are used to make predictions on the causal path of the observed data and the causal path of the unobserved data to generate unbiased soft labels; Perform knowledge distillation on the student model based on unbiased soft labels to obtain a knowledge distillation debiased recommendation model; Use knowledge distillation to debias the recommendation model for user prediction.

2. According to the causal theory-based knowledge distillation debiasing recommendation method of claim 1, it is characterized in that: The step of obtaining the user observed rating data includes obtaining the observed user rating data for the product, wherein: is a set of M users, is a collection of N items; Represented as the true feedback label matrix collected from the user interaction history, where each item Indicates user For items The real rating.

3. The knowledge distillation debiasing recommendation method based on causal theory according to claim 2 is characterized in that: The causal path of the observed data is generated based on the user rating data set, including inputting the user observed rating data into the display feedback teacher model, the display feedback teacher model makes predictions based on the observed data and generates soft labels, and the path for generating soft labels serves as the causal path of the observed data.

4. The method for debiasing recommendation based on knowledge distillation based on causal theory according to claim 3, characterized in that: The causal effect based on quantifying selection bias supplements the causal path of unobserved data, including solving the click data Biased causal effect formula and unobserved data in the case of The causal effect formula for the case is, where Indicates that the conditional distribution A random variable under which the variable Intervention to a specific value ,variable right The causal effect of The change in value, The value changes, therefore, for users , along the path hour, right The causal effect formula is expressed as: in represents the baseline situation for comparison, the formula is expressed as Conditions and Baseline Conditions The difference below.

5. The method for debiasing recommendation based on knowledge distillation based on causal theory according to claim 4, characterized in that: The causal effect based on the quantitative selection bias supplements the causal path of the unobserved data, and further includes obtaining an unbiased causal effect formula by merging the biased causal effect formula and the causal effect formula, and synthesizing the unbiased causal effects of the two paths through the unbiased causal effect formula to generate an unbiased soft label. The unbiased causal effect formula is expressed as: Among them, users ;thing ; Unbiased soft label .

6. The knowledge distillation debiasing recommendation method based on causal theory according to claim 5 is characterized in that: The method uses the Debiasing-KD method to perform debiased knowledge distillation on the causal path of observed data and the causal path of unobserved data, respectively, to generate unbiased soft labels, including performing debiased knowledge distillation based on the Debiasing-KD method, wherein the causal path obtained by using the explicit feedback teacher model to act on the observed user feedback data; and the causal path obtained by using the implicit feedback teacher model to act on the unobserved user feedback data, and finally generating unbiased soft labels.

7. The method for debiasing recommendation based on knowledge distillation based on causal theory according to claim 6, characterized in that: The method performs knowledge distillation on the student model based on the unbiased soft label to obtain the knowledge distillation debiased recommendation model, including using the student model to learn with the unbiased soft label as the fitting target, based on the knowledge learned by the teacher model encoded with the unbiased soft label, and based on the loss function, making the student model learn the teacher model knowledge in the unbiased soft label during the learning process. The student model training loss function is expressed as: in Loss function for fitting the student model predictions to the unbiased soft labels.

8. A knowledge distillation debiasing recommendation system based on causal theory, characterized in that: include: The data acquisition module is configured to acquire a user rating dataset; The model building module is configured to build a knowledge distillation debiased recommendation model, wherein two teacher models and one student model are built to generate the causal path of the observed data according to the user rating data set; the causal path of the unobserved data is supplemented based on the causal effect of the quantitative selection bias; the two teacher models are used to predict the causal path of the observed data and the causal path of the unobserved data respectively to generate unbiased soft labels; The knowledge distillation module is configured to perform knowledge distillation on the student model based on unbiased soft labels to obtain a knowledge distillation debiased recommendation model; The prediction module is configured to use the knowledge distillation debiased recommendation model to perform user prediction.

9. A computer-readable storage medium storing a plurality of instructions, characterized in that: The instructions are suitable for being loaded by a processor of a terminal device and executing the method according to claim 1 .

10. A terminal device, comprising a processor and a computer-readable storage medium, wherein the processor is used to implement each instruction; and the computer-readable storage medium is used to store multiple instructions, characterized in that: The instructions are suitable for being loaded by a processor and executing the method as claimed in claim 1 .

Citation Information

Patent Citations

  • Recommendation method and system based on knowledge distillation and causal reasoning

    CN115292587A

  • Method for relieving backdoor attack by using label-free data

    CN115879538A

  • Neural network distillation method and device

    CN116249991A

  • Group recommendation method based on causal depolarization

    CN118708818A

  • Language modal depolarization visual question answering method based on knowledge distillation

    CN118885586A