A method for gain modeling of a recommendation system in presence of unobserved confounding

Abstract

The present application relates to a kind of gain modeling method of recommendation system when unobserved confounding exists, belong to information recommendation technical field, solve the problem that existing technology cannot accurately estimate gain when unobserved confounding exists.Method includes: first training set and second training set are constructed;The sample in first training set is the sample obtained from observational study;The sample in second training set is the sample obtained from randomized controlled trial;Each sample includes user article pair feature, the processing scheme of sample and whether the real outcome of user purchases article;Pre-training model is constructed, and pre-training model is used to predict the potential outcome of sample under different processing scheme;Based on first training set, pre-training model is trained to obtain trained pre-training model;Based on trained pre-training model, fine-tuning model is constructed, and based on second training set, fine-tuning model is trained to obtain the gain prediction model eliminating the influence of unobserved confounding. Accurate unbiased gain estimation is realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of recommendation system, and particularly relates to a gain modeling method of recommendation system in the presence of unobserved confounding. BACKGROUND

[0002] CATE (conditional average treatment effect) refers to the average causal effect of an intervention on a result of interest given a feature, and plays an important role in many fields such as e-commerce, healthcare and economics. In a recommendation system, the platform wants to predict how recommending a particular product or advertisement to a particular user affects the probability of purchase, and thus the total profit. It is worth noting that predicting the conditional average causal effect is very different from traditional machine learning prediction problems. For example, for a particular user, if the user is pushed an advertisement, the probability of the user purchasing a product is 0.9. The traditional prediction problem will only focus on predicting that the user has a probability of 0.9 to purchase the product, and will not focus on what the probability of the user purchasing the product is if the user is not pushed the advertisement. If the user is not pushed the advertisement, the probability of the user purchasing the product is also 0.9, and the causal effect estimated by the causal-based method will be 0.9-0.9=0, so the user will not be recommended the advertisement. However, the traditional prediction problem will consider that the probability of the user purchasing the product is 0.9, which is a very large probability, and thus the user will be recommended the advertisement, resulting in waste of resources. By estimating or predicting the conditional average causal effect of the user in the presence and absence of intervention, i.e. the gain, the intervention measures (pushing advertisements, coupons, etc.) can be more efficiently and accurately implemented on the target population.

[0003] A basic problem in estimating CATE is that we can only observe one outcome for a particular individual. For example, for a particular user, if an e-commerce platform pushes an advertisement, we can only observe the purchase outcome of the user being pushed the advertisement. Conversely, if the e-commerce platform does not push the advertisement, we can only observe the purchase outcome of the user not being pushed the advertisement. Since only one outcome can be observed forever, causal inference methods need to be used to estimate CATE, and traditional machine learning prediction methods cannot be used.

[0004] With the significant progress of reinforcement learning and deep learning, in order to improve the accuracy of CATE estimation, representation-based learning methods have attracted more and more attention due to their excellent performance. These methods focus on generating feature representations, aiming to reduce confounding bias by minimizing the difference in feature representation distribution between the intervention group and the control group. In order to obtain such representations, previous methods have developed a large number of theories and conducted extensive practice. For example, some of them use IPM (integral probability metric) for regularization, some methods emphasize local similarity preservation, and some methods use target learning and optimal transport techniques to estimate CATE. However, the above methods ignore unobserved confounders, which are very common in real-world scenarios. For example, in a recommendation system, the user's financial income status is sensitive and usually cannot be collected. These unobserved variables can affect the treatment plan and the result, thereby causing confounding bias in causal effect estimation. SUMMARY

[0005] In view of the above analysis, the embodiments of the present application aim to provide a gain modeling method for a recommendation system in the presence of unobserved confounding, to solve the problem that the existing gain estimation cannot be accurately performed in the presence of unobserved confounding.

[0006] In one aspect, the embodiments of the present application provide a gain modeling method for a recommendation system in the presence of unobserved confounding, comprising the following steps:

[0007] A first training set and a second training set are constructed; the samples in the first training set are obtained from an observational study; the samples in the second training set are obtained from a randomized controlled trial; each sample includes user-item pair features, sample treatment, and whether the user purchased the item as the true outcome;

[0008] A pre-trained model is constructed, which is used to predict the potential outcome of the sample under different treatments; the pre-trained model is trained based on the first training set to obtain a trained pre-trained model;

[0009] A fine-tuning model is constructed based on the trained pre-trained model, and the fine-tuning model is trained based on the second training set to obtain a gain prediction model that eliminates the influence of unobserved confounding.

[0010] Based on the further improvement of the above method, the pre-trained model includes a first representation module, a reconstruction module, and a first prediction module;

[0011] The first representation module is used to extract the first representation of the sample based on the user-item pair features;

[0012] The reconstruction module is used to reconstruct the sample based on the first representation;

[0013] The first prediction module is configured to predict potential outcomes of the sample under different processing schemes based on the first representation.

[0014] Based on the further improvement of the above method, a fine-tuning model is constructed based on the trained pre-training model, comprising:

[0015] The parameters of the first representation module of the pre-training model are fixed and used as a module of the fine-tuning model;

[0016] A second representation module and a second prediction module are added based on the first representation module;

[0017] The second representation module is configured to extract a second representation of the sample based on the user-item pair feature;

[0018] The second prediction module is configured to predict potential outcomes of the sample under different processing schemes based on the first representation and the second representation, to obtain a gain prediction result eliminating unobserved confounding effects;

[0019] The parameters of the second prediction module are initialized based on the first prediction module.

[0020] Based on the further improvement of the above method, the training loss of the fine-tuning model is calculated using the following formula:

[0021]

[0022] Wherein, represents the prediction loss of the second prediction module, represents the mutual information loss of the first representation and the second representation, represents the parameter bias loss, and λ3 and λ4 represent weights.

[0023] Based on the further improvement of the above method, the mutual information loss of the first representation and the second representation is calculated using the following formula:

[0024]

[0025]

[0026] Wherein, q θ (p(·|·) represents the variational approximation of p(·|·), p(·|·) represents the conditional probability, Z represents the first representation variable of the current training batch sample, Z i represents the first representation of the i-th sample in the current training batch, Z U represents the second representation variable of the current training batch sample, Z U i represents the second representation of the i-th sample in the current training batch, and N represents the number of samples in the current training batch.

[0027] Based on the further improvement of the above method, the parameter deviation loss is calculated by the following formula:

[0028]

[0029] wherein, represents the parameter of the first prediction head of the second prediction module at the initial time, represents the parameter of the second prediction head of the second prediction module at the initial time, represents the current parameter of the first prediction head of the second prediction module, represents the current parameter of the second prediction head of the second prediction module, and ||·||2 represents the l2 norm.

[0030] Based on the further improvement of the above method, the prediction loss of the second prediction module is calculated by the following formula:

[0031]

[0032] wherein, N represents the number of samples in the current training batch, l(·,·) represents the loss function, Y i represents the true outcome of the i-th sample in the current training batch, represents the latent outcome of the i-th sample predicted by the second prediction module, w i represents the weight of the i-th sample T i represents the treatment scheme of the i-th sample, n represents the number of samples in the second training set, T j represents the treatment scheme of the j-th sample in the second training set.

[0033] Based on the further improvement of the above method, the training loss of the pre-trained model is calculated by the following formula:

[0034]

[0035] wherein, represents the prediction loss of the first prediction module, represents the reconstruction loss, represents the feature imbalance loss, and λ1 and λ2 represent the weights.

[0036] Based on the further improvement of the above method, the prediction loss of the first prediction module is calculated by the following formula:

[0037]

[0038] wherein, N represents the number of samples in the current training batch, l(·,·) represents the loss function, Y i represents the true outcome of the i-th sample in the current training batch, wi represents the potential outcome of the i-th sample predicted by the first prediction module, w i wi represents the weight of the i-th sample, mi represents the treatment scheme of the i-th sample, m represents the number of samples in the first training set, T j mj represents the treatment scheme of the j-th sample in the first training set.

[0039] Based on the further improvement of the above method, the feature imbalance loss is obtained by calculating the integral probability metric between the first representations of the samples with intervention treatment scheme and the samples without intervention treatment scheme in the current training batch.

[0040] Compared with the prior art, the present application constructs the first training set and the second training set by extracting samples from the observational study and the self-controlled test respectively, trains a pre-training model based on the samples of the observational study in a two-stage framework, that is, to obtain the basic representation of the feature to estimate counterfactual outcomes, and then adjusts the bias of the representation learned from the observational study data based on the samples of the small-scale randomized controlled test, so as to obtain an unbiased gain prediction model eliminating unobserved confounding effects, avoiding overfitting problem. According to the trained gain prediction model, the intervention gain can be accurately predicted, so that the intervention measures can be accurately implemented according to the predicted gain, and the efficiency can be improved and the resource allocation can be optimized.

[0041] In the present application, the above technical solutions can also be combined with each other to realize more preferred combination schemes. Other features and advantages of the present application will be described in the subsequent specification, and some advantages will become apparent from the specification, or will be understood by implementing the present application. The purpose and other advantages of the present application can be achieved and obtained by the contents specifically pointed out in the specification and the drawings. BRIEF DESCRIPTION OF DRAWINGS

[0042] The accompanying drawings are included to provide a further understanding of the application and are incorporated in and constitute a part of this specification, illustrate embodiments of the application and together with the description serve to explain the principles of the application, and should not be considered as limiting the scope of the application.

[0043] Figure 1 Flowchart of the gain modeling method of the recommendation system in the presence of unobserved confounding for the embodiments of the present application;

[0044] Figure 2 Schematic diagram of the two-stage CATE estimation framework for the embodiments of the present application. DETAILED DESCRIPTION

[0045] The preferred embodiments of the present application will be specifically described below in conjunction with the drawings, wherein the drawings constitute a part of this application and are used to illustrate the principles of the embodiments of the present application, and should not be considered as limiting the scope of the present application.

[0046] Unobserved confounding refers to the existence of unobserved variables in the study that affect the relationship between the intervention and the outcome, which can cause bias in the CATE estimate. Observational studies (OBS) are naturally collected data in real life, and due to the existence of unobserved confounding in these data, the CATE estimate based on the sample obtained from the observational study alone will be biased. Randomized controlled trial (RCT) data is considered the gold standard for causal effect estimation. Randomized controlled trial data, that is, artificially randomly assigned treatment schemes (that is, individuals are in the intervention group or the control group), is not affected by confounding from randomized controlled trials (RCT) data, however, the collection of these data has high cost, and the sample size is usually small. Directly training a causal effect prediction model on RCT data alone faces the risk of overfitting due to insufficient RCT sample size.

[0047] To accurately estimate the gain, one specific embodiment of the present application discloses a gain modeling method for a recommendation system in the presence of unobserved confounding, as shown in Figure 1 The method comprises the following steps:

[0048] S1, constructing a first training set and a second training set; the samples in the first training set are obtained from observational studies; the samples in the second training set are obtained from randomized controlled trials; each sample includes user-item pair features, sample treatment schemes, and the true outcome of whether the user purchased the item;

[0049] S2, constructing a pre-trained model, the pre-trained model is used to predict the potential outcome of the sample under different treatment schemes; training the pre-trained model based on the first training set to obtain a trained pre-trained model;

[0050] S3, constructing a fine-tuning model based on the trained pre-trained model, training the fine-tuning model based on the second training set to obtain a gain prediction model that eliminates the influence of unobserved confounding.

[0051] Compared with the prior art, the gain modeling method for a recommendation system in the presence of unobserved confounding provided by the embodiment extracts samples from observational studies and randomized controlled trials to construct a first training set and a second training set, respectively, and uses a two-stage framework, that is, first trains a pre-trained model based on the samples of the observational study to obtain the basic representation of the features to estimate counterfactual outcomes, and then adjusts the bias of the representation learned from the observational study data based on the small-scale randomized controlled trial samples, so as to obtain an unbiased gain prediction model that eliminates the influence of unobserved confounding, thereby avoiding the problem of overfitting. According to the trained gain prediction model, the intervention gain can be accurately predicted, so that the intervention measures can be accurately implemented according to the predicted gain, and the efficiency can be improved and the resource allocation can be optimized.

[0052] Moreover, the prior art often makes gain estimation under strong assumptions, for example, assuming that all individual statistical characteristics under no unobserved confounding are similar to individuals under unobserved confounding, independent of the assumption, linearly additive generation assumption, these assumptions cannot be tested by data, and are easy to violate in practice, and the present application does not depend on these assumptions, does not limit the data generation process (for example, linear or additive noise assumption), and is therefore more in line with practical application conditions, and is robust for different intensities of unobserved confounding and different proportions of RCT data.

[0053] In implementation, each sample contains user-item pair feature X, treatment scheme T (T ∈ {0, 1}, T = 1 represents an intervention group, and an intervention measure is implemented on the user, and T = 0 represents a control group, and no intervention measure is implemented on the user) and true outcome Y ∈ {0, 1} (Y = 1 represents that the user has purchased the item, and Y = 0 represents that the user has not purchased the item). In addition, G represents the set to which the sample belongs, represents that the sample belongs to the OBS set, represents that the sample belongs to the RCT set, that is, the second training set, and there is no confounding bias. The i-th sample is represented as D i = (X i , T i , Y i , G i ). According to the Neyman-Rubin latent outcome framework, Y (1), Y (0) are defined as latent outcomes, Y (0) represents a latent outcome without implementing an intervention measure, and Y (1) represents a latent outcome with implementing an intervention measure. Only one outcome can be observed for a specific individual, and therefore the true outcome Y of the sample is one of the latent outcomes Y (1) or Y (0).

[0054] In implementation, the user-item pair feature can be a feature obtained according to user attributes and item attributes.

[0055] The propensity score, that is, the probability of implementing an intervention measure on the sample, is e (X, G) = P (T = 1 | | (X, G)), and the conditional average outcome is μ (X, G) = E [Y | X, G]. The CATE is defined as the conditional expectation difference of the latent outcome, and is represented as follows:

[0056] τ (x) = E [Y (1) - Y (0) | X = x].

[0057] That is, the mean of the difference between the latent outcome of implementing an intervention measure and the latent outcome of not implementing an intervention measure when the feature of the sample is x.

[0058] To identify CATE from observational data, two additional assumptions are needed besides SUTVA (stable unit treatment value assumption):

[0059] Assumption 1: Ignorability (Y(1), Y(0)) T | X

[0060] This means that, given the features X, the potential outcomes are independent of the intervention assignment.

[0061] Assumption 2: Positivity

[0062] This means that, for all features X, the probability of implementing the intervention e(X, G) must be between 0 and 1 in both RCT and OBS data, ensuring that each individual has a chance of receiving different treatment regimes.

[0063] Assumption 1 is also known as no unobserved confounding, which is true by default in RCT due to random treatment assignment. We can identify and compute CATE based on RCT data as follows:

[0064]

[0065] In OBS data, there can be unobserved confounding. Therefore, for observational data, we do not assume no confounding, i.e., the following relationship does not hold:

[0066]

[0067] This means we may not be able to identify τ(x) based on OBS data alone. We express the difference in conditional average outcomes in observational data with the following equation:

[0068]

[0069] Due to unobserved confounding, ω(x) ≠ τ(x) holds for any x. The difference between these two quantities is exactly the confounding effect, which we call the residual function, denoted as:

[0070] n(x) = τ(x) - ω(x).

[0071] The residual function η(x) captures the bias between CATE and the conditional average outcome in observational data due to unobserved confounding.

[0072] Therefore, the residual function is defined as follows:

[0073]

[0074]

[0075] Here, η t (x) represents the residual between OBS data and RCT data in the intervention group, while η c (x) represents the residual between OBS data and RCT data in the control group. These two residual functions reflect the impact of unobserved confounders on different treatment groups.

[0076] Based on the above analysis, the present application adopts a two-stage CATE estimation framework of the pre-training-fine-tuning paradigm, as shown in Figure 2 , the first stage utilizes large-scale OBS data to train the basic representation of features, and the second stage uses relatively small-scale unbiased RCT data to calibrate the bias in the representation learned from OBS data to train an unbiased prediction model.

[0077] In implementation, the first stage trains a pre-training model based on a first training set. Specifically, the pre-training model includes a first representation module, a reconstruction module, and a first prediction module;

[0078] The first representation module is used to extract a first representation of the user-item pair feature of the sample based on the user-item pair feature of the sample;

[0079] The reconstruction module is used to reconstruct the sample based on the first representation;

[0080] The first prediction module is used to predict the potential outcome of the sample under different treatment schemes based on the first representation.

[0081] The goal of the pre-training module is to accurately estimate the potential outcome of the OBS data. Therefore, first, the deep features of the sample, i.e., the first representation, are extracted by the first representation module, and then the first prediction module is used to predict the potential outcome based on the deep features. In implementation, the first representation module can use a multi-layer feedforward neural network, denoted as φ. The first representation module obtains the first representation Z of the intervention group and control group features X. In other words, for a single sample Whether T i = 0 or T i = 1, the representation Z i = φ(X i ) is the same. In order to better estimate the causal effect, we use the idea of feature balancing in the representation space proposed by Shalit et al. Specifically, the representation of the intervention group is considered as an independent and identically distributed sample randomly drawn from the distribution , and the control group is similarly considered as an independent and identically distributed sample randomly drawn from . We expect that the representation distribution of the intervention group and the control group is similar.

[0082] In implementation, the first prediction module includes two prediction heads denoted as h0 and h1, which respectively predict the potential outcomes of the sample under the processing scheme T=0 and the processing scheme T=1. The prediction heads employ multi-layer perceptrons.

[0083] To ensure that the extracted first representation retains as much information about the original features as possible, a reconstruction module is employed to reconstruct the sample based on the first representation. In implementation, the reconstruction module employs a decoder network ψ to reconstruct the original features

[0084] By adding the reconstruction module, the learned first representation is able to contain information in the features, not just the information needed to fit the predicted outcomes Y, thus enhancing the generalization capability of the first representation module.

[0085] Specifically, the training loss of the pre-trained model is calculated using the following formula:

[0086]

[0087] wherein, represents the prediction loss of the first prediction module, represents the reconstruction loss, represents the feature imbalance loss, and λ1 and λ2 represent weights.

[0088] Specifically, the prediction loss of the first prediction module is calculated using the following formula:

[0089]

[0090] wherein, N represents the number of samples in the current training batch, l(·,·) represents a loss function, Y i represents the true outcome of the i-th sample in the current training batch, represents the potential outcome of the i-th sample predicted by the first prediction module, w i represents the weight of the i-th sample. In implementation, the loss function l(·,·) can employ a cross-entropy loss.

[0091] It should be noted that the two prediction heads of the first prediction module respectively predict the potential outcomes of the sample in the intervention group and the control group, while the true outcome of the sample is one of them, and thus, if the sample is actually in the intervention group, the prediction result of the prediction head h1 is taken, and if the sample is actually in the control group, the prediction result of the prediction head h0 is taken. the prediction result of the prediction head h1 is taken, and if the sample is actually in the control group, the prediction result of the prediction head h0 is taken. the prediction result of the prediction head h1 is taken, and if the sample is actually in the control group, the prediction result of the prediction head h0 is taken.

[0092] In implementation, the weight of the i-th sample is calculated using the following formula:

[0093]

[0094] where m represents the number of samples in the first training set, T j represents the processing scheme of the jth sample in the first training set, T i represents the processing scheme of the ith sample.

[0095] To balance the characteristics between the intervention group and the control group, a feature imbalance loss is obtained by calculating the integral probability metric between the first representation of the sample with intervention processing scheme and the first representation of the sample with non-intervention processing scheme in the current training batch.

[0096] After training the pre-trained model based on the first training set to obtain a trained pre-trained model, a fine-tuning model is constructed in the second stage to obtain a gain prediction model that eliminates unobserved confounding effects.

[0097] Specifically, the fine-tuning model is constructed based on the trained pre-trained model, comprising:

[0098] The parameters of the first representation module of the pre-trained model are fixed and used as a module of the fine-tuning model;

[0099] A second representation module and a second prediction module are added based on the first representation module;

[0100] The second representation module is used to extract the second representation of the sample based on the user-item pair characteristics;

[0101] The second prediction module is used to predict the potential outcome of the sample under different processing schemes based on the first representation and the second representation, to obtain a gain prediction result that eliminates unobserved confounding effects;

[0102] The parameters of the second prediction module are initialized based on the first prediction module.

[0103] In the second stage, the fine-tuning model keeps the bias representation Z generated by φ unchanged, but regards it as part of the representation, while introducing another representation module (second representation module) φ U The additional representation (second representation) Z U generated is used to adjust the final representation to make it unbiased.

[0104] In implementation, the second representation module can adopt a feedforward neural network, φ U The width and depth of φ U can be adjusted according to the size of the RCT data. If the size of the RCT data is comparable to that of the OBS data, we can use the same architecture as φ. In reality, the RCT data is relatively scarce compared to the OBS data, so the size of φ U should be smaller.

[0105] Similarly, the fine-tuning model uses a second prediction module to predict the potential outcomes of samples under different treatment schemes based on the first and second representations. The second prediction module includes a first prediction head g0 and a second prediction head g1. The first prediction head g0 is used to predict the potential outcomes of samples in the control group, and the second prediction head g1 is used to predict the potential outcomes of samples in the intervention group.

[0106] It should be noted that g0,g1 and h0,h1 have the same depth l p However, the width of each layer of g0 and g1 is greater than that of h0 and h1.

[0107] In implementation, the model architecture for the second stage of fine-tuning differs from that of the first stage. Therefore, the initialization of model parameters is crucial to the effectiveness of the fine-tuning stage. The goal of initialization is to ensure that the model produces the same predictions as the pre-trained model trained in the first stage. Therefore, the parameters of the second prediction module are initialized based on the first prediction module. That is, the prediction head g0 of the second prediction module is initialized based on the prediction head h0 of the first prediction module, and the prediction head g1 of the second prediction module is initialized based on the prediction head h1 of the first prediction module.

[0108] To simplify the representation, g is used. t This represents the prediction header of the second prediction module. t This represents the prediction header of the first prediction module, t∈{0,1}.

[0109] During implementation, firstly g t The parameters are divided into blocks:

[0110]

[0111] in, G represents t Weight parameters from layer (l-1) to layer l. It is g t The bias parameters of the l-th layer shape and same. h t Weight parameters from layer (l-1) to layer l. It is h t The bias parameters of the l-th layer. l = 1, 2, ..., l p .

[0112] Based on the parameter partitioning, the following initialization strategy is proposed:

[0113]

[0114] Imminent parameters Initialize to Parameters, parameters Initialize to The parameter is set to 0, and the other parameters are initialized to 0.

[0115] In implementation, the Modified Linear Unit (ReLU) is used as the activation function so that g t Discontinued and h t Compared to additional neurons.

[0116] Then, the fine-tuned model is trained based on the second training set.

[0117] Specifically, the training loss of the fine-tuned model is calculated using the following formula:

[0118]

[0119] in, This represents the prediction loss of the second prediction module. This represents the mutual information loss between the first and second representations. λ3 and λ4 represent the parameter bias loss, and λ3 and λ4 represent the weights.

[0120] Specifically, the prediction loss of the second prediction module is calculated using the following formula:

[0121]

[0122] Where N represents the number of samples in the current training batch, l(·,·) represents the loss function, and Y i This represents the true outcome of the i-th sample in the current training batch. w represents the potential outcome predicted by the second prediction module for the i-th sample. i The weight of the i-th sample n represents the number of samples in the second training set, T i T represents the processing scheme for the i-th sample. j This represents the processing scheme for the j-th sample in the second training set.

[0123] Similarly, if the sample is actually in the intervention group, then Take the prediction result of the second prediction head g1. If the sample is actually in the control group, then... Take the prediction result of the first prediction head g0.

[0124] To ensure Z U To capture features different from Z, mutual information is used to control the overlap between the two representations; that is, the mutual information loss of the first and second representations is added to the loss function. Specifically, the mutual information loss of the first and second representations is calculated using the following formula:

[0125]

[0126]

[0127] where q θ (·|·) denotes the variational approximation of p(·|·), p(·|·) denotes the conditional probability, Z denotes the first representation variable of the current training batch samples, Z i denotes the first representation of the i-th sample of the current training batch, Z U denotes the second representation variable of the current training batch samples, Z U i denotes the second representation of the i-th sample of the current training batch, N denotes the number of samples of the current training batch.

[0128] It is assumed that unobserved confounders can cause the distribution of OBS data to deviate slightly from that of RCT data. Therefore, the fine-tuning model in the second stage should not deviate significantly from the pre-training model in the first stage. Let the initial parameters of the fine-tuning model be To constrain the deviation from the initial value, a parameter deviation loss is added to the loss function. Specifically, the parameter deviation loss is calculated using the following formula:

[0129]

[0130] where, denotes the parameters of the first prediction head of the second prediction module at the initial time, denotes the parameters of the second prediction head of the second prediction module at the initial time, denotes the current parameters of the first prediction head of the second prediction module, denotes the current parameters of the second prediction head of the second prediction module, ||·||2 denotes the l2 norm.

[0131] Since there are no unobserved confounders in RCT data, by adjusting the unobserved confounders according to RCT data in the second stage, the representation learned from OBS data is calibrated, the influence of unobserved confounders is eliminated, and a gain prediction model that eliminates the influence of unobserved confounders is obtained. To avoid overfitting caused by small-scale RCT data in the second stage, the network parameters are partially initialized in the second stage, using the parameters of the pre-training network in the first stage instead of training an independent network. Compared with the prior art, our method has the advantage of not limiting the data generation process (e.g., linear or additive noise assumption), and can effectively alleviate the overfitting problem.

[0132] For the user-item pair to be processed, its features are input into the gain prediction model, and the second prediction module predicts its potential outcomes in the prediction group and the control group. The difference between the potential outcomes in the prediction group and the control group is the intervention gain, which can be used to determine whether to intervene in the user. For example, if the intervention gain exceeds the predicted threshold, intervention measures are implemented for the user, otherwise no intervention measures are implemented.

[0133] The present application significantly improves the estimation accuracy of conditional average treatment effect (CATE) by combining large-scale observational study data and small-scale randomized controlled trial (RCT) data. It can effectively reduce the bias caused by unmeasured confounding factors and provide more reliable causal inference basis for policymakers, thereby optimizing resource allocation and intervention measures.

[0134] In implementation, the Jobs dataset is used to verify the method of the present application. The Jobs dataset is a benchmark dataset commonly used in causal gain modeling, which includes observational data (OBS data) and randomized controlled trial data (RCT data). The Jobs dataset has a binary treatment variable T i , which represents whether the i-th user has accepted job training (T i = 1 represents training, T i = 0 represents no training), and an outcome variable Y i , which is the employment status after training (Y i = 1 represents employment, Y i = 0 represents unemployment). The covariates include age, education, previous income, etc. In implementation, the OBS data and a small amount of RCT data are taken as the training set to train the gain prediction model according to the method of the present application, and the other RCT data are taken as the validation set for effect verification.

[0135] In implementation, the strategy risk R pol , a commonly used evaluation index on the Jobs dataset, is used to evaluate the effect of the algorithm of the present application and existing algorithms. Specifically, R pol is defined as:

[0136]

[0137] represents the expectation, represents the probability, Y(1) represents the actual result of the sample implementing the intervention measure, represents the predicted potential result of the sample implementing the intervention measure, represents the predicted potential result of the sample not implementing the intervention measure, T = 1 represents the intervention group of the sample, and T = 0 represents the control group of the sample. It is worth noting that R Pol is measured on randomized controlled data, so it is an unbiased evaluation criterion. The specific experimental results are shown in Table 1 below, where out-of-sample means that the results are evaluated on unbiased data in the test set, and in-sample means that the results are evaluated on unbiased data in the training set.

[0138] Table 1 Comparison of effects of different methods

[0139]

[0140] We consider three categories of baseline methods: the first category is based on meta-learning, including T-learner method, the second category is based on tree model, including Causal Forest, the third category is based on representation learning, including TarNet, DragonNet and ESCFR. We can find that the risk of our proposed method is statistically significantly lower than the baseline method both in-sample and out-of-sample, because we can use RCT data more reasonably, and the flexible model structure makes our method avoid the risk of overfitting.

[0141] Those skilled in the art can understand that all or part of the processes of the above-mentioned embodiment methods can be completed by instructing relevant hardware through a computer program, and the program can be stored in a computer readable storage medium. The computer readable storage medium is a disk, an optical disk, a read-only memory, a random access memory, etc.

[0142] The above describes only the preferred specific embodiments of the present application, but the protection scope of the present application is not limited to this. Any person skilled in the art can easily think of changes or replacements within the technical scope disclosed by the present application, which should be covered within the protection scope of the present application.

Claims

1. A method for gain modeling of a recommender system in presence of unobserved confounding, the method comprising: The method comprises the following steps: constructing a first training set and a second training set; samples in the first training set are obtained from an observational study; samples in the second training set are obtained from a randomized controlled trial; each sample comprises user-item pair features, a treatment scheme of the sample, and a real outcome of whether the user purchases the item; constructing a pre-training model, which is used to predict potential outcomes of samples under different treatment schemes; training the pre-training model based on the first training set to obtain a trained pre-training model; constructing a fine-tuning model based on the trained pre-training model, and training the fine-tuning model based on the second training set to obtain a gain prediction model that eliminates unobserved confounding effects; the gain is a conditional average causal effect of potential outcomes under different treatment schemes; the pre-training model comprises a first representation module, a reconstruction module, and a first prediction module; the first representation module is used to extract first representations of samples based on user-item pair features; the reconstruction module is used to reconstruct samples based on the first representations; the first prediction module is used to predict potential outcomes of samples under different treatment schemes based on the first representations; constructing a fine-tuning model based on the trained pre-training model comprises: fixing parameters of the first representation module of the pre-training model and taking the parameters as a module of the fine-tuning model; adding a second representation module and a second prediction module to the first representation module; the second representation module is used to extract second representations of samples based on user-item pair features; the second prediction module is used to predict potential outcomes of samples under different treatment schemes based on the first representations and the second representations, and obtain gain prediction results that eliminate unobserved confounding effects; parameters of the second prediction module are initialized based on the first prediction module.

2. The method of claim 1, wherein the gain modeling method for a presence unobserved mixed recommendation system is characterized by, The training loss of the fine-tuning model is calculated by using the following formula: wherein, represents a prediction loss of the second prediction module, represents a mutual information loss of the first representation and the second representation, represents a parameter bias loss, and λ3 and λ4 represent weights.

3. The method of claim 2, wherein the gain modeling method for a presence unobserved hybrid recommendation system is characterized by, The mutual information loss of the first representations and the second representations is calculated by using the following formula: where q θ (·|·) denotes a variational approximation of p(·|·), p(·|·) denotes a conditional probability, Z denotes a first representation variable of a current training batch sample, Z i denotes a first representation of an i-th sample of the current training batch, Z U denotes a second representation variable of a current training batch sample, Z U i denotes a second representation of an i-th sample of the current training batch, N denotes a number of samples of the current training batch.

4. The method of claim 2, wherein the gain modeling method for a presence-unobserved mixed recommendation system is characterized by, The parameter bias loss is calculated by using the following formula: wherein, denotes a parameter of the first prediction head of the second prediction module at initialization, denotes a parameter of the second prediction head of the second prediction module at initialization, denotes a parameter of the first prediction head of the second prediction module at the current time, denotes a parameter of the second prediction head of the second prediction module at the current time, || · ||2denotes the l2-norm.

5. The method of claim 2, wherein the gain modeling method for a presence-unobserved mixed recommendation system is characterized by, The prediction loss of the second prediction module is calculated by using the following formula: wherein N represents the number of samples of the current training batch, l(·, ·) represents a loss function, Y i represents the true outcome of the i-th sample of the current training batch, represents the latent outcome of the i-th sample predicted by the second prediction module, w i represents the weight of the i-th sample, T i represents the treatment scheme of the i-th sample, n represents the number of samples of the second training set, T j represents the treatment scheme of the j-th sample in the second training set.

6. The method of claim 1, wherein the gain modeling method for a recommendation system in presence of unobserved mixtures is characterized by, The training loss of the pre-training model is calculated by using the following formula: wherein, represents a prediction loss of the first prediction module, represents a reconstruction loss, represents a feature imbalance loss, and λ1and λ2represent weights.

7. The method of claim 6, wherein the gain modeling method for a presence unobserved hybrid recommendation system is characterized by, The prediction loss of the first prediction module is calculated by using the following formula: wherein N represents the number of samples of the current training batch, l(·, ·) represents a loss function, Y i represents the true outcome of the i-th sample of the current training batch, represents the latent outcome of the i-th sample predicted by the first prediction module, w i represents the weight of the i-th sample, T i represents the treatment scheme of the i-th sample, m represents the number of samples of the first training set, T j represents the treatment scheme of the j-th sample in the first training set.

8. The method of claim 6, wherein the gain modeling method for a presence-unobserved mixed time recommendation system is characterized by, The feature imbalance loss is calculated by calculating the integral probability metric between the first representations of samples with the intervention treatment scheme and the first representations of samples with the non-intervention treatment scheme in the current training batch.

Images