Learning device and learning method

Abstract

This learning device is for learning a model for estimating a treatment effect, and comprises a learning unit for learning the model by minimizing an objective function using an inverse probability weight. The objective function includes an inverse probability weight correction model in which, among a plurality of inverse probability weight values assigned to a plurality of individuals, the values of a predetermined number of upper inverse probability weights are replaced with other values.

Description

Learning device and learning method

[0001] The present invention relates to a technique for estimating treatment effects (causal effects), which are measures for quantifying the effects of treatments such as drug administration or participation in educational programs when they are administered to individuals.

[0002] Treatment effect (causal effect) is a measure that quantifies the effect of a treatment, such as administering a drug or participating in an educational program, on outcome variables such as cholesterol levels or grades when administered to an individual. Accurate estimation of treatment effect is expected to lead to a variety of applications, including personalized medicine, personalized education, and targeted advertising.

[0003] It is not possible to calculate such a treatment effect for each individual because, although the treatment effect is defined as the difference between the potential outcome with the treatment and the potential outcome without the treatment, when an individual is treated, only the potential outcome with the treatment is observed, not the outcome without the treatment.

[0004] For this reason, in conventional techniques, the conditional average treatment effect (CATE) has been used as an evaluation index of treatment effects that can be estimated from observed data. CATE is defined as the average amount of treatment effects for multiple individuals who have the same attributes for all observed individual characteristics.

[0005] In order to estimate CATE, it is necessary to properly correct for bias in the data (selection bias) due to characteristics that affect both the treatment variable and the outcome variable, called confounding variables.

[0006] For example, consider the case of evaluating the impact of medical treatment on the health status of cancer patients. In this case, age is a potential confounding variable. Older patients have a higher mortality rate, and many patients choose radiation therapy due to the risks of surgery. Therefore, fewer older patients choose surgery.

[0007] A widely used approach to correct such bias in data is a technique called inverse probability weighting (IPW) (Non-Patent Document 1). IPW constructs estimators using inverse probability weights, defined as the inverse of each individual's probability of receiving treatment (propensity score). The advantage of this approach is that by minimizing the weighted mean prediction error, it is possible to construct estimators while distinguishing whether given input features are confounding variables or features that do not require correction (Non-Patent Documents 2 and 3). This is an important advantage for accurately determining treatment effects (Non-Patent Document 4).

[0008] Paul R. Rosenbaum and Rubin B. Donald. "The central role of the propensity score in observational studies for causal effects". Biometrika, 70(1): 41-55, 1983. Kun Kuang, Peng Cui, Hao Zou, Bo Li, Jianrong Tao, Fei Wu, and Shiqiang Yang. "Data-Driven Variable Decomposition for Treatment Effect Estimation". IEEE Transactions on Knowledge and Data Engineering. 34(5): 2120-2134, 2022.Negar Hassanpour and Russell Greiner. "Learning Disentangled Representations for Counterfactual Regression". ICLR, 2020.Brian C. Sauer, Alan Brookhart, Jason Roy, and Tyler Vanderweele. "A Review of Covariate Selection for Nonexperimental Comparative Effectiveness Research". Pharmacoepidemiology and drug safety, 22(11): 1139-1145, 2013.

[0009] However, when high-dimensional data containing many features are given, it is difficult to estimate the propensity score with high accuracy. Specifically, the inverse probability weights often take extremely large values, which deteriorates the estimation accuracy of CATE.

[0010] The present invention has been made in consideration of the above points, and aims to provide a technology for effectively correcting the values ​​of inverse probability weights when training a model for estimating treatment effects, thereby obtaining a model that estimates treatment effects with higher accuracy.

[0011] According to the disclosed technology, there is provided a learning device that learns a model for estimating treatment effects, the learning device comprising: a learning unit that learns the model by minimizing an objective function using inverse probability weights, wherein the objective function has an inverse probability weight correction model that replaces a predetermined number of inverse probability weight values, among multiple inverse probability weight values ​​assigned to multiple individuals, with different values.

[0012] The disclosed technology makes it possible to obtain a model that estimates treatment effects with higher accuracy.

[0013] It is a block diagram of the learning device 100 according to an embodiment of the present invention. It is a flowchart showing the operation of the learning device 100. It is a diagram showing an example of the hardware configuration of the device.

[0014] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. The embodiment described below is merely an example, and the embodiment to which the present invention is applied is not limited to the following embodiment.

[0015] As described above, when a treatment such as administering a drug or participating in an educational program is administered to an individual, there is a treatment effect (causal effect) as a measure to quantify the effect. This treatment effect often varies depending on the characteristic values ​​(attributes) of each individual. In this embodiment, a model is trained to estimate the magnitude of the treatment effect from high-dimensional data for each individual, which consists of the values ​​of many of the individual's characteristics, whether or not the treatment was administered, and the resulting values.

[0016] In particular, in this embodiment, a correction model with inverse probability weighting, defined as the reciprocal of the probability value representing the presence or absence of treatment, is used in the objective function for learning a model that estimates the magnitude of the treatment effect. Specifically, this is as follows:

[0017] (Overview of the Technology According to the Present Embodiment) When using high-dimensional data containing many features, the inverse probability weights often take excessively large values, making it difficult to obtain a model that estimates CATE with high accuracy even if the weighted mean error is minimized using the inverse probability weight values ​​as is, as in the techniques of Non-Patent Documents 2 and 3. Therefore, in the present embodiment, a correction model is proposed for effectively correcting the inverse probability weight values, and by minimizing an objective function using the correction model, it is possible to obtain a model that estimates CATE with high accuracy.

[0018] That is, in this embodiment, a correction model of inverse probability weights is used in which large inverse probability weights are replaced with cumulative distribution function values ​​of the generalized Pareto distribution in the objective function for learning a model that estimates CATE with high accuracy. The technology according to this embodiment will be described in more detail below.

[0019] In the following description, for convenience of description, bold letters are not used for vectors in the text of the specification, but it is clear from the context that they indicate vectors. Also, for convenience of description, the symbol T is used to represent transposition in the text of the specification. Also, for convenience of description, the symbol that precedes a letter is placed at the top left of the letter. "^P" is an example.

[0020] (Basic processing content for learning a model for estimating CATE) In this embodiment, a binary variable A∈{0, 1} is used to represent one of two treatment methods, and a random variable X=[X １ , ..., X ｄ ] Ｔ , and a variable Y is used to represent the result. Here, each feature X in the variable X is ｍ (m∈{1, ..., d}) is assumed to take either a continuous value or a discrete value, and the variable Y is assumed to take either a continuous value or a discrete value. In this embodiment, the following data consisting of observed values ​​of these variables for n individuals is used as observed data. The observed data is used for model training.

[0021] Using the above variables, the potential outcome of a treatment is expressed as Y ０ , the potential outcome of no treatment is Y １ Then, the treatment effect for each individual is the difference between the potential outcomes, Y １ -Y ０ However, the outcome variable Y is defined as Y = (1-A)Y depending on the value of the treatment variable A (1 or 0). ０ +AY １ Therefore, the outcome variable Y is defined as the potential outcome Y ０ and potential outcome Y １ Because the treatment effect represents one of the potential outcomes, it is not possible to calculate the treatment effect for each individual.

[0022] Therefore, in this embodiment, we estimate CATE, which is an evaluation index of the magnitude of the treatment effect. CATE is a conditional expectation value defined as follows:

[0023] To train a model to estimate this conditional expectation, we assign weights w ｉ ∈R, using the function w(a, x), ｉ :=w(a ｉ , x ｉ The method for formulating the weighting function w differs depending on the estimation technique, but for example, the following function w(a, x) can be used based on the technique of Non-Patent Document 3. The weight expressed by the function w(a, x) corresponds to the inverse probability weight.

[0024] In the above formula (2), ^P(A=a) (a∈{0,1}) represents the empirical distribution, and Δ(x) represents the correlation coefficient from feature x to h given by the deep learning model (multilayer perceptron model) described later. Δ is a function for obtaining a ∑-dimensional feature vector, and π(A = a|Δ(x)) represents a model called a propensity score model that expresses the conditional probability P(A = a|Δ(x)) that an individual with feature vector Δ(x) receives treatment A = a. For example, in the technology of Non-Patent Document 3, it is given by the following logistic regression model.

[0025] In the above formula (3), α∈R ｈΔ×１ , β∈R are the parameters of the logistic regression model.

[0026] The weight w in equation (2) ｉ :=w(a ｉ , x ｉ ), the technology of Non-Patent Document 3 simultaneously learns the models shown in 1., 2., and 3. below.

[0027] 1. A multilayer perceptron model Γ(x), Δ(x), and Y(x) for computing a feature vector from feature x. 2. A potential outcome Y. ０ and Y １ Multilayer perceptron model h to predict ０ and h １ 3. π in equation (3), and feature representation Φ(x):=[Δ(x), Δ(x)] Ｔ Similarly, π to obtain the probability that A = a ０ Specifically, to simultaneously train the above models, the following objective function is minimized:

[0028] In equation (4), l is a function that measures the prediction error, and MMD is a distribution distance function called kernel maximum mean discrepancy (MMD). MMD is described in reference 1: Arthur Gretton, Karsten M. Borgwardt, Malte J. Rasch, Bernhard Schölkopf, and Alexander Smola. "A Kernel Two-Sample Test". Journal of Machine Learning Research, 13(25):723-773, 2012.

[0029] The above-mentioned parameter W ０ , b ０ Logistic regression model π ０ is as follows:

[0030] Furthermore, Ω in equation (4) is a penalty function that penalizes the complexity of the model.

[0031] In this embodiment, we propose a correction model for inverse probability weighting, which is applicable to the treatment effect estimation method based on inverse probability weighting, as a technology not found in the prior art. The correction model will be described in detail below.

[0032] (Regarding the inverse probability weight correction model) In this embodiment, of the weight values ​​assigned to each individual based on the weight function w(a, x), the top M weights are corrected. Specifically, a generalized Pareto distribution is considered in which the parameters are estimated using the top M weight values, and the top M weight values ​​are replaced with the function value of the cumulative distribution function of that Pareto distribution. The value of M is given by the following formula based on the value of the data sample size n:

[0033] however,

[0034] represents the floor function, which is a function that returns the largest integer less than or equal to a real number x.

[0035] The cumulative distribution function F(w) of the generalized Pareto distribution is defined as follows:

[0036] In the above equation (5), μ∈R, σ>0, and ξ∈R represent the location parameter, scale parameter, and shape parameter of the generalized Pareto distribution, respectively. The location parameter μ is the weight vector w sorted in ascending order. ＊ The M-th value from the top among the component values ​​of , i.e., is given by the following equation (6).

[0037] There are several methods for estimating the scale parameter σ and the shape parameter ξ. Any of the methods may be used, but here we will explain the simplest method, which uses the empirical mean and empirical variance. The (empirical) mean and (empirical) variance of the top M weight values ​​are calculated as ^m ｗ , ^S ２ ｗAssuming that these follow the generalized Pareto distribution, the following equations (7) and (8) are established from the formulas for the mean and variance of the generalized Pareto distribution.

[0038]

[0039] When these two equations are considered as simultaneous equations for the unknown parameters σ and ξ and are solved, the following equations (9) and (10) are obtained.

[0040]

[0041] The inverse cumulative distribution function F is calculated using the distribution parameter values ​​^μ, ^σ, and ^ξ obtained by equations (6), (9), and (10). －１ (α):=inf{wεR|F(w)≧α}(αε[0,1]) is obtained as the following equation (11).

[0042] Next, the cumulative distribution function ^F of Equation (11) corresponding to the top M weights is calculated. －１ To obtain the input value for , calculate the rank of each weight. Specifically, weight w = [w １ ,...,w ｎ ] Ｔ Ranking of each value r = [r １ , ..., r ｎ ] Ｔ is calculated by the function r = r(w). For example, w ３ ≧w １ ≧w ２ w = [w １ , w ２ , w ３ ] Ｔ If you input r(w) = [2, 3, 1] Ｔ The reason for this is that the first component is the second largest, the second component is the third largest, and the third component is the largest. Here, the output vector r of the function r cannot be differentiated with respect to the input variable w, and therefore cannot be used in a deep learning model. For example, in the above example, the first component r of the output vector r is １ is the first component w of the input w １ By changing the value by , we can see that it is a piecewise constant function expressed by the following equation.

[0043] Such piecewise constant functions have derivatives that are always zero or undefined, and therefore cannot be learned using backpropagation, a method commonly used in deep learning. To circumvent this problem, we use the technique described in Reference 2, "Mathieu Blondel, Olivier Teboul, Quentin Berthet, and Josip Djolonga. "Fast Differentiable Sorting and Ranking". Proceedings of the 37th International Conference on Machine Learning (ICML), PMLR 119, 2020."

[0044] That is, a everywhere differentiable function r ε (w), and instead of the function r(w), use the function r ε Using (w), r = r ε (w) where the weight w is the ranking r = [r １ , ..., r ｎ ] is calculated.

[0045] In this way, we estimate the parameters of the generalized Pareto distribution, calculate the ranking r of each weight, and then calculate the i-th weight w ｉ Consider replacing with the weights shown in the following equation (12):

[0046] In the above equation (12), I(·) represents an indicator function that returns 1 if · is true and 0 if · is false, and ρ(r)=min(max(r ｉ , 1 / 2), M+(1 / 2)). Here, since equation (12) contains the indicator function I(·), it cannot be differentiated. Therefore, the indicator function I(·) is approximated as follows using a hyperbolic tangent function (tanh function):

[0047] In equation (13), k∈R is a hyperparameter that determines the smoothness of the hyperbolic tangent function.

[0048] Furthermore, the indicator function I(·) may be approximated using a sigmoid function, as in the following equation (14).

[0049]

[0050] Next, as an example, the configuration and operation of a learning device 100 that applies the above-described inverse probability weight correction model will be described. Note that, as described above, the inverse probability weight correction model is a model that calculates weights after correcting the top M weights, and this model is a neural network model that can be trained by backpropagation.

[0051] In this example, it is assumed that the above-mentioned inverse probability weight correction model is applied to the treatment effect estimation technique of Non-Patent Document 3. In this case, by using the inverse probability weight correction model, the weight w of the objective function in Equation (4) can be calculated. ｉ :=w(a ｉ , x ｉ ) into the corrected weights of Eq. (12) ～ w ｉ Then, we can minimize the objective function in equation (4). This allows us to obtain a model that estimates the treatment effect with higher accuracy.

[0052] In this embodiment, the objective function includes the correction model of the inverse probability weight. The correction model of the inverse probability weight determines the weight w ｉ , the corrected weights ～ w ｉ can be replaced with

[0053] Note that obtaining a model by minimizing an objective function is itself an existing technique, as described in, for example, Non-Patent Document 3. The obtained model is a machine learning model including the above-mentioned multilayer perceptron model and logistic regression model.

[0054] Fig. 1 shows an example configuration of a learning device 100. As shown in Fig. 1, the learning device 100 includes an input unit 110, a data storage unit 120, a parameter storage unit 130, a learning unit 140, and an output unit 150. Fig. 2 is a flowchart showing the operation procedure of the learning device 100. The operation of the learning device 100 will be described in accordance with the procedure shown in Fig. 2.

[0055] <S1 (Step 1): Input> The input unit 110 receives observation data and hyperparameter values. The observation data is training data used to minimize the objective function. The observation data is stored in the data storage unit 120, and the hyperparameters are stored in the parameter storage unit 130.

[0056] The hyperparameters include hyperparameters related to the inverse probability weight correction model and hyperparameters related to the objective function of equation (4).

[0057] The hyperparameter for the inverse probability weight correction model is, for example, the hyperparameter ε described in Reference 2. The hyperparameter for the objective function of Equation (4) is, for example, the regularization parameter in the objective function (λ in the case of Equation (4)). ＭＭＤ , λ ｃｅ , λ ｒｅｇ )

[0058] <S2: Learning> The learning unit 140 reads the observation data (learning data) from the data storage unit 120, reads the hyperparameters from the parameter storage unit 130, and minimizes the objective function of equation (4) using the observation data and the hyperparameters to obtain a trained model (specifically, optimized parameters that constitute the model). Backpropagation (error backpropagation) is used for learning (minimizing the objective function of equation (4)).

[0059] As described above, the objective function of this embodiment is not the objective function of equation (4) itself, but the weight w ｉ :=w(a ｉ , x ｉ ) into the corrected weights of Eq. (12) ～ w ｉ In addition, I(·) in equation (12) is approximated using equation (13), which enables learning using backpropagation.

[0060] <S3: Output> The output unit 150 outputs the trained model obtained by the training unit 140. The trained model is input to, for example, an estimation device that estimates a treatment effect, and is used for estimation by the estimation device. Note that the training device 100 may also serve as the estimation device.

[0061] When the estimation device uses a trained model to estimate the treatment effect of an individual who has received treatment, the model outputs the treatment effect (specifically, CATE) by inputting the characteristics of the individual into the model.

[0062] (Hardware Configuration Example) Any of the devices (learning device 100, estimation device) described in this embodiment can be realized, for example, by causing a computer to execute a program. This computer may be a physical computer or a virtual machine on the cloud.

[0063] That is, the device can be realized by executing a program corresponding to the processing performed by the device using hardware resources such as a CPU and memory built into a computer. The program can be recorded on a computer-readable recording medium (such as a portable memory) and stored or distributed. The program can also be provided via a network such as the Internet or email.

[0064] Fig. 3 is a diagram showing an example of the hardware configuration of the computer. The computer in Fig. 3 includes a drive device 1000, an auxiliary storage device 1002, a memory device 1003, a CPU 1004, an interface device 1005, a display device 1006, an input device 1007, an output device 1008, and the like, all of which are interconnected via a bus B. The computer may further include a GPU.

[0065] The program that realizes the processing on the computer is provided by a recording medium 1001, such as a CD-ROM or a memory card. When the recording medium 1001 storing the program is set in the drive device 1000, the program is installed from the recording medium 1001 to the auxiliary storage device 1002 via the drive device 1000. However, the program does not necessarily have to be installed from the recording medium 1001, but may be downloaded from another computer via a network. The auxiliary storage device 1002 stores the installed program as well as necessary files, data, etc.

[0066] The memory device 1003 reads and stores a program from the auxiliary storage device 1002 when an instruction to start the program is received. The CPU 1004 realizes functions related to the device in accordance with the program stored in the memory device 1003. The interface device 1005 is used as an interface for connecting to a network, etc. The display device 1006 displays a GUI (Graphical User Interface) or the like according to the program. The input device 1007 is composed of a keyboard, mouse, buttons, a touch panel, etc., and is used to input various operation instructions. The output device 1008 outputs the results of calculations.

[0067] (Effects of the embodiment) According to the technology of the present embodiment, an objective function for learning a model for estimating the treatment effect is constructed using a correction model that effectively corrects the value of the inverse probability weight for estimating the magnitude of the treatment effect, so that a model that estimates the treatment effect with higher accuracy can be obtained.

[0068] The following additional notes are provided regarding the above-described embodiments.

[0069] <Additional Notes> (Additional Item 1) A learning device that learns a model for estimating treatment effects, comprising: a learning unit that learns the model by minimizing an objective function using inverse probability weights, wherein the objective function has an inverse probability weight correction model that replaces a predetermined number of inverse probability weight values, among multiple inverse probability weight values ​​assigned to multiple individuals, with another value. (Additional Item 2) The learning device according to Additional Item 1, wherein the another value is a cumulative distribution function value of a generalized Pareto distribution. (Additional Item 3) The learning device according to Additional Item 1 or 2, wherein the correction model includes an indicator function that returns 1 or 0 depending on the ranking of the magnitudes of the inverse probability weights, and the indicator function is approximated using a hyperbolic tangent function or a sigmoid function. (Supplementary Item 4) A learning method executed by a learning device that learns a model for estimating treatment effects, comprising a learning step of learning the model by minimizing an objective function using inverse probability weights, wherein the objective function has an inverse probability weight correction model that replaces a predetermined number of inverse probability weight values, among multiple inverse probability weight values ​​assigned to multiple individuals, with different values.

[0070] Although the present embodiment has been described above, the present invention is not limited to such a specific embodiment, and various modifications and changes are possible within the scope of the gist of the present invention described in the claims.

[0071] REFERENCE SIGNS LIST 100 Learning device 110 Input unit 120 Data storage unit 130 Parameter storage unit 140 Learning unit 150 Output unit 1000 Drive device 1001 Recording medium 1002 Auxiliary storage device 1003 Memory device 1004 CPU 1005 Interface device 1006 Display device 1007 Input device 1008 Output device

Claims

1. A learning device that learns a model for estimating a treatment effect, comprising a learning unit that learns the model by minimizing an objective function using inverse probability weights, wherein the objective function has a correction model for inverse probability weights that replaces the values of the top predetermined number of inverse probability weights among the values of the plurality of inverse probability weights assigned to a plurality of individuals with another value.

2. The learning device according to claim 1, wherein the other value is a cumulative distribution function value of a generalized Pareto distribution.

3. The learning device according to claim 1 or 2, wherein the correction model includes an indicator function that returns 1 or 0 according to the rank of the magnitude of the inverse probability weights, and the indicator function is approximated using a hyperbolic tangent function or a sigmoid function.

4. A learning method executed by a learning device that learns a model for estimating a treatment effect, comprising a learning step of learning the model by minimizing an objective function using inverse probability weights, wherein the objective function has a correction model for inverse probability weights that replaces the values of the top predetermined number of inverse probability weights among the values of the plurality of inverse probability weights assigned to a plurality of individuals with another value.