Meta-learning method, meta-learning device, and program

Abstract

In a meta-learning method according to one embodiment of the present disclosure, a computer executes: a first input procedure for inputting a dataset for learning a plurality of tasks; a selection procedure for selecting one task from the plurality of tasks; a creation procedure for creating a support set configured from at least some data for learning from the dataset for learning the one task, and a query set configured from a feature quantity included in the at least some data for learning; a first estimation procedure for estimating a first parameter that represents a per-task parameter from among parameters in each of a plurality of machine learning models so that the first parameter is compatible with the support set; and a learning procedure for learning a second parameter that represents a task-shared parameter from among the parameters in each of the plurality of machine learning models so that the error between a predicted value that is predicted from a first feature quantity included in the query set by combining the plurality of machine learning models, and training data for the predicted value, is small.

Description

[Technical Field]

[0001] The present disclosure relates to a meta-learning method, a meta-learning device, and a program. [Background technology]

[0002] In machine learning methods, model parameters are usually learned using task-specific training datasets. To achieve high performance on a target task, a large amount of task-specific training data is required. However, depending on the task, preparing a sufficient amount of training data can be costly.

[0003] To solve the above problems, meta-learning methods have been proposed that utilize training data for different tasks to achieve high performance in a target task even with a small amount of training data (see, for example, Non-Patent Document 1).However, the existing meta-learning methods have a problem in that they cannot be applied to prediction methods that combine multiple machine learning methods.

[0004] Related technologies that have been proposed include a technology that combines multiple regression models to estimate causal effects (see, for example, Non-Patent Document 2) and a technology that estimates uncertainty using a regression model and subsequent calibration (see, for example, Non-Patent Document 3). [Prior art documents] [Non-patent literature]

[0005] [Non-Patent Document 1] Finn, Chelsea, Pieter Abbeel, and Sergey Levine. "Model-agnostic meta-learning for fast adaptation of deep networks." Proceedings of the 34th International Conference on Machine Learning, 2017. [Non-patent document 2] Curth, Alicia, and Mihaela van der Schaar. "Nonparametric estimation of heterogeneous treatment effects: From theory to learning algorithms." International Conference on Artificial Intelligence and Statistics. 2021. [Non-patent document 3] Marx, Charles, et al. "Modular Conformal Calibration." International Conference on Machine Learning. 2022. Summary of the Invention [Problem to be solved by the invention]

[0006] The present disclosure has been made in consideration of the above points, and aims to provide a technology for meta-learning model parameters of a method that combines multiple machine learning methods to make predictions. [Means for solving the problem]

[0007] A meta-learning method according to one aspect of the present disclosure includes a computer-implemented method including: a first input step of inputting training datasets for multiple tasks; a selection step of selecting one task from the multiple tasks; a creation step of creating a support set consisting of at least a portion of the training data in the training dataset for the one task and a query set consisting of features included in at least a portion of the training data in the training dataset; a first estimation step of estimating a first parameter representing a parameter for each task from among parameters possessed by each of multiple machine learning models so as to conform to the support set; and a learning step of learning a second parameter representing a parameter common to the tasks from among parameters possessed by each of the multiple machine learning models so as to reduce an error between a predicted value predicted from the first feature included in the query set by combining the multiple machine learning models and training data for the predicted value. [Effects of the Invention]

[0008] A technology is provided for meta-learning model parameters of a prediction method that combines multiple machine learning methods. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram illustrating an example of the hardware configuration of a meta-learning device according to an embodiment of the present invention. [Figure 2] FIG. 1 is a diagram illustrating an example of the functional configuration of a meta-learning device according to an embodiment of the present invention. [Figure 3] 10 is a flowchart illustrating an example of a meta-learning process according to the present embodiment. [Figure 4] 10 is a flowchart illustrating an example of a prediction process according to the present embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0010] An embodiment of the present invention will be described below. In the following embodiment, a meta-learning device 10 is described that, when given a training dataset for multiple tasks, performs meta-learning to generate model parameters for a prediction method that combines multiple machine learning methods. A case where the meta-learning device 10 predicts a target task using the model parameters learned by this meta-learning is also described.

[0011] In the following, as an example, a method for estimating (predicting) a causal effect by combining multiple machine learning methods will be described. However, this is just an example, and the following embodiment can be similarly applied to other problems, such as estimating (predicting) uncertainty, as long as the method involves combining multiple machine learning methods for prediction.

[0012] Here, the meta-learning device 10 has a "meta-learning time" during which it meta-learns model parameters common to multiple tasks (hereinafter also referred to as task-common parameters), and a "prediction time" during which it estimates causal effects in a target task using the task-common parameters learned during the meta-learning time. Note that the "meta-learning time" may simply be called "learning time," for example. The "prediction time" may also be called "inference time," "test time," or the like.

[0013] During meta-learning, the meta-learning device 10 receives the learning data set for the t-th task as D t ={(x tn ,a tn ,y tn )|n=1, ,N t}, and a set of training datasets for T tasks is D={D t |t=1, ,T} is given, where x tn is the nth feature of the tth task, a tn is the nth treatment of the tth task, y tn represents the nth result of the tth task. t is the training dataset D for the t-th task tHereinafter, the set of training datasets for T tasks will be referred to as the "meta-learning dataset." The tth task will also be referred to as "task t."

[0014] At the time of prediction, the meta-learning device 10 receives a data set S′={(x n S' ,a n S' ,y n S' )|n=1, ,N S'} is given, where x n S' is the nth feature of the target task, a n S' is the nth action of the target task, y n S' represents the nth result of the target task. S' represents the number of data included in dataset S'. In this case, when a feature x' of the target task is given (i.e., when a feature x' whose procedure and outcome are unknown is given), the goal is to estimate the causal effect of that feature x'. Hereinafter, dataset S' will also be referred to as the "support set S'".

[0015] In the following, it is assumed that the treatment takes on a binary value of 0 or 1 (for example, 1 if the treatment was performed, and 0 if the treatment was not performed), but it may take on multiple values. Also, it is assumed that the feature amount is a vector representing the characteristics of the object that is the cause of the causal relationship, but the format is not limited to a vector and may be, for example, an image or a graph.

[0016] In the following embodiment, to estimate causal effects, three models (a treatment probability estimation model, an outcome estimation model, and a causal effect estimation model) are used, each realized by a different machine learning method and having task-common parameters and task-specific model parameters (hereinafter also referred to as task-specific parameters). The task-specific parameters of these three models are estimated to fit a certain dataset (a support set S, described below, during meta-learning, and a support set S' during prediction). Meanwhile, the task-common parameters are estimated (learned) to improve the accuracy of estimating causal effects during meta-learning. Then, during prediction, the task-specific parameters are estimated to fit the support set S' using the three models with the learned task-common parameters and the support set S', and the causal effect of the feature x' of the target task is estimated using the causal effect estimation model.

[0017] In the following embodiment, the meta-learning and prediction processes are performed by the same meta-learning device 10, but the meta-learning and prediction processes may be performed by different devices. In this case, the device that performs the prediction process may be called, for example, a "prediction device" or an "inference device."

[0018] <Example of hardware configuration of meta-learning device 10> An example of the hardware configuration of a meta-learning device 10 according to this embodiment is shown in Figure 1. As shown in Figure 1, the meta-learning device 10 according to this embodiment is realized by the hardware configuration of a general computer or computer system, and includes, for example, an input device 101, a display device 102, an external I / F 103, a communication I / F 104, a RAM (Random Access Memory) 105, a ROM (Read Only Memory) 106, an auxiliary storage device 107, and a processor 108. Furthermore, each of these pieces of hardware is connected to each other via a bus 109 so that they can communicate with each other.

[0019] The input device 101 is, for example, a keyboard, a mouse, a touch panel, a physical button, etc. The display device 102 is, for example, a display, a display panel, etc. Note that the meta-learning device 10 does not necessarily have to have at least one of the input device 101 and the display device 102, for example.

[0020] The external I / F 103 is an interface with an external device such as a recording medium 103a. The meta-learning device 10 can read from and write to the recording medium 103a via the external I / F 103. Examples of the recording medium 103a include a flexible disk, a CD (Compact Disc), a DVD (Digital Versatile Disk), an SD memory card (Secure Digital memory card), and a USB (Universal Serial Bus) memory card.

[0021] The communication I / F 104 is an interface for connecting the meta-learning device 10 to a communication network or the like. The RAM 105 is a volatile semiconductor memory (storage device) that temporarily stores programs and data. The ROM 106 is a non-volatile semiconductor memory (storage device) that can store programs and data even when the power is turned off. The auxiliary storage device 107 is a storage device (storage device) such as an HDD (Hard Disk Drive), SSD (Solid State Drive), or flash memory. The processor 108 is an arithmetic device such as a CPU (Central Processing Unit) or GPU (Graphics Processing Unit).

[0022] The meta-learning device 10 according to this embodiment has the hardware configuration shown in Figure 1, and is therefore capable of performing various processes described below. Note that the hardware configuration shown in Figure 1 is merely an example, and the hardware configuration of the meta-learning device 10 is not limited to this. For example, the meta-learning device 10 may have multiple auxiliary storage devices 107 or multiple processors 108, may not have some of the hardware shown in the figure, or may have various hardware components other than the hardware shown in the figure.

[0023] <Example of functional configuration of meta-learning device 10> An example of the functional configuration of the meta-learning device 10 according to this embodiment is shown in Figure 2. As shown in Figure 2, the meta-learning device 10 according to this embodiment includes an input unit 201, a pseudo-causal effect estimation unit 202, a meta-learning unit 203, a treatment probability estimation unit 204, an outcome estimation unit 205, a pseudo-outcome estimation unit 206, a causal effect estimation unit 207, and an output unit 208. Each of these units is realized, for example, by a processor 108 or the like executing one or more programs installed in the meta-learning device 10. The meta-learning device 10 according to this embodiment also includes a memory unit 209. The memory unit 209 may be realized, for example, by a storage device such as a database server connected to the meta-learning device 10 via a communication network or the like.

[0024] During meta-learning, the input unit 201 inputs a given meta-learning dataset D. During prediction, the input unit 201 inputs a given support set S' and a feature x' whose treatment and result are unknown.

[0025] During meta-learning, the pseudo-causal effect estimation unit 202 uses the learning data of each task to estimate a causal effect by an existing causal effect estimation method. Hereinafter, the causal effect estimated by the pseudo-causal effect estimation unit 202 will be referred to as a "pseudo-causal effect." In addition, the n-th pseudo-causal effect of the t-th task will be

[0026]

number

[0027] The pseudo-causal effect estimation unit 202 may estimate the pseudo-causal effect using any existing causal effect estimation method, for example, a DR-learner (doubly robust learner) or an RA-learner (regression adjustment learner) described in Non-Patent Document 2 or the like.

[0028] During meta-learning, the meta-learning unit 203 randomly selects one task t from the first task to the Tth task, and then generates a learning dataset D for the selected task t. t Support Set S t and a query set Q t Create a.

[0029] More specifically, first, the meta-learning unit 203 randomly selects one t∈{1, ,T}. Next, the meta-learning unit 203 selects a learning dataset D for task t. t Randomly select a random number of training data from , and renumber the training data to create a support set S t ={(x tn S ,a tn S ,y tn S )|n=1, ,N t S}. The meta-learning unit 203 also creates a learning dataset D t Randomly select a random number of training data from the list, and then use the feature x tn and its pseudo-causal effect ~ τ tnRenumbering the numbers in the query set Q t ={(x tn q , ~ τ tn q )|n=1, ,N t q} where N t S Support Set S t The number of data included in t q is a query set Q t is the number of data included in

[0030] In the following, for simplicity, we will omit the task "t" and define the support set as S={(x n S ,a n S ,y n S )|n=1, ,N S}, the query set is Q={(x n q , ~ τ n q )|n=1, ,N q} can also be written as

[0031] Furthermore, during meta-learning, the meta-learning unit 203 calculates the feature quantity x estimated by the causal effect estimation model when the support set S is fitted to three models (treatment probability estimation model, outcome estimation model, and causal effect estimation model). n q Causal effect (causal effect estimate) and pseudo-causal effect ~ τ n q The task-common parameters are updated using an optimization method such as the stochastic gradient method so that the feature x n q Causal effect estimates and pseudo-causal effects of ~ τ n q As a measure for measuring the closeness (distance) between, for example, square error or mean square error can be used.

[0032] The treatment probability estimation unit 204 is realized by a treatment probability estimation model. The treatment probability estimation unit 204 calculates the feature quantity x when the treatment probability estimation model is adapted to the support set S during meta-learning. n S Furthermore, the treatment probability estimation unit 204 estimates the probability of treatment for the feature quantity x when the treatment probability estimation model is adapted to the support set S′ at the time of prediction. n S' Estimate the treatment probability for .

[0033] The task-specific parameters of the treatment probability estimation model are estimated so as to increase the accuracy of treatment probability estimation in the support set (support set S in meta-learning, support set S' in prediction). For example, the treatment probability estimation model can be the model shown in the following equation (1).

[0034]

number

[0035] In the case of the treatment probability estimation model shown in the above formula (1), the task-specific parameters can be calculated simply by averaging the feature quantities, which enables efficient adaptation to the support set. However, using the model shown in the above formula (1) as the treatment probability estimation model is just one example, and other models such as a feedforward neural network can also be used as the treatment probability estimation model.

[0036] The result estimation unit 205 is realized by a result estimation model. The result estimation unit 205 calculates the feature quantity x when the result estimation model is adapted to the support set S during meta-learning. n S and the outcome for the action a∈{0, 1}. In addition, the outcome estimation unit 205 estimates the feature quantity x n S' and estimate the outcome for treatment a∈{0,1}.

[0037] The task-specific parameters of the outcome estimation model are estimated to increase the accuracy of outcome estimation in the support set (support set S for meta-learning, support set S' for prediction). For example, the model shown in the following equation (2) can be used as the outcome estimation model.

[0038]

number

[0039] In the case of the result estimation model shown in the above formula (2), since the task-specific parameters are a linear model, the optimal task-specific parameters that fit the support set can be analytically determined using the least squares method. This enables efficient fitting to the support set. However, using the model shown in the above formula (2) as the result estimation model is only an example, and it is also possible to use, for example, a Gaussian process instead of a linear model. It is also possible to use other models, such as a feedforward neural network, as the result estimation model.

[0040] During meta-learning and prediction, the pseudo-result estimation unit 206 uses the treatment probability estimated by the treatment probability estimation unit 204, the result estimated by the result estimation unit 205, and a support set (support set S during meta-learning, and support set S' during prediction) to estimate a result (hereinafter also referred to as a pseudo-result) for the feature quantities of the support set. At this time, the pseudo-result estimation unit 206 estimates the pseudo-result so that the expected value of the pseudo-result and the expected value of the causal effect are close to each other. For example, during meta-learning, the pseudo-result estimation unit 206 can estimate the pseudo-result using the following equation (3):

[0041]

number

[0042] In the above equation (3), the treatment probability estimation model π(·) and the outcome estimation model μ aBoth the treatment probability estimation model π(·) and the outcome estimation model μ (·) were used, but either one may be used. a Even when using only one of (·), the pseudo-results should be estimated so that the expected values ​​of the pseudo-results and the expected values ​​of the causal effects are close to each other.

[0043] The causal effect estimation unit 207 is realized by a causal effect estimation model. During meta-learning, the causal effect estimation unit 207 uses the pseudo result estimated by the pseudo result estimation unit 206 to calculate the feature quantity x when the causal effect estimation model is adapted to the support set S. n q Furthermore, at the time of prediction, the causal effect estimation unit 207 uses the pseudo result estimated by the pseudo result estimation unit 206 to estimate the causal effect of the feature x' when the causal effect estimation model is adapted to the support set S'.

[0044] The task-specific parameters of the causal effect estimation model are the features (feature x n S , and when predicting, feature x n S' ) is estimated so that it is close to the causal effect (causal effect estimate) of the feature estimated by the causal effect estimation model. For example, the model shown in the following formula (4) can be used as the causal effect estimation model.

[0045]

number

[0046] In the case of the causal effect estimation model shown in the above formula (4), since the task-specific parameters are a linear model, the optimal task-specific parameters that fit the support set can be analytically determined using the least squares method. This enables efficient fitting to the support set. However, using the model shown in the above formula (4) as the causal effect estimation model is only one example, and it is also possible to use, for example, a Gaussian process instead of a linear model. It is also possible to use other models, such as a feedforward neural network, as the causal effect estimation model.

[0047] At the time of prediction, the output unit 208 outputs the causal effect of the feature x' estimated by the causal effect estimation unit 207 to a predetermined output destination. Note that examples of the predetermined output destination include the storage unit 209, the display device 102 such as a display, and other devices connected via a communication network.

[0048] The storage unit 209 stores various data (for example, a meta-learning dataset D, a support set S' for a target task, feature quantities x' for a target task, task-specific parameters, task-common parameters, etc.).

[0049] <Meta-learning processing> An example of the meta-learning process according to this embodiment will be described below with reference to FIG.

[0050] First, the input unit 201 inputs a given meta-learning dataset D (step S101).

[0051] Next, the pseudo causal effect estimation unit 202 calculates the learning data (x tn ,a tn ,y tn ) and then use existing causal effect estimation methods to estimate the training data (x tn ,a tn ,y tn ) pseudo-causal effect ~ τ tn (Step S102). This allows us to estimate the set of pseudo-causal effects {{~ τ tn |n=1, ,N t}|t=1,···,T} is obtained.

[0052] Next, the meta-learning unit 203 randomly selects one task t from the first task to the Tth task, and then generates a learning dataset D for the selected task t. t From the support set S={(x n S ,a n S ,y n S )|n=1, ,N S} and a query set Q={(x n q , ~ τ n q )|n=1, ,N q} is created (step S103).

[0053] Next, the treatment probability estimation unit 204 calculates the feature quantity x when the treatment probability estimation model is adapted to the support set S. n S (Step S104). That is, the treatment probability estimation unit 204 first estimates task-specific parameters of the treatment probability estimation model, and then estimates the treatment probability for the feature quantity x n S Estimate the treatment probability for .

[0054] For example, when the treatment probability estimation model shown in the above formula (1) is used, the treatment probability estimation unit 204 first calculates a n S = 0 for feature x n S The mean vector of θ p0 and a n S = 1 for feature x n S The mean vector of θ p1 Next, the treatment probability estimation unit 204 calculates the treatment probability π(x nS ) is estimated. This gives the set of treatment probabilities {π(x n S )|n=1, ,N S} is obtained.

[0055] Next, the result estimation unit 205 calculates the feature quantity x when the result estimation model is adapted to the support set S. n S and estimates the result for the action a∈{0, 1} (step S105). That is, the result estimation unit 205 first estimates the task-specific parameters of the result estimation model, and then estimates the feature x n S and estimate the outcome for treatment a∈{0,1}.

[0056] For example, when using the result estimation model shown in the above formula (2), the result estimation unit 205 first calculates a←a n S For μ a (x n S ) and the corresponding result y n S The task-specific parameter θ a This can be analytically determined by, for example, the least squares method. Next, the result estimation unit 205 estimates the result μ a (x n S ) is estimated. This gives the resulting set {{μ a (x n S )|n=1, ,N S}|a=0,1} is obtained.

[0057] Next, the pseudo result estimation unit 206 uses the treatment probability estimated in step S104, the result estimated in step S105, and the support set S to calculate the feature x n S A pseudo result for is estimated (step S106).

[0058] For example, the pseudo-outcome estimator 206 may calculate the treatment probability π(x n S ) and the result μ0(x n S ) and μ1(x n S ) and (x n S ,a n S ,y n S ) and the pseudo result is calculated using the above formula (3). ~ y n S This will generate a set of pseudo results { ~ y n S |n=1, ,N S} is obtained.

[0059] Next, the causal effect estimation unit 207 calculates the feature quantity x when the causal effect estimation model is adapted to the support set S. n q That is, the causal effect estimation unit 207 first estimates task-specific parameters of the causal effect estimation model, and then estimates the causal effect of the feature quantity x n q Estimate the causal effect of

[0060] For example, when the causal effect estimation model shown in the above formula (4) is used, the causal effect estimation unit 207 first calculates the causal effect estimate τ(x n S ) and the corresponding pseudo-results ~ y n S The task-specific parameter θ y This can be analytically determined by, for example, the least squares method. Next, the causal effect estimation unit 207 estimates the causal effect estimate value τ(x n q ) is estimated. This gives us the set of causal effect estimates {τ(x n q )|n=1, ,Nq} is obtained.

[0061] Next, the meta-learning unit 203 updates the task-common parameters so that the causal effect estimate estimated in step S107 above and the corresponding pseudo-causal effect become closer to each other (step S108).

[0062] For example, the meta-learning unit 203 calculates the causal effect estimate τ(x n q ) and the corresponding pseudo-causal effects ~ τ n q The task-common parameter φ is set so that p ,φ a ,φ y This can be achieved by using an optimization method such as a gradient method.

[0063] Next, the meta-learning unit 203 determines whether a predetermined termination condition is met (step S109).

[0064] If it is determined in step S109 above that the termination condition is not satisfied, the meta-learning unit 203 returns to step S103. As a result, steps S103 to S108 are repeatedly executed until the termination condition is satisfied. On the other hand, if it is determined in step S109 above that the termination condition is satisfied, the meta-learning unit 203 terminates the meta-learning process. As a result, learned task-common parameters are obtained.

[0065] The termination conditions may be, for example, "the update amount of the task common parameters is less than a predetermined threshold," "the number of repetitions of steps S103 to S108 is equal to or greater than a predetermined threshold," or "the decrease in the value of the objective function (e.g., the squared error or mean squared error between the causal effect estimate and the pseudo-causal effect) is less than a predetermined threshold."

[0066] <Prediction processing> An example of the prediction process according to this embodiment will be described below with reference to Fig. 4. In the following, it is assumed that the task-common parameters have already been learned.

[0067] First, the input unit 201 receives a given support set S'={(x n S' ,a n S' ,y n S' )|n=1, ,N S'} and the feature quantity x' whose action and result are unknown (step S201).

[0068] Next, the treatment probability estimation unit 204 calculates the feature quantity x when the treatment probability estimation model is adapted to the support set S′. n S' (Step S202). That is, the treatment probability estimation unit 204 first estimates task-specific parameters of the treatment probability estimation model, and then estimates the treatment probability for the feature quantity x n S' Estimate the treatment probability for .

[0069] For example, when the treatment probability estimation model shown in the above formula (1) is used, the treatment probability estimation unit 204 first calculates a n S' = 0 for feature x n S' The mean vector of θ p0 and a n S' Feature x where x = 1 n S' The mean vector of θ p1 Next, the treatment probability estimation unit 204 calculates the treatment probability π(x n S' ) is estimated. This gives the set of treatment probabilities {π(x n S' )|n=1, ,N S'} is obtained.

[0070] Next, the result estimation unit 205 calculates the feature quantity x when the result estimation model is adapted to the support set S′. n S' and estimates the result for the action a∈{0, 1} (step S203). That is, the result estimation unit 205 first estimates the task-specific parameters of the result estimation model, and then estimates the feature x n S' and estimate the outcome for treatment a∈{0,1}.

[0071] For example, when using the result estimation model shown in the above formula (2), the result estimation unit 205 first calculates a←a n S' For μ a (x n S' ) and the corresponding result y n S' The task-specific parameter θ a This can be analytically determined by, for example, the least squares method. Next, the result estimation unit 205 estimates the result μ a (x n S' ) is estimated. This gives the resulting set {{μ a (x n S' )|n=1, ,N S'}|a=0,1} is obtained.

[0072] Next, the pseudo result estimation unit 206 uses the treatment probability estimated in step S202, the result estimated in step S203, and the support set S′ to calculate the feature x n S' A pseudo result for is estimated (step S204).

[0073] For example, the pseudo-outcome estimator 206 may calculate the treatment probability π(x n S' ) and the result μ0(x n S' ) and μ1(x n S') and (x n S' ,a n S' ,y n S' ) and the pseudo result is calculated using the above formula (3). ~ y n S' This will generate a set of pseudo results { ~ y n S' |n=1, ,N S'} is obtained.

[0074] Next, the causal effect estimation unit 207 estimates the causal effect of the feature x' when the causal effect estimation model is adapted to the support set S' (step S205). That is, the causal effect estimation unit 207 first estimates task-specific parameters of the causal effect estimation model, and then estimates the causal effect of the feature x' using the causal effect estimation model having these task-specific parameters.

[0075] For example, when the causal effect estimation model shown in the above formula (4) is used, the causal effect estimation unit 207 first calculates the causal effect estimate τ(x n S' ) and the corresponding pseudo-results ~ y n S' The task-specific parameter θ y This can be analytically determined, for example, by the least squares method. Next, the causal effect estimation unit 207 estimates the causal effect estimate τ(x') using the causal effect estimation model shown in the above formula (4). This allows obtaining the causal effect estimate τ(x') of the feature x' whose treatment and outcome are unknown.

[0076] Then, the output unit 208 outputs the causal effect estimate τ(x′) estimated in the above step S205 to a predetermined output destination (step S206).

[0077] <Evaluation> To evaluate the meta-learning device 10 according to this embodiment, we compared it with existing methods using artificial data. As existing methods, we adopted the existing meta-learning method, multi-task learning method, and task-specific learning method.

[0078] The evaluation index used was the mean square error between the estimated causal effect and the correct answer. The evaluation results (mean and standard deviation) are shown in Table 1 below.

[0079] [Table 1] Here, the proposed method represents the meta-learning device 10 according to this embodiment.

[0080] As shown in Table 1 above, it can be seen that the meta-learning device 10 according to this embodiment achieves a lower error than existing methods.

[0081] <Summary> As described above, when given training datasets for multiple tasks, the meta-learning device 10 of this embodiment creates a support set S and a query set Q from the training datasets for each task, and then estimates task-specific parameters by fitting them to multiple models realized by multiple different machine learning methods. The meta-learning device 10 of this embodiment then estimates task-common parameters for each model so that the final prediction result is close to its pseudo-ground truth. This makes it possible to achieve high performance on a target task in a prediction method that combines multiple machine learning methods, even when only a small amount of training data for the target task is provided.

[0082] In the above embodiment, the task-specific parameters of the treatment probability estimation model are estimated by the treatment probability estimation unit 204, but the task-specific parameters of the treatment probability estimation model may be estimated by the meta-learning unit 203. Similarly, the task-specific parameters of the result estimation model are estimated by the result estimation unit 205, but the task-specific parameters of the result estimation model may be estimated by the meta-learning unit 203. Similarly, the task-specific parameters of the causal effect estimation model are estimated by the causal effect estimation unit 207, but the task-specific parameters of the causal effect estimation model may be estimated by the meta-learning unit 203.

[0083] The present invention is not limited to the above-described specifically disclosed embodiments, and various modifications, changes, and combinations with known technologies are possible without departing from the scope of the claims. [Explanation of symbols]

[0084] 10 Meta-learning device 101 Input Device 102 Display device 103 External I / F 103a Recording media 104 Communication I / F 105 RAM 106 ROM 107 Auxiliary storage 108 processors 109 Bus 201 Input section 202 Pseudo-causal effect estimation part 203 Meta-Learning Department 204 Treatment Probability Estimation Unit 205 Result estimation section 206 Pseudo result estimation part 207 Causal Effect Estimation Section 208 Output section 209 Memory section

Claims

1. a first input step of inputting training datasets for multiple tasks; a selection step of selecting one task from the plurality of tasks; a creation step of creating a support set consisting of at least a portion of training data in a training dataset for the one task, and a query set consisting of features included in at least a portion of the training data in the training dataset; a first estimation procedure for estimating first parameters representing parameters for each task among parameters of each of a plurality of machine learning models so as to conform to the support set; a learning procedure for learning a second parameter representing a task-common parameter among parameters possessed by each of the plurality of machine learning models, so that an error between a predicted value predicted from a first feature included in the query set by combining the plurality of machine learning models and training data for the predicted value is small; A meta-learning method performed by a computer.

2. a second input step of inputting a dataset consisting of data of a target task and a second feature quantity whose predicted value is to be predicted in the target task; a second estimation procedure for estimating the first parameter among parameters of each of the plurality of machine learning models so as to fit the dataset; a prediction step of predicting a predicted value from the second feature amount by combining the plurality of machine learning models; The meta-learning method of claim 1 , further comprising the steps of:

3. the learning data includes the first feature amount, a treatment for an object having a feature represented by the first feature amount, and a result of the object, in association with each other; 3. The meta-learning method according to claim 1, wherein the plurality of machine learning models include a treatment probability estimation model that receives input features and estimates the probability that a treatment has been performed, an outcome estimation model that receives input features and a treatment and estimates an outcome, and a causal effect estimation model that receives input features and estimates a causal effect as the predicted value.

4. The treatment probability estimation model includes the first parameter θ ｐ and the second parameter φ ｐ , the outcome estimation model includes the first parameter θ ａ and the second parameter φ ａ , the causal effect estimation model includes the first parameter θ ｙ and the second parameter φ ｙ Each has The first estimation procedure includes: The first parameter θ ｐ and the first parameter θ ａ and the first parameter θ ｙ and are estimated to fit the support set; The learning procedure includes: The second parameter φ is set so that the error is small. ｐ and the second parameter φ ａ and the second parameter φ ｙ The meta-learning method according to claim 3, wherein the method learns:

5. the computer further executes a pseudo-outcome estimation step of estimating a pseudo-outcome using a probability estimated by inputting the first feature into the treatment probability estimation model and a result estimated by inputting the first feature and a possible value of the treatment into the outcome estimation model; The first estimation procedure includes: The average of the first feature amounts associated with the same treatment is calculated as the first parameter θ ｐ It is estimated as The first parameter θ is adjusted so as to reduce an error between a result estimated by inputting the first feature amount and a treatment associated with the first feature amount into the result estimation model and a result associated with the first feature amount. ａ Estimate The first parameter θ is set so as to reduce an error between the causal effect estimated by inputting the first feature amount into the causal effect estimation model and the pseudo result. ｙ The meta-learning method of claim 4, wherein:

6. 6. The meta-learning method according to claim 5, wherein the computer further executes a pseudo-causal effect estimation procedure of estimating, as the training data, a pseudo-causal effect between the first feature included in the training data and the result by using a causal estimation method.

7. a first input unit for inputting training datasets for a plurality of tasks; a selection unit that selects one task from the plurality of tasks; a creation unit that creates a support set configured with at least a portion of training data of a training dataset for the one task, and a query set that is configured with features included in at least a portion of the training data of the training dataset; a first estimation unit that estimates first parameters representing parameters for each task among parameters possessed by each of a plurality of machine learning models so as to conform to the support set; a learning unit that learns a second parameter representing a task-common parameter among parameters possessed by each of the plurality of machine learning models so as to reduce an error between a predicted value predicted from a first feature included in the query set by combining the plurality of machine learning models and training data for the predicted value; A meta-learning device having

8. a first input step of inputting training datasets for multiple tasks; a selection step of selecting one task from the plurality of tasks; a creation step of creating a support set consisting of at least a portion of training data in a training dataset for the one task, and a query set consisting of features included in at least a portion of the training data in the training dataset; a first estimation procedure for estimating first parameters representing parameters for each task among parameters of each of a plurality of machine learning models so as to conform to the support set; a learning procedure for learning a second parameter representing a task-common parameter among parameters possessed by each of the plurality of machine learning models, so that an error between a predicted value predicted from a first feature included in the query set by combining the plurality of machine learning models and training data for the predicted value is small; A program that causes a computer to execute the following.

Images