Learning model generation device, learning model generation method, and program

Abstract

To appropriately generate a learning model for estimating an action plan matched to a target person.SOLUTION: A learning model generation device 10 comprises: a movement unit 12 which executes movement processing to move a sample for which an output error of a (t+1)-th learning model for observation data at time t+1 is larger than a predetermined amount among samples included in a target sample group to a source sample group from the target sample group; and a generation unit 13 which generates a plurality of weak learners by using at least observation data at time t to time T for the samples included in the target sample group after the movement processing and the samples included in the source sample group of after the movement processing, and generates a t-th learning model at least on the basis of each of the plurality of generated weak learners and a classification error which is evaluated using the observation data at time t for the samples included in the target sample group after the movement processing, for each of the plurality of generated weak learners.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] This disclosure relates to a learning model generation device, a method for generating a learning model, and a program, and more particularly to a learning model generation device, a method for generating a learning model, and a program for generating a learning model that dynamically estimates an action plan. [Background technology]

[0002] In medical settings, doctors record treatment plans for treating patients' illnesses and manage the implementation status of these plans. For example, Patent Document 1 discloses an illness management system that allows doctors to create treatment plans. [Prior art documents] [Patent Documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2002-163374 [Overview of the Initiative] [Problems that the invention aims to solve]

[0004] However, as described in Patent Document 1 above, doctors have to analyze the patient's current condition based on various patient information and then create a treatment plan tailored to the patient in accordance with treatment guidelines, which places a heavy burden on them. Furthermore, the quality of the treatment plan depends on the doctor's experience. Therefore, there is a need to support the creation of treatment plans in medical settings by automatically generating patient-specific treatment plans using a learning model. The above problem is not limited to medical settings, but is also true in educational settings or sports training, etc.

[0005] In view of the above-mentioned problems, the purpose of this disclosure is to provide a learning model generation device, a learning model generation method, and a program that suitably generate a learning model for creating an action plan tailored to the target individual. [Means for solving the problem]

[0006] A learning model generation device according to one aspect of this disclosure is: A moving unit performs a moving process to move samples from the target sample group to the source sample group if, among multiple samples included in the target sample group, the output error of the (t+1)th-order learning model for the observed data at time t+1 (where t is a natural number) is greater than a predetermined amount. Multiple weak learners are generated using at least the observation data from time t to time T of at least one sample included in the target sample group after the movement processing, and at least one sample included in the source sample group after the movement processing. A generation unit that generates an t-th order learning model based on each of the plurality of weak learners generated, and a classification error evaluated for each of the plurality of weak learners generated using the observation data at time t of at least one sample included in the target sample group after the transfer process. Equipped with, The aforementioned observational data includes at least the state and behavior of the sample at a specific time between time T and time T. The aforementioned t-th order learning model takes at least the state at time t as input and outputs the action at time t.

[0007] A method for generating a learning model according to one aspect of this disclosure is: A transfer process is performed to move samples from the target sample group to the source sample group if the output error of the (t+1)th-order learning model for the observed data at time t+1 (where t is a natural number) is greater than a predetermined amount. Multiple weak learners are generated using at least the observation data from time t to time T of at least one sample included in the target sample group after the movement processing, and at least one sample included in the source sample group after the movement processing. An t-th order learning model is generated based at least on each of the generated weak learners and the classification error for each of the generated weak learners, which is evaluated using the observation data at time t of at least one sample included in the target sample group after the transfer process. The aforementioned observational data includes at least the state and behavior of the sample at a specific time between time T and time T. The aforementioned t-th order learning model takes at least the state at time t as input and outputs the action at time t.

[0008] A program relating to one aspect of this disclosure is On the computer, A transfer process is performed to move samples from the target sample group to the source sample group if, among multiple samples included in the target sample group, the output error of the (t+1)th-order learning model for the observed data at time t+1 (where t is a natural number) is greater than a predetermined amount. A process to generate multiple weak learners using at least observation data from time t to time T of at least one sample included in the target sample group after the movement process and at least one sample included in the source sample group after the movement process, A process for generating an t-th order learning model based on each of the multiple weak learners generated, and the classification error for each of the multiple weak learners generated, using the observation data at time t of at least one sample included in the target sample group after the transfer process. Make it run. The aforementioned observational data includes at least the state and behavior of the sample at a specific time between time T and time T. The aforementioned t-th order learning model takes at least the state at time t as input and outputs the action at time t. [Effects of the Invention]

[0009] This disclosure provides a learning model generation device, a learning model generation method, and a program that suitably generate a learning model for estimating an action plan tailored to a target individual. [Brief explanation of the drawing]

[0010] [Figure 1] This is a block diagram showing the configuration of the learning model generation device according to Embodiment 1. [Figure 2] This is a flowchart showing the flow of the method for generating a learning model according to Embodiment 1. [Figure 3] This is a flowchart showing the flow of the method for generating a t-th order learning model according to Embodiment 1. [Figure 4] This is a block diagram showing the overall configuration of the system according to Embodiment 2. [Figure 5] This diagram schematically shows the processing flow of the system according to Embodiment 2. [Figure 6] This figure illustrates the method for deriving the weak learner included in the j-th order learning model according to Embodiment 2. [Figure 7] This is a block diagram showing the configuration of the learning model generation device according to Embodiment 2. [Figure 8] This figure shows an example of the data structure of the storage unit according to Embodiment 2. [Figure 9] This is a flowchart showing the flow of the method for generating a learning model according to Embodiment 2. [Figure 10] This is a flowchart showing the flow of the method for generating a t-th order learning model according to Embodiment 2. [Figure 11] This figure shows an example of an algorithm for generating a t-th order learning model according to Embodiment 2. [Figure 12] This figure shows an example of an algorithm for generating a t-th order learning model according to Embodiment 3. [Figure 13] This figure illustrates the method for deriving the weak learner included in the t-order learning model according to Embodiment 4. [Figure 14]This figure illustrates the method for deriving the weak learner included in the t-order learning model according to Embodiment 5. [Figure 15] This figure shows an example of an algorithm for generating a t-th order learning model according to Embodiment 5. [Figure 16] This is a diagram showing an example of a computer configuration. [Figure 17] This diagram provides a schematic overview of the processing flow of the related systems. [Figure 18] This is a diagram illustrating the derivation method of related t-th order learning models. [Modes for carrying out the invention]

[0011] Embodiments of the present disclosure will be described in detail below with reference to the drawings. In each drawing, the same or corresponding elements are denoted by the same reference numerals, and redundant explanations will be omitted where necessary for clarity.

[0012] <Problems with the embodiment> First, we will describe in detail the problems of at least one embodiment of this disclosure. To support the creation of patient treatment plans in medical settings, computers are being used to automatically generate treatment plans. For example, it is being considered to find past treatment plans of patients with the same disease as the target patient and create a similar treatment plan for the target patient.

[0013] However, even among patients with the same disease, there are differences in characteristics and information such as disease-related genes. Therefore, it is necessary to create a treatment plan tailored to each individual based on patient-specific information. This is expected to improve the effectiveness of treatment for patients.

[0014] Furthermore, it is being considered to sequentially select treatments tailored to each individual's characteristics, taking into account their individual history, after the start of treatment. This is expected to enhance treatment effectiveness and allow for the proposal of treatments that are less burdensome and less expensive for the patient.

[0015] In such a background, a system using a learning model has been developed that sequentially and automatically selects a treatment that maximizes the treatment effect of a patient according to the patient's reaction. Note that the above system can be used not only in the medical field but also in the educational field or sports training, etc. Therefore, hereinafter, instead of the term "treatment", the term "action", which is a higher concept, will be used.

[0016] FIG. 17 is a diagram schematically showing the processing flow of a related system. The system proposes an action to be performed at that time, which maximizes the treatment effect of the patient, using a different learning model for each time j. The time may be an absolute time or a relative time. In the case of a relative time, the time may be called a stage. Also, the time may refer to a point on the time axis or a predetermined period on the time axis. Hereinafter, j is a natural number. For example, time j = 1 indicates the first day of treatment, time j = 2 indicates the second day of treatment, time j = t indicates the t-th day of treatment, and time j = T (T is a natural number greater than t) may indicate the final time, that is, the last day of treatment.

[0017] For example, the j-th learning model D * j takes as input the state X jh observed for the subject h at time j. And the j-th learning model D * j estimates the action A jh of the subject h at time j. The estimated action A jh is an action that maximizes the total effect obtained by the subject h from time j to the final time T. For example, in FIG. 17, as the j-th learning models corresponding to times j = t - 1, t, t + 1, the (t - 1)-th learning model D * t-1 , the t-th learning model D * t , and the (t + 1)-th learning model D * t+1 are shown respectively.

[0018] The system's processing is divided into two phases: a model generation phase that generates a j-th order learning model for time j=1 to T, and an estimation phase that plans the actions of subject h using the j-th order learning model for time j=1 to T.

[0019] (Model generation phase) j-order learning model D * j This refers to the target sample group (hereinafter referred to as the T sample group) TG at time j. j It is generated using the observed data. The sample group TG is the set of patients (i.e., samples) whose observed data is used as training data during learning. Note that the observed data of sample i at time j is state X ji Action A ji And, effect Y ji A vector {X} that combines these two elements. ji ,A ji ,Y ji It is represented by}. Effect Y ji This indicates the amount of effect obtained by the actions taken at a specific time within the time range j=1 to T, for a sample with a particular state at that time. First, observational data for samples i=1,2,...,n (where n is a natural number) at time j=1 to T are prepared.

[0020] For example, consider the t-th order learning model D when j=t. * t The T sample group TG at time t is used to generate the t The observed data is used in the t-th order learning model D * t This is the training data for learning the t-th order learning model D. * t The training data for learning the model consists of observed data from samples that are included in a group of T samples, where i = 1, 2, ..., n (where n is a natural number).

[0021] The generation of each learning model is performed backward, going back in time j. That is, the t-th order learning model D when j=t * t This is the (t+1)th order learning model D when j=(t+1). *t+1 It is generated after the (t+1)th order learning model D is generated. * t+1 In learning, the T sample group TG corresponding to time t+1 is used. t+1 Observational data from the samples included in are used. Here, the (t+1)th order learning model D * t+1 All observational data from samples that were not optimal were processed using the t-th order learning model D. * t Therefore, the sample group TG corresponding to time t is discarded without being used. t The observed data of the samples included are the T sample group TG corresponding to time t+1. t+1 Among the samples included, the (t+1)th order learning model D * t+1 This is the sample after removing the samples that were not optimal. In other words, it is the (t+1)th order learned model D * t+1 This was the optimal sample.

[0022] Therefore, generally, the T sample group TG corresponding to time t is used. t The number of samples is the T sample group TG corresponding to time (t+1). t+1 This becomes smaller than the sample size. When the sample size decreases, the amount of observational data used as training data decreases, making it difficult to generate a learning model with high estimation accuracy.

[0023] Furthermore, "j-th order learning model D * j A "sample that was not optimal" refers to a sample i whose observed data contains state X. ji The j-th order learning model D * j When entered, the observed data includes behavior A ji and the j-th order learning model D * j This shows samples where the error between the output and the result is greater than a predetermined amount. Below, the above error is referred to as "j-th order learning model D". * j This is called the "output error of the j-th order learning model D". *j The sample that was not optimal was the j-th order learning model D * j The output error of the j-th order learning model D is greater than 0. * j This indicates that it misclassified. Conversely, "j-th order learning model D * j The sample that was optimal was the j-th order learning model D * j This shows a sample where the output error is below a predetermined amount. As an example, the following is "j-th order learning model D * j The "optimal sample" is the one included in the observational data. ji and the j-th order learning model D * j This indicates that the output matches, meaning the output error is 0.

[0024] (Estimated phase) On the other hand, the estimation of behavior is performed forward as time j progresses. For example, if the current time j is t, then the state X of subject h observed at the current time t is... th The t-th order learning model D * t By inputting this, the action A of the subject h at the current time t will be determined. th We obtain the state X of subject h observed at time t+1. Then, as time passes and it becomes time t+1, we obtain the state X of subject h observed at time t+1. (t+1)h By inputting this, the action A of subject h at time t+1 will be determined. (t+1)h This is how the necessary actions are estimated sequentially as time progresses. Therefore, the action plan is created dynamically.

[0025] The above problem can also be understood from mathematical formulas.

[0026] Figure 18 shows the related t-th order learning model D. * t This is a diagram illustrating the derivation method of the t-th order learning model D. * t This can be derived using the following equation (1).

number

[0027] The function f included in formula (1) is a function corresponding to the t-th learning model D * t Therefore, finding the function f is the same as finding the t-th learning model D * t

[0028] Block 900 shown in FIG. 18 indicates the total effect (reward) obtained from time (t + 1) to the final time T. Block 902 indicates the effect (reward) at time t. L is a classification loss function, and block 903 indicates the loss at time t.

[0029] Here, block 904 included in block 901 indicates the output error. Block 901 indicates that it becomes 1 when the output error is 0 at all times from time (t + 1) to time T, and becomes 0 otherwise. The fact that block 901 becomes 0 means that the observed data of that sample is discarded. That is, if there is a time when the output error is not 0 even once between time (t + 1) and time T, the observed data of that sample is discarded at that point. Therefore, it can be understood that the number of samples decreases as time progresses backward.

[0030] At least one of the following embodiments solves such problems.

[0031] <Embodiment 1> Next, Embodiment 1 of the present invention will be described. Embodiment 1 may be described as an overview of the embodiments described later. FIG. 1 is a block diagram showing the configuration of a learning model generation device 10 according to Embodiment 1. The learning model generation device 10 is a computer device that sequentially generates a learning model corresponding to each time in order to dynamically create an action plan. Specifically, the learning model generation device 10 uses the observation data at time j = 1 to T for samples of i = 1, 2,..., n to generate T j-th learning models D * j ​while generating it while going back in time corresponding to the model (D * T →D * T-1 →…→D * t+1 →D * t →…→D * 1). The learning model generation device 10 may be a computer system including one or more computer devices.

[0032] Here, the observation data at an arbitrary time j of sample i includes the state X of sample i at time j ji and the action A ji at least.

[0033] Also, the j-th learning model D * j is a learned model that outputs the action A jh at time j with at least the state X jh of the subject h at time j as an input. For example, the j-th learning model D * j is a model obtained by ensembling weak learners. An example of an ensembled model is boosting. In the following, the j-th learning model D * j is assumed to adopt the AdaBoost algorithm, which is an example of boosting. Also, the j-th learning model D * j will be described as being represented by a weighted sum of weak learners.

[0034] As shown in FIG. 1, the learning model generation device 10 includes a moving unit 12 and a generation unit 13. The moving unit 12 executes a moving process. The moving process is for the (t + 1)-th learning model D for the observation data at time t + 1 among the plurality of samples included in the T sample group * t+1This method involves moving samples with an output error greater than a predetermined amount from the T sample group to the source sample group. Hereafter, the source sample group will be referred to as the S sample group. Furthermore, samples included in the T sample group corresponding to a given time are sometimes referred to as T samples, and samples included in the S sample group corresponding to a given time are sometimes referred to as S samples.

[0035] Furthermore, "movement" can refer to either a physical or logical movement. A physical movement might involve changing the storage location. A logical movement might involve changing the attributes of the sample (such as its affiliation or type).

[0036] Furthermore, "greater than a predetermined amount" does not necessarily mean this, but it can mean greater than 0. In other words, the moving unit 12 moves the samples that were not optimal at time t+1 from the T sample group to the S sample group at time t. The T sample group at time t will not include the samples that were not optimal.

[0037] The generation unit 13 uses the observed data from time t to time T of the samples included in the T sample group and the observed data from time t to time T of the samples included in the S sample group to create the t-th order learning model D * t Generates.

[0038] Specifically, the generation unit 13 generates multiple weak learners, and combines these multiple weak learners to create the t-th order learning model D. * t This generates the t-th order learning model D. * t The weak learner included is the state X of subject i at time t. ti As input, action A at time t ti Outputs.

[0039] More specifically, the generation unit 13 first generates the observed data {X} from time t to time T of the samples included in the T sample group after the movement processing and the samples included in the S sample group after the movement processing.ji ,A ji ,Y ji Multiple weak learners are generated using (j=t,t+1,...,T, i=1,2,...,n) as training data. In this case, the generation unit 13 may use the observation data from time t to time T of all samples included in the T sample group as training data, or it may use the observation data from time t to time T of some of the samples. The same applies to the samples included in the S sample group.

[0040] Next, the generation unit 13 evaluates the classification error for each of the multiple weak learners using the observed data at time t of the samples included in the T sample group after the transfer process. In other words, the generation unit 13 calculates the classification error for the T samples.

[0041] Finally, the generation unit 13 generates the t-th order learning model D based at least on each of the multiple weak learners and the corresponding classification error. * t The generation unit 13 generates the t-th order learning model D by combining weak learners weighted with weights corresponding to the classification error mentioned above. * t Generates.

[0042] Figure 2 is a flowchart showing the flow of the method for generating a learning model according to Embodiment 1. First, the learning model generation device 10 acquires observation data for each sample in the T sample group (S10). Next, the generation unit 13 of the learning model generation device 10 generates the j-th order learning model D * j Generate (S11).

[0043] Next, the mobile unit 12 of the learning model generation device 10 repeats the processing shown in S12 to S13 for each sample in the T sample group at time j. In S12, the mobile unit 12 generates the j-th order learning model D for the observed data of that sample at time j. * j It determines whether or not there is an output error. An output error indicates a misjudgment. At this time, the movement unit 12 determines whether or not there is an output error in the j-th order learning model D *j State X included in the observed data of sample i at time j. ji The input is used, and the resulting output value and the behavior A included in the observed data are used. ji The difference between the two is calculated as the output error. If there is an output error (Yes in S12), the moving unit 12 moves the sample from the T sample group to the S sample group without discarding it (S13). On the other hand, if there is no output error (No in S12), the moving unit 12 does not move the sample from the T sample group to the S sample group, but leaves it in the T sample group.

[0044] After performing the above process for all samples in the T sample group, the learning model generator 10 decrements time j (S14). Then, if time j is greater than 0 (Yes in S15), the learning model generator 10 returns to the process in S11, and if time j becomes 0 (No in S15), the process terminates.

[0045] Figure 3 is a flowchart showing the flow of the method for generating an t-th order learning model according to Embodiment 1 when j=t. First, the generation unit 13 generates a weak learner using the observed data from time j=t to time T of the samples included in the T sample group and the S sample group (S20). Next, the generation unit 13 evaluates the classification error of the weak learner using the observed data at time t of multiple samples included in the T sample group (S21). Through S20 to S21, the generation unit 13 generates multiple weak learners and the classification error corresponding to each weak learner. Next, the generation unit 13 generates an t-th order learning model based at least on the generated weak learners and the corresponding classification errors (S22).

[0046] Alternatively, the generation unit 13 may repeat steps S20 to S21 for the number of weak learners to be generated, and then execute step S22.

[0047] Thus, according to Embodiment 1, in generating the weak learner included in the learning model corresponding to the target time t, in addition to T samples, S samples that were deemed suboptimal in the learning model corresponding to the later time t+1 are used. Note that the learning model corresponding to the later time t+1 is generated before the learning model corresponding to the target time t. Therefore, the training data used for learning can be augmented. As a result, a learning model can be generated that accurately estimates the actions of each individual at each time point.

[0048] <Embodiment 2> Next, Embodiment 2 of this disclosure will be described. Figure 4 is a block diagram showing the overall configuration of System 1 according to Embodiment 2. System 1 is a computer system for dynamically creating action plans tailored to each individual. System 1 comprises a learning model generation device 10a, a learning model storage device 20, and an estimation device 30. The learning model generation device 10a, the learning model storage device 20, and the estimation device 30 are connected to each other in a manner that allows them to communicate with one another.

[0049] The learning model generation device 10a is an example of the learning model generation device 10 described above. The learning model generation device 10 generates a learning model that estimates the action A that should be taken at each time point. The action A that should be taken is the action that will maximize the effect obtained from that time point onward.

[0050] The learning model storage device 20 is a storage device that stores the learning model generated by the learning model generation device 10a at each time point.

[0051] The estimation device 30 dynamically creates an action plan for subject h. Specifically, the estimation device 30 reads the learning model stored in the learning model memory device 20 and uses the learning model to sequentially estimate the action A that subject h should take at the target time.

[0052] Figure 5 is a schematic diagram showing the processing flow of System 1 according to Embodiment 2. The processing flow of System 1 according to Embodiment 2 is basically the same as the processing flow shown in Figure 17. The learning model generation device 10a performs the model generation phase processing shown by the dotted line in Figure 5, and the estimation device 30 performs the estimation phase processing shown by the dashed line in Figure 5.

[0053] In Figure 5, similar to Figure 17, the T sample group TG t Therefore, the T sample group TG at time t+1 t+1 Among the samples included, the (t+1)th order learning model D * t+1 This shows that samples that were not optimal are removed. However, in Figure 5, the T sample group TG at time t+1 t+1 Among the samples included, the (t+1)th order learning model D * t+1 The samples that were not optimal were the S sample group SG at time t. t It differs from Figure 17 in that it is included in the following. In this embodiment 2, the S sample group SG t This includes the (t+1)th order learning model D * t+1 This includes only the samples that were not optimal. However, in addition to these, the S sample group SG t This may include at least some of the samples that were not optimal for the learning model corresponding to a later time (e.g., j=t+2).

[0054] Then the learning model generation device 10a generates the t-th learning model D * t When generating the T sample group TG, t In addition to the observational data of the samples included, S sample group SG t Observational data from the samples included in the sample will be used.

[0055] In this second embodiment, the j-th order learning model D * j This consists of M weak learners (first weak learner T). (1) , second weak learning device T (2) ,...M-th weak learner T(M) It is expressed as a weighted sum of (M is a natural number). Specifically, the j-th order learning model D * j This is given by equation (2) below.

number

[0056] For example, confidence level α j (m) This is given by equation (3) below.

number

[0057] For example, the second classification error err2 (m) This is given by equation (4) below.

number

[0058] Furthermore, the second classification error err2 (m) The coefficient ξ included i This is given by equation (5) below.

number

[0059] Figure 6 shows the t-th order learning model D according to Embodiment 2. * t Weak learner T included (m) This is a diagram to explain the derivation method. First, the weak learner T (m) This relates to f included in equation (1). f included in equation (1) is given by the following equation (6).

number

[0060] g(X) represents a K-dimensional vector. The relationship between g(X) and T(X) is given by equation (7) below.

number

[0061] The derivation of the optimized g(X) is given by equation (8-1) below.

number

[0062] β m z is the parameter for the m-th weak learner. i A is patient i ti This represents the following: Here, z represents a K-dimensional vector, given by equation (9) below.

number

[0063] The objective function shown after arg min is sometimes called the first classification error err1. The first classification error err1 is equal to the weak learner T (m) This corresponds to the classification error evaluated using the observed data of the T sample at time t and the observed data of the S sample at time t.

[0064] Finding the optimized g(X) corresponds to finding the g(X) that minimizes the first classification error err1.

[0065] Block 900' in Figure 6, corresponding to equation (8), corresponds to blocks 900 and 901 in Figure 18, corresponding to equation (1). Similarly, block 902 in Figure 6, corresponding to equation (8), is the same as block 902 in Figure 18, corresponding to equation (1). However, Figure 6 differs from Figure 18 in that block 100 is included instead of block 903. Block 100 shows the loss at time t for sample i.

[0066] ω contained in block 100 i ω is the weight added to the loss for sample i. i m―1 This is given by equation (10).

number

[0067] Furthermore, f is the weighted sum of m-1, and is given by the following equation (11).

number

[0068] weight ω iThis indicates the degree to which the observed data of sample i influences the optimization of g(X) (i.e., the training of the weak learner T(X)). In this embodiment 2, the weight ω i This can be updated each time a trained weak learner is generated. The mode of update differs depending on whether sample i is classified as a T sample or an S sample at the corresponding time. The weight when sample i is a T sample is ω i T The weight when sample i is an S sample is ω i S Let's assume that.

[0069] Figure 7 is a block diagram showing the configuration of the learning model generation device 10a according to Embodiment 2. The learning model generation device 10a comprises a storage unit 11, a movement unit 12a, a generation unit 13a, and an output unit 18.

[0070] The memory unit 11 is a memory device that stores observed data from time j=1 to T for samples i=1 to n.

[0071] The movement unit 12a is an example of the movement unit 12 described above. The generation unit 13a is an example of the generation unit 13 described above. The generation unit 13a includes a weak learner generation unit 14, a confidence calculation unit 15, a weight update unit 16, and a learning model generation unit 17. The movement unit 12a and the generation unit 13a sequentially generate the j-th order learning model D backward from j=T. * j Generate the j-th order learning model D. * j This is output to the output unit 18.

[0072] Output unit 18 outputs the generated j-th order learning model D * j The output unit 18 outputs the generated j-th order learning model D. * j The learned model is stored in the learning model memory 20.

[0073] Figure 8 shows an example of the data structure of the storage unit 11 according to Embodiment 2. The observation data stored in the storage unit 11 is (state X) of samples i=1 to n from time j=1 to T. ji ,Action A ji Effect Y ji The observed data is divided into three parts: d_TG, which is the observed data of the samples from the T sample group TG (T samples); d_SG, which is the observed data of the samples from the S sample group SG (S samples); and d_NG, which is the observed data of the samples from the discarded sample group NG (N samples).

[0074] T-order learning model D corresponding to the final time j=T * T At the time of generation, all samples are included in the T sample group TG. All observational data stored in the memory unit 11 are classified as observational data d_TG. At this time, the number of samples included in the S sample group SG is 0, and observational data d_SG does not exist. Also at this time, the number of samples included in the N sample group NG is 0, and observational data d_NG does not exist.

[0075] As j, which corresponds to the generated learning model, decreases, the number of samples included in the T sample group TG decreases, and the number of samples included in either the S sample group SG or the N sample group NG increases. Therefore, as j decreases, the number of observational data classified into observational data d_TG decreases, and the number of observational data classified into either observational data d_SG or observational data d_NG increases.

[0076] Furthermore, the samples included in the S sample group SG are those that were not deemed optimal by the learning model corresponding to the time immediately following the time corresponding to the generating learning model (e.g., time t) (e.g., time t+1).

[0077] Next, the specific processing of each element will be explained using Figures 9 to 11. First, Figure 9 is a flowchart showing the flow of the method for generating a learning model according to Embodiment 2. The steps shown in Figure 9 include S100 in addition to the steps shown in Figure 2.

[0078] In S100, the movement unit 12a moves the samples from the S sample group SG to the N sample group NG and discards the samples included in the S sample group SG. Specifically, the movement unit 12a reclassifies the observation data that was previously classified as observation data d_SG as observation data d_NG. By initializing the S sample group SG, only the samples that were optimal up to the most recent point but were not optimal recently can be considered in the generation of the learning model. This reduces the impact on the decrease in the estimation accuracy of the learning model caused by using the observation data of the non-optimal samples as training data, and allows for the appropriate augmentation of the training data.

[0079] Furthermore, in S12-S13, the moving unit 12a applies the (t+1)th-order learning model D to the observed data at time t+1 from the T sample group TG. * t+1 Samples that cause output errors are moved from the T sample group TG to the S sample group SG. Specifically, the moving unit 12a moves the (t+1)th order learning model D from the observed data that was classified as observed data d_TG. * t+1 Observational data that produces output errors are reclassified into observational data d_SG.

[0080] Figure 10 is a flowchart showing the flow of the method for generating an t-th order learning model according to Embodiment 2 when j=t. Figure 11 is a diagram showing an example of an algorithm for generating an t-th order learning model according to Embodiment 2.

[0081] First, in S110 of Figure 10, the weak learner generation unit 14 of the generation unit 13a sets various parameters. Specifically, as shown in paragraph 1 of Figure 11, the weak learner generation unit 14 sets the weights ω of the S samples. i S The coefficient α used to update S,iSet the weight ω for each sample, as shown in paragraph 2 of Figure 11. i (ω i T or ω i S Initialize ).

[0082] Next, the process shown in S111 to S115 below is repeated M times. M is predetermined. In iteration m, at S111, the weak learner generation unit 14 generates the t-th order learning model D * t Among the M weak learners included, the mth weak learner T (m) This generates the following. At this time, the weak learner generation unit 14 generates the weights ω set for each sample. i The observed data d_TG from time t to time T for T samples and observed data d_SG from time t to time T for S samples are used, weighted accordingly. The weak learner generation unit 14 then finds a weak learner that minimizes the first classification error err1, which is evaluated using the observed data of T samples and S samples, and uses this as the weak learner T. (m) Specifically, the weak learner generation unit 14 generates a weak learner T using equation (8), as shown in paragraph 5 of Figure 11. (m) Generates.

[0083] In S112, the confidence calculation unit 15 of the generation unit 13a uses the observed data of T samples at time t to generate a weak learner T (m) Second classification error err2 (m) This is evaluated. Specifically, as shown in paragraphs 6-7 of Figure 11, the confidence calculation unit 15 calculates the second classification error using equation (4).

[0084] In S113, the confidence calculation unit 15 calculates the second classification error err2 (m) Based on the weak learner T (m) Confidence level α j (m) The confidence level is calculated. Specifically, as shown in paragraph 8 of Figure 11, the confidence level calculation unit 15 calculates the confidence level using equation (3).

[0085] Next, the weight update unit 16 of the generation unit 13a repeats the process shown in S114 to S115 for each sample. In this process, the weight update unit 16 performs different processing for T samples and S samples. Specifically, the weight update unit 16 applies the weak learner T to the observed data d_TG at time t among the T samples. (m) For samples with output errors, the weight ω i T Increase (Yes in S114 → S115). In addition to or instead of this, the weight update unit 16 selects the weak learner T for the observed data d_SG at time t from the S samples. (m) For samples with output errors, the weight ω i S This reduces the (Yes in S114 → S115). On the other hand, the weight update unit 16 does not update the weights for samples where the output error is less than or equal to a predetermined amount, or where there is no output error, regardless of whether it is T samples or S samples (No in S114). As a result, the observed data of the sample that was optimal at time t becomes relatively more influential as the iterations progress compared to the observed data of the sample that was not optimal at time t. In other words, the t-th order learning model can be generated by giving more weight to the sample that was optimal at time t than to the sample that was not optimal. Therefore, the t-th order learning model D * t The estimation accuracy improves.

[0086] More specifically, the weight update unit 16 updates the weight ω in the manner shown in paragraphs 9-11 of Figure 11. i T and ω i S The weight update unit 16 updates the weak learner T for the observed data d_TG at time t out of the T samples. (m) For samples with output errors, the weak learner T (m) Confidence level α t (m) Weight ω i T You may increase the confidence α as shown in equation (3). t (m) This is the second classification error err2 (m)Because it is calculated based on the weight ω i T This is the second classification error err2 (m) It increases accordingly. Therefore, for samples that were optimal at time t, the larger the second classification error, the stronger the influence can be as the iterations progress. This allows for more favorable emphasis on samples that were optimal at time t than on samples that were not optimal in the generation of the t-th order learning model. As a result, the t-th order learning model D * t The estimation accuracy is further improved. Furthermore, the weight update unit 16 updates the coefficient α set in S110 for samples among the S samples that have an output error of the weak learner T(m) with respect to the observed data d_SG at time t. S,i , in other words, a predetermined coefficient α S,i Weight ω i S It is permissible to reduce it.

[0087] The weight update unit 16 performs the processes shown in S114 to S115 for all samples, and then proceeds to the next iteration m+1.

[0088] By repeating this M times, the generation unit 13a generates M weak learners T (m) (First weak learner T) (1) , second weak learning device T (2) ,...M-th weak learner T (M) ) and the confidence level α corresponding to each weak learner t (m) (First confidence level α) t (1) , second confidence level α t (2) ...Mth confidence level α t (M) ) generates.

[0089] Then, in S116, the learning model generation unit 17 of the generation unit 13a generates M weak learners T (m) For each of these, the corresponding confidence level α t (m) By combining weighted values, we can create a t-th order learning model D * tSpecifically, as shown in paragraph 13 of Figure 11, the learning model generation unit 17 uses equation (2) to generate each weak learner T (m) For this, the corresponding confidence level α t (m) The weighted values ​​are added together to form the t-th order learning model D. * t Generates.

[0090] Thus, according to Embodiment 2, similar to Embodiment 1, the training data used to generate the learning model, particularly the weak learner, can be augmented. This makes it possible to generate a learning model that accurately estimates the actions of each individual at each moment in time.

[0091] Furthermore, in the process of generating multiple weak learners included in the learning model, the influence weights ω are set so that T samples have a greater influence on learning than S samples. i This will be updated, thus improving the estimation accuracy of the learning model.

[0092] <Embodiment 3> Next, Embodiment 3 of this disclosure will be described. Embodiment 3 is a t-th order learning model D * t When generating multiple weak learners included in the (t+1)th order learning model D * t+1 The results of the generation are considered. Specifically, the generation unit 13a generates the t-th order learning model D * t The weight ω indicates the degree of influence that S samples have on the training of the weak learner included in the dataset. i S The S samples then form the (t+1)th-order learning model D. * t+1 The optimal amount is determined based on the quantity that was deemed suboptimal.

[0093] The flow of the method for generating the t-th order learning model according to Embodiment 3 is basically the same as the steps shown in Figure 10, so below we will explain only the differences using Figure 12. Figure 12 shows the t-th order learning model D according to Embodiment 3. * tThis figure shows an example of an algorithm for generating [something].

[0094] First, in S110, the weak learner generation unit 14 generates a (t+1)th-order learning model D for the observed data d_SG at time t of the S sample. * t+1 Output error τ i Based on this, weight ω i S Determine the initial value of the above output error τ. i This corresponds to the "quantity that was deemed suboptimal" mentioned above. Specifically, as shown in paragraph 1 of Figure 12, the weak learner generation unit 14 generates the output error τ. i Based on this, weight ω i S The coefficient α for determining the initial value S,i Determine the initial value of the output error τ, which allows us to determine the degree of influence that S sample has even among S samples. i It can be varied accordingly. For example, the weak learner generator 14 assigns a weight ω to S samples with a larger output error τi. i S By reducing the initial value of the output error τ, the influence is reduced. i The smaller the S-sample size, the more it can be designed to influence the learning of the weak learner.

[0095] Furthermore, the weight update unit 16 updates the weight ω of the S sample in S115. i S The amount of weight reduction when updating is also the output error τ mentioned above. i The decision is made based on the following: Specifically, as shown in paragraph 10 of Figure 12, the weight update unit 16 determines the weight ω to be used in the next iteration m+1. i S The above output error τ i The coefficient α includes S,i It is updated to have a negative correlation with this. This reduces the output error τ. i The larger the S sample, the greater the decrease in weight, and as a result, the impact can be reduced with each increase in the number of iterations. Therefore, even between S samples, the output error τ i Depending on the circumstances, the degree of influence exerted by the S sample can be significantly different.

[0096] In Figure 12, the generation unit 13a has weight ω i S The initial value of the output error τ i The decision is based on the weight ω i S The amount of decrease when updating is the output error τ i We are doing both making decisions based on [the above] and [the other], but we may omit either one.

[0097] <Embodiment 4> Next, Embodiment 4 of the present disclosure will be described. In Embodiment 4, the generation unit 13a generates the t-th order learning model D * t When generating the (t+1)th order learning model D, for S samples, the effect at time t+1 is taken by subtracting a predetermined amount from the amount of effect Y contained in the observed data at time t+1. This allows S samples to be used in the (t+1)th order learning model D. * t+1 This sample was deemed not optimal for the t-th order learning model D. * t This can be explicitly taught during the generation process.

[0098] Figure 13 shows the t-th order learning model D according to Embodiment 4. * t Weak learner T included (m) This is a diagram illustrating the derivation method. In Embodiment 4, the weak learner T (m) This can be derived from equation (12) below, instead of equation (8).

number

[0099] As shown in Figure 13, equation (12) differs from equation (8) in that it has block 900'' instead of block 900'. In block 900'', S corresponds to the effect at time t+1. t+1,i However, Y included in the observational data t+1,i Using this, the following equation (13) (block 120) is given.

number

[0100] λ is a tuning parameter less than 1. By multiplying the effect Y at time t+1 for S samples by λ, the amount of effect Y at time t+1 can be reduced for S samples. On the other hand, for T samples, the observed amount of effect Y at time t+1 is used. This allows a learning model to be generated while taking into account that there are S samples.

[0101] Furthermore, the λ applied to the effect of S samples at time t+1 is determined for each sample by the (t+1)th order learning model D for the observed data at time t+1 of that sample. * t+1 Output error τ i It may be determined based on the following. For example, the weak learner generation unit 14 has an output error τ. i For small S samples, assign λ=0.9, and output error τ i For S samples with a large value, λ=0.5 may be assigned. In this way, the weak learner generation unit 14 generates an output error τ i The larger the value of S samples, that is, the further S samples were from the optimal value, the larger the amount that can be subtracted from the effect Y at time t+1.

[0102] <Embodiment 5> Next, Embodiment 5 of the present disclosure will be described. In Embodiment 5, the generation unit 13a uses cost-sensitive learning when generating weak learners to be included in the learning model.

[0103] Figure 14 shows the t-th order learning model D according to Embodiment 5. * t Weak learner T included (m) This is a diagram to explain the derivation method.

[0104] In Embodiment 5, the weak learner T (m) This can be derived from equation (14) below, instead of equation (8).

number

[0105] In equation (14), the cost function C is included in the summation. * Equation (8) differs from this in that it introduces block 130 in Figure 14. Cost function C * This is a weak learner T (m) If there is an output error, that is, if the weak learner T (m) This system imposes penalties if it makes a wrong judgment. For example, the cost function C * This is a weak learner T (m) If the output error is large, a large penalty will be imposed on the weak learner T. (m) If the output error is small, it is designed to impose a small penalty.

[0106] For example, the cost function C when K=5 * It is given by equation (15) below. Note that C * (p,q) represents the element in column p, row q of the matrix that shows the cost function (cost matrix).

number

[0107] t-th order learning model D according to Embodiment 5 * t The process for generating the model is basically the same as the steps shown in Figure 10, so below we will explain only the differences using Figure 15. Figure 15 shows the t-th order learning model D according to Embodiment 5. * t This figure shows an example of an algorithm for generating [something].

[0108] In S111, the weak learner generation unit 14 generates a weak learner T using equation (14) instead of equation (8), as shown in paragraph 5 of Figure 15. (m) This generates a weak learner T. (m) The weak learner T tries to ensure that the behavior estimated by the weak learner T does not deviate too much from the behavior A observed in the sample. (m) This is learned. As a result, the weak learner T (m) However, this allows us to more clearly distinguish between S samples that were close to optimal and S samples that were far from optimal. In other words, weak learner T (m) However, it becomes possible to more accurately identify the S sample that was closest to optimal.

[0109] Next, the physical configuration of the learning model generation devices 10, 10a and estimation device 30 included in System 1 will be described. Figure 18 shows an example of the configuration of a computer that can be used as the learning model generation devices 10, 10a or estimation device 30. The computer 1000 has a processor 1010, a storage unit 1020, a ROM (Read Only Memory) 1030, a RAM (Random Access Memory) 1040, a communication interface (IF: Interface) 1050, and a user interface 1060.

[0110] The communication interface 1050 is an interface for connecting the computer 1000 to a communication network via wired communication means or wireless communication means. The user interface 1060 includes a display unit, such as a display. The user interface 1060 also includes input units such as a keyboard, mouse, and touch panel. Note that the user interface 1060 is not mandatory.

[0111] The memory unit 1020 is an auxiliary storage device capable of holding various types of data. The memory unit 1020 does not necessarily have to be part of the computer 1000; it may be an external storage device or cloud storage connected to the computer 1000 via a network.

[0112] ROM 1030 is a non-volatile memory device. A semiconductor memory device, such as a relatively small-capacity flash memory, is used for ROM 1030. The program executed by processor 1010 can be stored in storage unit 1020 or ROM 1030. Storage unit 1020 or ROM 1030 stores various programs for realizing the functions of each part within the learning model generation device 10, 10a or estimation device 30.

[0113] The above program can be stored using various types of non-temporary computer-readable media and supplied to computer 1000. Non-temporary computer-readable media include various types of tangible storage media. Examples of non-temporary computer-readable media include magnetic recording media such as flexible disks, magnetic tapes, or hard disks; magneto-optical recording media such as magneto-optical disks; optical disc media such as CDs (compact discs) or DVDs (digital versatile disks); and semiconductor memories such as mask ROMs, PROMs (programmable ROMs), EPROMs (erasable PROMs), flash ROMs, or RAMs. The program may also be supplied to the computer using various types of temporary computer-readable media. Examples of temporary computer-readable media include electrical signals, optical signals, and electromagnetic waves. Temporary computer-readable media can be supplied to the computer via wired communication channels such as electric wires and optical fibers, or via wireless communication channels.

[0114] RAM 1040 is a volatile memory device. Various semiconductor memory devices such as DRAM (Dynamic Random Access Memory) or SRAM (Static Random Access Memory) can be used for RAM 1040. RAM 1040 can be used as an internal buffer for temporarily storing data, etc. Processor 1010 loads the program stored in memory unit 1020 or ROM 1030 into RAM 1040 and executes it. Processor 1010 may be a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit). By executing the program, the functions of each part in the learning model generation device 10, 10a or estimation device 30 can be realized. Processor 1010 may have an internal buffer that can temporarily store data, etc.

[0115] It should be noted that the present invention is not limited to the embodiments described above, and can be modified as appropriate without departing from the spirit of the invention.

[0116] Some or all of the above embodiments may also be described as follows, but are not limited to the following: (Note 1) A moving unit performs a moving process to move samples from the target sample group to the source sample group if, among multiple samples included in the target sample group, the output error of the (t+1)th-order learning model for the observed data at time t+1 (where t is a natural number) is greater than a predetermined amount. Multiple weak learners are generated using at least the observation data from time t to time T of at least one sample included in the target sample group after the movement processing, and at least one sample included in the source sample group after the movement processing. A generation unit that generates an t-th order learning model based on each of the plurality of weak learners generated, and a classification error evaluated for each of the plurality of weak learners generated using the observation data at time t of at least one sample included in the target sample group after the transfer process. comprising The observation data includes at least the state and behavior of samples at specific times up to time T. The t-th learning model takes at least the state at time t as an input and outputs the behavior at time t. Learning model generation device. (Appendix 2) After discarding the samples included in the source sample group, the moving unit moves the samples for which the output error of the (t + 1)-th learning model with respect to the observation data at time t + 1 is greater than a predetermined amount to the source sample group. The learning model generation device according to Appendix 1. (Appendix 3) The plurality of weak learners includes at least a first weak learner and a second weak learner. The generation unit generates the first weak learner using the observation data weighted with weights set for each sample, For the samples included in the target sample group after the moving process, for which the output error of the first weak learner with respect to the observation data at time t is greater than a predetermined amount, the weight is increased. For the samples included in the source sample group after the moving process, for which the output error of the first weak learner with respect to the observation data at time t is greater than a predetermined amount, the weight is decreased. generates the second weak learner using the observation data weighted with weights updated for each sample. The learning model generation device according to Appendix 1 or 2. (Appendix 4) For the samples included in the target sample group after the moving process, for which the output error of the first weak learner with respect to the observation data at time t is greater than a predetermined amount, the weight is increased according to the classification error evaluated for the first weak learner. For samples included in the source sample group after the aforementioned transfer process, the weight of samples whose output error of the first weak learner for the observed data at time t is greater than a predetermined amount will be reduced according to a predetermined coefficient. The learning model generation device described in Appendix 3. (Note 5) The generation unit determines, for each sample included in the source sample group after the movement process, the initial value of the weight of the observed data for that sample and the amount of weight reduction when updating the weight of that sample, based on the output error of the (t+1)th-order learning model for the observed data of that sample at time t+1. A learning model generation device as described in Appendix 3 or 4. (Note 6) The generating unit is For each sample included in the source sample group after the aforementioned transfer process, the larger the output error of the (t+1)th-order learning model for the observed data at time t+1 of that sample, the smaller the initial value of the weight of the observed data for that sample. or For each sample included in the source sample group after the aforementioned transfer process, the larger the output error of the (t+1)th-order learning model with respect to the observed data at time t+1 for that sample, the larger the amount of weight reduction when updating the weight of that sample. The learning model generation device described in Appendix 5. (Note 7) The observed data includes the amount of effect obtained by the action at a specific time, for a sample having a state at a specific time up to time T. A learning model generation device as described in any one of the items 1 to 6 of the appendix. (Note 8) When the generation unit generates the multiple weak learners, for each sample included in the source sample group after the transfer process, it uses information obtained by subtracting an amount corresponding to the output error of the (t+1)th-order learning model from the amount of effect included in the observed data at time t+1 as the effect of that sample at time t+1. The learning model generation device described in Appendix 7. (Note 9) When the generation unit reduces the amount of the effect, it increases the amount of reduction for samples where the output error of the (t+1)th-order learning model at time t+1 is large. The learning model generation device described in Appendix 8. (Note 10) The generation unit uses cost-sensitive learning when generating each of the plurality of weak learners. A learning model generation device as described in any one of the items 1 to 9 in the appendix. (Note 11) The generating unit is For each of the generated weak learners, the confidence level of the weak learner is calculated based at least on the classification error, which is evaluated using the time t observation data of at least one sample included in the target sample group after the transfer process. The t-th order learning model is generated by combining each of the generated weak learners, weighted by the corresponding confidence level. A learning model generation device as described in any one of the items 1 to 10 in the appendix. (Note 12) A transfer process is performed to move samples from the target sample group to the source sample group if the output error of the (t+1)th-order learning model for the observed data at time t+1 (where t is a natural number) is greater than a predetermined amount. Multiple weak learners are generated using at least the observation data from time t to time T of at least one sample included in the target sample group after the movement processing, and at least one sample included in the source sample group after the movement processing. An t-th order learning model is generated based at least on each of the generated weak learners and the classification error for each of the generated weak learners, which is evaluated using the observation data at time t of at least one sample included in the target sample group after the transfer process. A method for generating a learning model, The aforementioned observational data includes at least the state and behavior of the sample at a specific time between time T and time T. The aforementioned t-th order learning model takes at least the state at time t as input and outputs the action at time t. Generation method. (Note 13) On the computer, A transfer process is performed to move samples from the target sample group to the source sample group if, among multiple samples included in the target sample group, the output error of the (t+1)th-order learning model for the observed data at time t+1 (where t is a natural number) is greater than a predetermined amount. A process to generate multiple weak learners using at least observation data from time t to time T of at least one sample included in the target sample group after the movement process and at least one sample included in the source sample group after the movement process, A process for generating an t-th order learning model based on each of the multiple weak learners generated, and the classification error for each of the multiple weak learners generated, using the observation data at time t of at least one sample included in the target sample group after the transfer process. A program to execute, The aforementioned observational data includes at least the state and behavior of the sample at a specific time between time T and time T. The aforementioned t-th order learning model takes at least the state at time t as input and outputs the action at time t. program. [Explanation of Symbols]

[0117] 1 System 10,10a Learning Model Generation Device 11 Storage section 12,12a Moving part 13,13a Generation part 14 Weak Learner Generation Unit 15. Confidence Calculation Unit 16. Weight update section 17. Learning Model Generation Unit 18 Output unit 20 Learning model storage device 30 Estimation device 100, 120, 130, 900, 900’, 900’’, 901, 902, 903, 904 Blocks 1000 Computer 1010 Processor 1020 Memory unit 1030 ROM 1040 RAM 1050 Communication interface (IF) 1060 User interface (IF) TG Target sample group SG Source sample group

Claims

1. A moving unit performs a moving process to move samples from the target sample group to the source sample group if, among multiple samples included in the target sample group, the output error of the (t+1)th-order learning model for the observed data at time t+1 (where t is a natural number) is greater than a predetermined amount. Multiple weak learners are generated using at least the observation data from time t to time T of at least one sample included in the target sample group after the movement processing, and at least one sample included in the source sample group after the movement processing. A generation unit that generates an t-th order learning model based at least on each of the plurality of weak learners generated, and the classification error for each of the plurality of weak learners generated, using the observation data at time t of at least one sample included in the target sample group after the transfer processing. Equipped with, The aforementioned observational data includes at least the state and behavior of the sample at a specific time between now and time T, The aforementioned t-th order learning model takes the state at time t as input and outputs the action at time t. A learning model generation device.

2. The moving unit discards samples included in the source sample group and then moves samples whose output error of the (t+1)th-order learning model for the observed data at time t+1 is greater than a predetermined amount to the source sample group. A learning model generation device according to claim 1.

3. The aforementioned plurality of weak learners include at least a first weak learner and a second weak learner, The generating unit is The first weak learner is generated using observation data weighted with weights set for each sample. Among the samples included in the target sample group after the aforementioned transfer process, for samples where the output error of the first weak learner with respect to the observed data at time t is greater than a predetermined amount, the weight is increased. Among the samples included in the source sample group after the aforementioned transfer process, for samples where the output error of the first weak learner with respect to the observed data at time t is greater than a predetermined amount, the weight is reduced. The second weak learner is generated using observational data weighted with weights updated for each sample. A learning model generation device according to claim 1 or 2.

4. Among the samples included in the target sample group after the transfer process, for samples whose output error of the first weak learner with respect to the observed data at time t is greater than a predetermined amount, the weight is increased according to the classification error evaluated for the first weak learner. For samples included in the source sample group after the aforementioned transfer process, the weight of samples whose output error of the first weak learner for the observed data at time t is greater than a predetermined amount will be reduced according to a predetermined coefficient. The learning model generation apparatus according to claim 3.

5. The generation unit determines, for each sample included in the source sample group after the movement process, the initial value of the weight of the observed data for that sample and the amount of weight reduction when updating the weight of that sample, based on the output error of the (t+1)th-order learning model for the observed data of that sample at time t+1. The learning model generation apparatus according to claim 3.

6. The observed data includes the amount of effect obtained by the action at a specific time, for a sample having a state at a specific time up to time T. A learning model generation device according to claim 1 or 2.

7. When generating the multiple weak learners, the generation unit uses information obtained by subtracting an amount corresponding to the output error of the (t+1)th-order learning model from the amount of effect contained in the observed data at time t+1, as the effect of each sample included in the source sample group after the transfer process. The learning model generation apparatus according to claim 6.

8. The generation unit uses cost-sensitive learning when generating each of the plurality of weak learners. A learning model generation device according to claim 1 or 2.

9. A transfer process is performed to move samples from the target sample group to the source sample group if the output error of the (t+1)th-order learning model for the observed data at time t+1 (where t is a natural number) is greater than a predetermined amount. Multiple weak learners are generated using at least the observation data from time t to time T of at least one sample included in the target sample group after the movement processing, and at least one sample included in the source sample group after the movement processing. An t-th order learning model is generated based at least on each of the generated weak learners and the classification error for each of the generated weak learners, which is evaluated using the observation data at time t of at least one sample included in the target sample group after the movement processing. A method for generating a learning model, The aforementioned observational data includes at least the state and behavior of the sample at a specific time between now and time T, The aforementioned t-th order learning model takes the state at time t as input and outputs the action at time t. Generation method.

10. On the computer, A transfer process is performed to move samples from the target sample group to the source sample group if, among multiple samples included in the target sample group, the output error of the (t+1)th-order learning model for the observed data at time t+1 (where t is a natural number) is greater than a predetermined amount. A process to generate multiple weak learners using at least observation data from time t to time T of at least one sample included in the target sample group after the movement process and at least one sample included in the source sample group after the movement process, A process for generating an t-th order learning model based on each of the multiple weak learners generated, and the classification error for each of the multiple weak learners generated, using the observation data at time t of at least one sample included in the target sample group after the transfer process. A program to execute, The aforementioned observational data includes at least the state and behavior of the sample at a specific time between now and time T, The aforementioned t-th order learning model takes the state at time t as input and outputs the action at time t. program.

Images