Pipeline set generation program, pipeline set generation method, and information processing device

Abstract

To provide a pipeline set generation program capable of improving the convenience of machine learning automation, a pipeline set generation method, and an information processing device.SOLUTION: A pipeline set program causes a computer to execute processing for: acquiring a pipeline set where each pipeline includes a machine learning model on the basis of a plurality of tasks; generating a second pipeline by executing simplification processing including at least one of processing for deleting components included in the pipeline and processing for changing hyper parameters of components included in the pipeline to default values; acquiring an evaluation value of the second pipeline by executing the second pipeline for the plurality of tasks; and adding the second pipeline to the pipeline set on the basis of the evaluation value.SELECTED DRAWING: Figure 6

Description

[Technical field]

[0001] The present invention relates to a pipeline set generating program, a pipeline set generating method, and an information processing device. [Background technology]

[0002] In analysis using machine learning, the machine learning algorithms, such as the machine learning model to be used and the format of the input data, vary depending on the data to be analyzed and the purpose. In addition, the prediction accuracy of the machine learning model can be improved by appropriately tuning the hyperparameters. Therefore, in the past, when performing analysis using machine learning, data processing and formatting, feature engineering, hyperparameter optimization, and machine learning model design were performed manually by machine learning experts.

[0003] Thus, to properly perform analysis using machine learning, advanced data science knowledge and techniques are required. This has made it difficult for general users to use machine learning. Therefore, research is being conducted into automating machine learning using AutoML (Automated Machine Learning), which enables analysis using machine learning without advanced techniques or knowledge. AutoML automatically processes and formats data, engineer features, and generates machine learning models.

[0004] Here, a series of processes including multiple preprocessing steps such as data processing and shaping, feature engineering, and hyperparameter adjustment, as well as prediction using the data generated in each process and a machine learning model, is called a "pipeline." In other words, a pipeline represents a series of processes in which prediction processing is performed using a learner including a machine learning model after zero or more preprocessing steps. The pipeline includes hyperparameters for each process.

[0005] In addition, a collection of information to be predicted, which includes a dataset, a target variable, an evaluation index, etc., is called a "task." In other words, AutoML determines an appropriate pipeline for a specific input task. However, the appropriate pipeline here is a pipeline that has a certain degree of prediction accuracy, and may not be the optimal pipeline.

[0006] In AutoML, by retaining a pipeline set of a small number of robust pipelines that have a certain degree of prediction accuracy for many tasks, it becomes possible to quickly determine an appropriate pipeline for a specific task. Therefore, retaining a pipeline set of a small number of robust pipelines that have a certain degree of prediction accuracy for many tasks as pipelines to be selected is a promising method for enabling AutoML to quickly determine an appropriate pipeline.

[0007] One technique for retaining a set of candidate pipelines is to run AutoML on a large number of tasks and select a subset of the suitable pipelines for each task selected by AutoML.

[0008] There are other techniques for automating machine learning, such as the following. For example, a technique has been proposed in which a set corresponding to a new project is searched for among existing projects based on a new dataset and a task, and a new pipeline generated by merging the pipelines of the search results projects is applied to the new project. A technique has also been proposed in which machine learning models are generated using multiple initial machine learning pipelines determined based on the type of training data and the model category, and the machine learning model to be used is determined from the evaluation results of each machine learning model. A technique has also been proposed in which a group of pipelines is determined based on pipeline configuration metadata, a hyperparameter set is generated for each pipeline, and a candidate pipeline is selected by ranking the performance of each pipeline. A technique has also been proposed in which a target column in a provided dataset is identified, and an optimized pipeline is generated based on the identified target column and the provided search budget. [Prior art documents] [Patent documents]

[0009] [Patent Document 1] JP 2022-42497 A [Patent Document 2] Patent Publication No. 2021-2315 [Patent Document 3] US Patent Application Publication No. 2022 / 0051049 [Patent Document 4] US Patent Application Publication No. 2021 / 0097444 Summary of the Invention [Problem to be solved by the invention]

[0010] However, a suitable pipeline for a specific task is not necessarily a robust pipeline. In other words, a suitable pipeline for a specific task is specialized for that specific task and may not be robust when other pipelines are included. Therefore, a set of pipelines obtained as a subset of a set of suitable pipelines for each task may not be sufficiently robust. Therefore, it is difficult to improve the convenience of automating machine learning.

[0011] In addition, even in a technology that merges pipelines of projects searched based on a new data set, etc., to generate a new pipeline and adapt it to the new project, robustness is not ensured by merging, and it is difficult to obtain a robust pipeline set. In addition, none of the technology that determines the machine learning model to be used from the model evaluation results, the technology that ranks the performance of the pipeline to select candidates, and the technology that generates a pipeline based on the target sequence and the search budget take robustness into consideration. Therefore, it is difficult to obtain a robust pipeline set using any of the technologies. Therefore, it is difficult to improve the convenience of automating machine learning.

[0012] The disclosed technology has been made in consideration of the above, and aims to provide a pipeline set generation program, a pipeline set generation method, and an information processing device that improve the convenience of automating machine learning. [Means for solving the problem]

[0013] In one aspect of the pipeline set generation program, pipeline set generation method, and information processing device disclosed in the present application, a computer is caused to execute a process of obtaining a pipeline set based on a plurality of tasks, in which each pipeline includes a machine learning model, performing a simplification process on a first pipeline of the pipeline set, the simplification process including at least one of a process of deleting components included in the pipeline and a process of changing hyperparameters of the components included in the pipeline to default values, thereby generating a second pipeline, executing the second pipeline for the plurality of tasks, obtaining an evaluation value of the second pipeline, and adding the second pipeline to the pipeline set based on the evaluation value. Effect of the Invention

[0014] In one aspect, the present invention can improve the convenience of automating machine learning. [Brief description of the drawings]

[0015] [Figure 1] FIG. 1 is a block diagram of an information processing apparatus according to an embodiment. [Diagram 2] FIG. 2 is a diagram illustrating an example of a pipeline. [Diagram 3] FIG. 3 is a diagram illustrating an example of generation of an initial pipeline set. [Figure 4] FIG. 4 is a diagram showing an example of a simplified pipeline. [Diagram 5] FIG. 5 is a diagram illustrating an example of a pipeline replacement process. [Figure 6] FIG. 6 is a flowchart of a process for generating a pipeline set of a robust pipeline according to an embodiment. [Figure 7] FIG. 7 is a diagram showing the hardware configuration of a computer. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0016] Hereinafter, the pipeline set generating program, the pipeline set generating method, and the information processing device disclosed in the present application will be described in detail with reference to the drawings. Note that the pipeline set generating program, the pipeline set generating method, and the information processing device disclosed in the present application are not limited to the following embodiments. EXAMPLES

[0017] 1 is a block diagram of an information processing device according to an embodiment. The information processing device 1 is connected to a user terminal 2 which is an input source of data and an output destination of a processing result. The information processing device 1 includes a control unit 10, a receiving unit 11, and an output unit 12.

[0018] The reception unit 11 receives input of a task set and an AutoML program from the user terminal 2. The AutoML program is a program that the user uses for analysis using machine learning. The task set is a collection of multiple tasks. The tasks are preferably similar to tasks that the user uses when performing machine learning using the AutoML program. The AutoML program outputs a suitable pipeline based on the input tasks. A suitable pipeline for a particular task is a pipeline that has the best predictive performance among pipelines that can be generated for a particular task.

[0019] Fig. 2 is a diagram showing an example of a pipeline. Here, the pipeline used in the following description will be described with reference to Fig. 2. The pipeline 200 in Fig. 2 is an example of a pipeline that determines whether or not a person is suffering from a disease from a given data set, with the objective variable being "disease."

[0020] The pipeline 200 includes components 201 to 203. The component 201 executes a process of filling in missing values ​​in a given data set. "strategy='median'" in the component 201 represents a hyperparameter. The component 202 executes a process of normalizing the data so that the average is 0 and the standard deviation is 1. The component 203 is a learning device. C=0.1234 in the component 203 is a hyperparameter.

[0021] For example, the pipeline 200 receives the height, weight, and age data set shown in table 211 as input and outputs whether or not each combination of height, weight, and age has a disease. In this case, the component 201 fills in missing values ​​in table 211 and converts the data set into the data set shown in table 212.

[0022] Next, each data in table 212 is normalized by part 202, and the data set is converted into the data set shown in table 213. In this case, since disease is the objective variable, preprocessing is not applied. After that, the data set including height, weight, and age that has been preprocessed in part 203 is input, and a judgment result on the occurrence of the disease is output from the learning device.

[0023] Continuing the explanation by returning to Fig. 1, the reception unit 11 outputs the received task set and AutoML program to the execution unit 101 of the control unit 10.

[0024] Here, in this embodiment, when a suitable pipeline obtained by AutoML for a certain task is determined, the prediction performance of the target pipeline to be evaluated is expressed as a deviation corresponding to the difference from the prediction performance of the suitable pipeline. In detail, the deviation is an index indicating how much the prediction performance when the target pipeline is used for a specific task deviates from the prediction performance when a suitable pipeline for the specific task is used. That is, the deviation is expressed by a formula: "deviation=(prediction performance of suitable pipeline)-(prediction performance of target pipeline)". The deviation is also sometimes called "regret".

[0025] The reception unit 11 also receives information on a target deviation, which is a target value of the deviation for each task, from the user terminal 2. Here, the target deviation may be different for each task or may be the same. The reception unit 11 then outputs the received target deviation to the initial pipeline set generation unit 102 of the control unit 10. This target deviation may be input in advance.

[0026] The control unit 10 acquires a subset from the pipeline set of suitable pipelines obtained from the AutoML program for each task of the task set. Next, the control unit 10 executes a simplification process for a first pipeline included in the pipeline set, the simplification process including at least one of a process of deleting components included in the pipeline and a process of changing hyperparameters of the components included in the pipeline to default values, to generate a second pipeline. Next, the control unit 10 executes the second pipeline for a plurality of tasks to acquire an evaluation value of the second pipeline. Then, the control unit 10 adds the second pipeline to the pipeline set based on the evaluation value to generate a new pipeline set.

[0027] The control unit 10 will be described in detail below. The control unit 10 includes an execution unit 101, an initial pipeline set generation unit 102, a modified pipeline generation unit 103, and a replacement unit 104, as shown in FIG.

[0028] The execution unit 101 receives an input of a task set and an AutoML program from the reception unit 11. Next, the execution unit 101 executes the AutoML program for each task included in the task set. Then, the execution unit 101 obtains a suitable pipeline for each task that is output as a result of the execution of the AutoML program. Then, the execution unit 101 outputs the obtained suitable pipeline for each task to the initial pipeline set generation unit 102.

[0029] The initial pipeline set generation unit 102 receives an input of a suitable pipeline for each task from the execution unit 101. The initial pipeline set generation unit 102 also receives an input of a target deviation degree from the reception unit 11.

[0030] Next, the initial pipeline set generation unit 102 calculates an error representing the difference between the deviation and the target deviation for each pair of a task and a pipeline. The minimum value of the error is set to 0. That is, the error can be expressed as an equation: "error = max (deviation - target deviation, 0)". Here, "max" is a function that selects the larger value of two elements. The error is sometimes called "loss". Here, when the target deviation is set to 0, that is, when the deviation matches the predicted performance of a suitable pipeline, the error matches the deviation.

[0031] Next, the initial pipeline set generation unit 102 initializes the pipeline set with an empty set. Next, the initial pipeline set generation unit 102 repeats the following process to generate an initial pipeline set. Specifically, the initial pipeline set generation unit 102 adds, one by one, each of the suitable pipelines for each task that is not included in the pipeline set to the pipeline set. Then, for each pipeline set to which a different pipeline is added, the initial pipeline set generation unit 102 calculates a total error by adding up the minimum value of the error for each task included in the pipeline set for all tasks. Then, the initial pipeline set generation unit 102 adds the pipeline with the smallest total error to the pipeline set.

[0032] The initial pipeline set generation unit 102 repeats adding pipelines to the pipeline set until a predetermined generation end condition of the initial pipeline set is satisfied. The generation end condition is set based on, for example, an upper limit on the number of pipelines included in the pipeline set, an elapsed time, and the like. Then, the initial pipeline set generation unit 102 outputs the pipeline set at the time when the generation end condition is satisfied as an initial pipeline set to the modified pipeline generation unit 103 and the replacement unit 104. In addition, the initial pipeline set generation unit 102 outputs all tasks and suitable pipeline prediction performance for all tasks to the modified pipeline generation unit 103. This initial pipeline set is an example of a "pipeline set obtained based on a plurality of tasks, each pipeline including a machine learning model."

[0033] Fig. 3 is a diagram showing an example of generation of an initial pipeline set. Here, an example of generation of an initial pipeline set by the initial pipeline set generation unit 102 will be described with reference to Fig. 3. Here, a case will be described in which the target deviation is 0, that is, the error coincides with the deviation. Also, here, there are tasks T1 to T4 and pipelines P1 to P4 which are suitable pipelines for each. Tables 121 and 122 in Fig. 3 show the error for each pair of tasks T1 to T4 and pipelines P1 to P4.

[0034] When the pipeline set is an empty set, the initial pipeline set generation unit 102 calculates the errors for the tasks T1 to T4 of the pipelines P1 to P4 as shown in Table 121. In this case, since the pipeline set is an empty set, the minimum value of the error for each task T1 to T4 included in the pipeline set when each of the pipelines P1 to P4 is added to the pipeline set one by one matches the error of each of the pipelines P1 to P4. Therefore, the initial pipeline set generation unit 102 calculates the total error by simply calculating the sum of the errors for each of the pipelines P1 to P4. In this case, since the total error for the pipeline P2 is the smallest, the initial pipeline set generation unit 102 adds the pipeline P2 to the pipeline set.

[0035] Next, the initial pipeline set generation unit 102 adds each of the pipelines P1, P3, and P4 to the pipeline set including the pipeline P2, and obtains the minimum error value for each of the tasks T1 to T4. Table 122 shows the minimum error value for each of the tasks T1 to T4 when the pipelines P1, P3, and P4 are added to the pipeline set. The grayed-out part in table 122 shows the error of the pipeline P2 because the error of the pipeline P2 is the minimum value. The other part shows the error of the pipeline P1, P3, or P4 because the error of the added pipeline P1, P3, or P4 is smaller than the error of the pipeline P2. Then, the initial pipeline set generation unit 102 adds the errors in table 122 for each of the pipelines P1, P3, or P4 to obtain the sum of errors. In this case, the sum of errors of the pipeline P4 is the minimum, so the initial pipeline set generation unit 102 adds the pipeline P4 to the pipeline set. That is, the pipeline set includes the pipelines P2 and P4.

[0036] For example, in table 122, adding pipeline P3 results in zero error for tasks T2 and T3, and thus higher prediction performance for some tasks. However, adding pipeline P4 improves prediction performance for tasks T1 to T4 as a whole compared to adding pipeline P3, and the initial pipeline set generation unit 102 is able to generate a pipeline set of more robust pipelines.

[0037] Returning to FIG. 1, the explanation will be continued. The modified pipeline generation unit 103 receives an input of an initial pipeline set from the initial pipeline set generation unit 102. Next, the modified pipeline generation unit 103 randomly selects one pipeline from the initial pipeline set. This selected pipeline is an example of a "first pipeline." Then, the modified pipeline generation unit 103 executes a simplification process that applies changes to the selected pipeline to simplify the pipeline, thereby generating a modified pipeline. This modified pipeline is an example of a "second pipeline."

[0038] The simplification process includes, for example, a change to delete one pre-processing component, a change to set the hyperparameter of a component to a default value, etc. The simplification process also includes a change to change the learning device to a learning device with a simpler configuration, a change to a hyperparameter that determines the complexity of the pipeline in a direction to simplify the complexity of the pipeline, etc. Here, the components correspond to each stage of processing included in the pipeline.

[0039] By deleting one preprocessing component, one processing stage is reduced, and therefore the pipeline can be said to be simplified. In addition, since the default values ​​used for the hyperparameters of the components are simpler than the values ​​obtained by learning, it can be said that the pipeline is simplified by setting the hyperparameters of the components to their default values. In addition, changes to make the learning device a learning device with a simpler configuration include, for example, changing the multilayer perceptron to a linear model. In addition, changes to make the hyperparameters that determine the complexity of the machine learning model less complex include increasing the value of the regularization term (e.g., increasing C in LogisticRegression) and decreasing the depth of the decision tree.

[0040] Fig. 4 is a diagram showing an example of simplification of a pipeline. Some examples of simplification of a pipeline will now be described with reference to Fig. 4. Here, a case where the same pipeline 200 as in Fig. 3 is simplified will be described.

[0041] For example, the modified pipeline generation unit 103 generates a modified pipeline 221 by making changes to the hyperparameters of the part 201 of the pipeline 200 so as to set them to default values. In the part 201 of the modified pipeline 221, the hyperparameter "strategy='median'" that was set in the part 201 of the pipeline 200 is removed and changed to the default value.

[0042] Furthermore, the modified pipeline generation unit 103 generates a modified pipeline 222 by making a change to delete the component 202 of the pipeline 200. The modified pipeline 222 has the component 202 removed and includes the components 201 and 203.

[0043] Furthermore, the modified pipeline generation unit 103 generates a modified pipeline 223 by making changes to the hyperparameters of the part 203 of the pipeline 200 so as to set them to default values. In the part 203 of the modified pipeline 223, the hyperparameter "c=0.1234" set in the part 203 of the pipeline 200 is removed and changed to the default value.

[0044] Then, the modified pipeline generation unit 103 outputs the generated modified pipeline to the replacement unit 104. Thereafter, upon receiving an instruction to generate a modified pipeline from the replacement unit 104, the modified pipeline generation unit 103 randomly selects a pipeline again, performs simplification processing on the selected pipeline, creates a modified pipeline, and outputs the modified pipeline to the replacement unit 104.

[0045] The replacement unit 104 receives inputs of the initial pipeline set, all tasks, and predicted performance of a suitable pipeline for all tasks from the initial pipeline set generator 102. The replacement unit 104 also receives inputs of a modified pipeline from the modified pipeline generator 103.

[0046] Next, the replacement unit 104 calculates the deviation for each task of the modified pipeline. Next, the replacement unit 104 detects one pipeline in the initial pipeline set that improves the sum of errors when changed to the modified pipeline. Then, the replacement unit 104 replaces the detected pipeline with the modified pipeline to generate a new pipeline set. Here, if there are multiple pipelines that improve the sum of errors, the replacement unit 104 can select the pipeline that reduces the sum of errors as the pipeline to be replaced.

[0047] Here, the suitable pipeline obtained by AutoML is an example of a "reference pipeline". The deviation or the sum of errors is an example of an "evaluation value". The replacement of the pipeline included in the pipeline set with the modified pipeline by the replacement unit 104 is an example of "adding the second pipeline to the pipeline set". The replacement can be considered as a procedure of adding the modified pipeline to the pipeline set and then deleting the corresponding original pipeline, and can be considered as a form of adding the modified pipeline. In this embodiment, the replacement is performed to make the pipeline set a fixed number, but the robustness of the pipelines included in the pipeline set is improved even if the modified pipeline is simply added to the pipeline set.

[0048] Thereafter, the replacing unit 104 requests the modified pipeline generating unit 103 to generate a modified pipeline, and obtains a new modified pipeline. At this time, the replacing unit 104 may request the generation of a modified pipeline based on the newly generated pipeline set, or may request the generation of a modified pipeline based on the initial pipeline set. Then, the replacing unit 104 replaces the pipelines included in the pipeline set with the new modified pipeline.

[0049] The replacement unit 104 repeats the replacement of the pipeline until a termination condition of the replacement process is satisfied. The termination condition of the replacement process is, for example, whether the replacement has been repeated a specified number of times or whether a specified time has elapsed. Then, when the termination condition of the replacement is satisfied, the replacement unit 104 outputs the pipeline set at that time to the output unit 12 as a pipeline set of a robust pipeline.

[0050] 5 is a diagram showing an example of a pipeline replacement process. Here, a case will be described in which pipelines P2 and P4 shown in table 311 are included in the pipeline set. The replacement unit 104 calculates the sum of errors of the original pipeline set as 0.3+0.0+0.3+0.0=0.6.

[0051] Then, the replacing unit 104 acquires, for example, pipeline #1, which is the modified pipeline 312. The replacing unit 104 calculates the sum of errors when replacing pipeline #1 with pipeline P2 as 0.2+0.4+0.3+0.0=0.9. The replacing unit 104 also calculates the sum of errors when replacing pipeline #1 with pipeline P4 as 0.2+0.0+0.3+0.2=0.7. In either case, the original sum of errors is not improved, so the replacing unit 104 does not replace the pipeline.

[0052] Furthermore, the replacement unit 104 acquires, for example, the pipeline #2 which is the modified pipeline 313. The replacement unit 104 calculates the sum of errors when the pipeline #2 is replaced with the pipeline P2 as 0.1+0.1+0.3+0.0=0.5. The replacement unit 104 calculates the sum of errors when the pipeline #2 is replaced with the pipeline P4 as 0.1+0.0+0.5+0.5=1.1. In this case, since the sum of errors is improved by replacing the pipeline #2 with the pipeline P2, the replacement unit 104 replaces the pipeline #2 with the pipeline P2 to generate a new pipeline set.

[0053] The output unit 12 receives an input of the pipeline set of the robust pipeline from the replacement unit 104. Then, the output unit 12 transmits the acquired pipeline set to the user terminal 2 to notify the user. The user can acquire a pipeline having a certain degree of prediction performance in a short time by executing an AutoML program using the pipeline set of the robust pipeline notified from the information processing device 1.

[0054] 6 is a flowchart of a process for generating a pipeline set of a robust pipeline according to an embodiment. Next, a flow of a process for generating a pipeline set of a robust pipeline by the information processing device 1 according to an embodiment will be described with reference to FIG.

[0055] A user inputs a task set and an AutoML program to the information processing device 1 using the user terminal 2 (step S1).

[0056] The reception unit 11 receives a task set and an AutoML program input from the user terminal 2, and outputs them to the execution unit 101. The execution unit 101 executes AutoML for all tasks in the task set, and obtains a suitable pipeline for each task (step S2).

[0057] The initial pipeline set generation unit 102 adds each suitable pipeline that is not included in the pipeline set to the pipeline set one by one, and calculates a total error by adding the minimum value of the errors included in the pipeline set for all tasks.The initial pipeline set generation unit 102 then repeats the process of adding the pipeline with the minimum total error to the pipeline set to generate an initial pipeline set that is a first pipeline set (step S3).

[0058] The modified pipeline generation unit 103 randomly selects one pipeline from the set of pipelines (step S4).

[0059] Next, the modified pipeline generator 103 applies simplifying changes to the selected pipeline to generate a modified pipeline (step S5).

[0060] The replacement unit 104 calculates the deviation of the modified pipeline for each task (step S6).

[0061] Next, the replacement unit 104 identifies a pipeline in the pipeline set that can be changed to a modified pipeline to improve the sum of errors, and then replaces the identified pipeline with the modified pipeline to generate a new pipeline set (step S7).

[0062] After that, the replacement unit 104 judges whether or not the termination condition of the replacement process is satisfied (step S8). If the termination condition is not satisfied (step S8: No), the replacement unit 104 requests the modified pipeline generation unit 103 to create a modified pipeline, and the process of generating a pipeline set of a robust pipeline returns to step S4.

[0063] On the other hand, if the termination condition is satisfied (step S8: Yes), the replacement unit 104 outputs the generated pipeline set to the output unit 12. The output unit 12 transmits the pipeline set of the robust pipeline input from the replacement unit 104 to the user terminal 2 (step S9).

[0064] (Hardware configuration) Fig. 7 is a hardware configuration diagram of a computer. The information processing device 1 can be realized by a computer 90 shown in Fig. 7. Next, an example of the hardware configuration of the information processing device 1 will be described with reference to Fig. 7.

[0065] 7, a computer 90 includes, for example, a CPU (Central Processing Unit) 91, a memory 92, a hard disk 93, and a network interface 94. The CPU 91 is connected to the memory 92, the hard disk 93, and the network interface 94 via a bus.

[0066] The network interface 94 is an interface for communication between the computer 90 and an external device. The network interface 94 relays communication between the user terminal 2 and the CPU 91, for example. The network interface 94 realizes the functions of the reception unit 11 and the output unit 12.

[0067] The hard disk 93 is an auxiliary storage device that stores various programs including a program for implementing the functions of the control unit 10 including the execution unit 101, the initial pipeline set generation unit 102, the modified pipeline generation unit 103, and the replacement unit 104 illustrated in FIG.

[0068] The memory 92 is a main storage device, and may be, for example, a dynamic random access memory (DRAM).

[0069] The CPU 91 reads out various programs from the hard disk 93, and loads and executes them in the memory 92. In this way, the CPU 91 realizes the functions of the control unit 10 including the execution unit 101, the initial pipeline set generation unit 102, the modified pipeline generation unit 103, and the replacement unit 104 illustrated in FIG.

[0070] As described above, the information processing device according to the present embodiment selects a pipeline from among suitable pipelines obtained by AutoML for each task, and generates a pipeline set of robust pipelines. Furthermore, the information processing device performs modification to simplify a pipeline included in the generated pipeline set, and generates a modified pipeline. Then, when replacement with the modified pipeline improves robustness, the information processing device replaces a pipeline included in the pipeline set with the modified pipeline, and generates a pipeline set of more robust pipelines.

[0071] This allows the user to receive a pipeline set of robust pipelines from the information processing device, and to easily obtain a pipeline having a certain degree of predictive performance by executing automation of machine learning using the pipeline set. Therefore, the convenience of automating machine learning can be improved. [Explanation of symbols]

[0072] 1. Information processing device 2. User terminal 10 Control section 11 Reception 12 Output section 101 Executive Department 102 Initial Pipeline Set Generation Unit 103 Modified Pipeline Generation Unit 104 Replacement part

Claims

1. Based on the tasks, we obtain a set of pipelines, each of which contains a machine learning model; performing a simplification process on a first pipeline of the pipeline set, the simplification process including at least one of a process of deleting components included in the pipeline and a process of changing hyperparameters of the components included in the pipeline to default values, to generate a second pipeline; Executing the second pipeline for the plurality of tasks to obtain an evaluation value of the second pipeline; adding the second pipeline to the pipeline set based on the evaluation value; A pipeline set generating program for causing a computer to execute a process.

2. 2. The pipeline set generation program according to claim 1, wherein the process of acquiring the evaluation value includes a process of acquiring, for each of the plurality of tasks, a difference between the prediction accuracy of a reference pipeline and the prediction accuracy of the second pipeline as the evaluation value.

3. 2. The pipeline set generation program according to claim 1, wherein the process of acquiring the pipeline set includes a process of acquiring a plurality of pipelines, executing each pipeline included in the plurality of pipelines for each of the plurality of tasks, acquiring the evaluation value for each pipeline included in the plurality of pipelines for each task, and acquiring the pipeline set based on the evaluation value.

4. The pipeline set generation program according to claim 1, characterized in that the simplification process includes a process of changing a learning device to a learning device having a simpler structure or a process of changing a hyperparameter that determines the complexity of a pipeline in a direction that simplifies the complexity of the pipeline.

5. The computer Based on the tasks, we obtain a set of pipelines, each of which contains a machine learning model; performing a simplification process on a first pipeline of the pipeline set, the simplification process including at least one of a process of deleting components included in the pipeline and a process of changing hyperparameters of the components included in the pipeline to default values, to generate a second pipeline; Executing the second pipeline for the plurality of tasks to obtain an evaluation value of the second pipeline; adding the second pipeline to the pipeline set based on the evaluation value; 23. A pipeline set generating method comprising:

6. Based on the tasks, we obtain a set of pipelines, each of which contains a machine learning model; performing a simplification process on a first pipeline of the pipeline set, the simplification process including at least one of a process of deleting components included in the pipeline and a process of changing hyperparameters of the components included in the pipeline to default values, to generate a second pipeline; Executing the second pipeline for the plurality of tasks to obtain an evaluation value of the second pipeline; Adding the second pipeline to the pipeline set based on the evaluation value. A control unit that executes the processing; An information processing device comprising:

Images