Machine learning pipeline component determination program, method, and apparatus

Abstract

To reduce a calculation cost for determining whether or not to add a new component to a search range of automatic machine learning.SOLUTION: A machine learning pipeline component determination apparatus is configured to: obtain a type of a first component; identify, by a program that generates machine learning pipelines including components selected from among a plurality of components depending on a task, one or more first machine learning pipelines including a component of the same type as the type of the first component among a plurality of machine learning pipelines output for a plurality of datasets; generate, for the respective one or more first machine learning pipelines, one or more second machine learning pipelines in which the component of the same type is changed to the first component; and determine whether or not to add the first component to the plurality of components based on a result of comparison between the performance of each of the one or more first machine learning pipelines and the performance of each of the one or more second machine learning pipelines.SELECTED DRAWING: Figure 10

Description

[Technical field]

[0001] The disclosed technology relates to a machine learning pipeline component determination program, a machine learning pipeline component determination method, and a machine learning pipeline component determination device. [Background technology]

[0002] Automated machine learning (AutoML) explores multiple components, such as components that perform data preprocessing and components that perform inference (machine learning models), to automatically generate the optimal machine learning pipeline (hereinafter simply referred to as "pipeline") for executing a task.

[0003] As a technique related to AutoML, for example, a method for categorizing spending data that does not require prior detailed knowledge of an organization's transaction data is proposed. The method applies natural language processing to text data from the transaction data to generate a clean data set (CDS). Logs for transactions are clustered based on similarity to form a minimal data set (MDS). A subset of high-value clusters is selected by requesting a user to manually categorize logs from each cluster in the subset. A model is then trained using the subset of manually categorized clusters and used to predict spending categories for the remaining logs with high accuracy. The AI ​​engine automatically analyzes the predictions based on the client context and either auto-tunes the machine learning model or identifies a new subset of clusters to be manually categorized.

[0004] Also, for example, a data classification system has been proposed that improves the efficiency of creating a classifier in an integrated classifier. This system integrates the classification results of a given data set by each of a plurality of classifiers, and stores the correspondence between the feature values ​​of the data set and the labels that are the classification results as learning data. This system also calculates a simultaneous unclassification rate at which multiple labels are unclassified for one data set based on the feature values ​​and the classification probability output by the integrated classifier learned from the classification results by each classifier for each unlabeled data set in the learning data. This system also calculates a simultaneous classification probability at which multiple labels are classified for one data set. This system then calculates a recommendation score by tallying up the multiplied value of the simultaneous unclassification rate and the simultaneous classification probability for each label, and identifies labels for which a classifier should be additionally created in order of recommendation score, and outputs the result as recommendation information.

[0005] Also, for example, a system for determining suitable hyperparameters for a machine learning model and feature engineering process is proposed, in which suitable machine learning models and associated hyperparameters are determined by analyzing a dataset, and suitable hyperparameter values ​​for compatible machine learning models having one or more common hyperparameters and a compatible dataset schema are identified. The hyperparameters may be ranked according to their respective respective impacts on model performance metrics, and hyperparameter values ​​identified as having greater impacts may be more aggressively searched for.

[0006] Also provided is a method for generating a predictive machine learning model, for example, receiving at a database server a selection of a predictive field from a plurality of fields, and generating, by the database server, a plurality of features from the dataset, where the plurality of features are automatically generated based at least in part on metadata associated with the dataset, and generating, by the database server, a predictive machine learning model based at least in part on the plurality of features, and transmitting, by the database server, an indication of a plurality of predicted values ​​for the predictive field based at least in part on the predictive machine learning model. [Prior art documents] [Patent documents]

[0007] [Patent Document 1] JP 2020-115346 A [Patent Document 2] JP 2020-008992 A [Patent Document 3] US Patent Application Publication No. 2020 / 0057958 [Patent Document 4] US Patent Application Publication No. 2019 / 0138946 Summary of the Invention [Problem to be solved by the invention]

[0008] In AutoML, it is important to consider what parts to include in the search range, which is a collection of parts. When a new part is created that is not included in the existing search range, it is necessary to determine whether or not the part should be added to the search range. For example, if a part can only perform the processes possible with parts included in the existing search range, there is no need to add it to the search range. It is desirable to be able to automatically determine whether or not such a new part can be added to the search range.

[0009] Therefore, it is possible to run AutoML on a large number of datasets, on the search range before and after adding the target component, and determine whether or not to add the target component based on whether the performance of the pipeline generated before and after adding the target component improves.

[0010] However, running AutoML on all combinations of multiple parts included in the search range before and after adding the target part and evaluating the performance of the generated pipeline is computationally very expensive.

[0011] In one aspect, the disclosed technology aims to reduce the computational cost of determining whether or not to add a new part to a search range of automatic machine learning. [Means for solving the problem]

[0012] In one embodiment, the disclosed technology obtains a type of a first part. The disclosed technology also identifies a first one or more machine learning pipelines including a part of the same type as the type of the first part among a plurality of machine learning pipelines output by an automatic machine learning program for a plurality of data sets. The automatic machine learning program is a program that generates a machine learning pipeline including a part selected from a plurality of parts according to a task. The disclosed technology also generates a second one or more machine learning pipelines for each of the first one or more machine learning pipelines by replacing the part of the same type with the first part. The disclosed technology then determines whether to add the first part to the plurality of parts based on a result of comparing the performance of each of the first one or more machine learning pipelines with the performance of each of the second one or more machine learning pipelines. Effect of the Invention

[0013] One aspect of the present invention is that it has an effect of reducing the computational cost for determining whether or not a new part can be added to the search range of automatic machine learning. [Brief description of the drawings]

[0014] [Figure 1] FIG. 13 is a diagram for explaining a reference technique. [Diagram 2] FIG. 2 is a functional block diagram of a machine learning pipeline component determination device. [Diagram 3] FIG. 13 is a diagram illustrating an example of a subset DB. [Figure 4] FIG. 1 illustrates an example of classification of parts into subsets. [Diagram 5] FIG. 13 illustrates an example of a performance DB. [Figure 6] FIG. 11 is a diagram for explaining an example of metadata calculation. [Figure 7] FIG. 13 is a diagram showing an example of metadata for each part for each data set. [Figure 8] FIG. 13 is a diagram illustrating an example of aggregated metadata. [Figure 9] FIG. 13 is a diagram for explaining acquisition of a subset for a target part. [Figure 10] FIG. 1 is a diagram showing an outline of the present embodiment. [Figure 11] FIG. 1 is a block diagram showing a schematic configuration of a computer that functions as a machine learning pipeline component determination device. [Figure 12] 13 is a flowchart illustrating an example of a preparation process. [Figure 13] 13 is a flowchart illustrating an example of a determination process. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0015] Hereinafter, an example of an embodiment of the disclosed technology will be described with reference to the drawings.

[0016] Before describing the details of the embodiment, a reference method that is assumed for determining whether or not a new part can be added to the search range of AutoML will be described.

[0017] As shown in FIG. 1, an information processing device that executes the reference method executes AutoML for a large number of data sets in the search ranges before and after adding a target component. The search range is a collection of multiple components, such as components that perform preprocessing on data and components that perform inference (machine learning models). By executing AutoML, a pipeline is generated for all combinations of components included in the search range. A pipeline is an entire system that executes tasks using components that perform preprocessing and components that perform inference, and indicates which components are used in which order. In the following explanation, it is assumed that the target component is a component that performs preprocessing, and the component that performs preprocessing may be simply referred to as a "component."

[0018] Furthermore, the information processing device executes AutoML to identify the best performance among the performances of a plurality of pipelines generated from the search range before the addition of the target component for each dataset. Similarly, the information processing device identifies the best performance among the performances of a plurality of pipelines generated from the search range after the addition of the target component for each dataset. Note that performance may refer to, for example, the accuracy of a prediction result output from a pipeline. Then, the information processing device evaluates whether or not the performance of each of the plurality of datasets has improved before and after the addition of the target component. If the performance has improved in any of the datasets, the information processing device determines to add the target component to the search range.

[0019] As described above, evaluating the performance of all pipelines configured with all combinations of components included in the search range before and after the addition of the target component for all data sets requires an enormous computational cost.

[0020] Therefore, in this embodiment, among the pipelines configured by the combination of parts included in the search range, the pipelines to be evaluated are narrowed down to be evaluated, thereby reducing the calculation cost. The machine learning pipeline part determination device according to this embodiment will be described in detail below.

[0021] 2, the machine learning pipeline component determination device 10 functionally includes a control unit 12. The control unit 12 includes a preparation unit 14, an acquisition unit 16, an identification unit 18, a generation unit 20, and a determination unit 22. In addition, a subset DB (Database) 34, a performance DB 36, and a classification model 38 are stored in a predetermined storage area of ​​the machine learning pipeline component determination device 10.

[0022] The preparation unit 14 acquires a dataset set 30 and a parts set 32. The dataset set 30 includes a plurality of datasets. Each dataset includes a plurality of data consisting of features for each of one or more columns. The parts set 32 ​​is a set of parts that are the search range of AutoML. As described above, in this embodiment, the target parts for additional determination are parts that perform preprocessing on data, and the parts included in the parts set 32 ​​are also parts that perform preprocessing. Note that a part that performs inference (machine learning model) is also necessary to generate a pipeline, which will be described later, but in this embodiment, the description of the parts set 32 ​​is omitted.

[0023] The preparation unit 14 classifies each part included in the part set 32 ​​into one of a plurality of subsets. The subset is an example of a "type" of the disclosed technology. For example, the preparation unit 14 may randomly classify the parts into each subset so that the parts are evenly classified into each subset. Furthermore, when each part is provided in a library such as Python, the preparation unit 14 may classify the part into each subset based on the package to which the part belongs in the library.

[0024] As shown in FIG. 3, the preparation unit 14 stores in the subset DB 34 a part ID, which is identification information of a part, and a subset ID, which is identification information of a subset, in association with each other. In the example of FIG. 3, each box represents one part, and "Pi" in the box is a part ID. In the example of FIG. 3, i=1, 2, 3, 4, 5. In addition, in the example of FIG. 3, an example is shown in which each part is classified into three subsets. In addition, for the classified parts, a subset ID (A, B, and C in the example of FIG. 3) is added in parentheses following the part ID. FIG. 4 shows an example in which each part is classified into a plurality of subsets (A, B, C, and D in the example of FIG. 4) using preprocessing parts in scikit-learn, an open source machine learning library implemented in Python, as an example.

[0025] Furthermore, the preparation unit 14 executes AutoML for each of the multiple datasets included in the dataset set 30, and identifies the pipeline with the highest performance (hereinafter referred to as the "best pipeline") and its performance. For example, as shown in Fig. 5, the preparation unit 14 stores the identified best pipeline and performance for each dataset in the performance DB 36 in association with each other. Note that in Fig. 5, the components indicated by M1 and M2 are components that execute inference (machine learning models).

[0026] Furthermore, the preparation unit 14 trains a classification model 38, which is a machine learning model for classifying subsets of parts, using training data in which the features of each of the multiple parts included in the parts set 32 ​​are associated with the subsets into which the parts are classified. When the features of a part are input, the classification model 38 outputs a subset into which the part is classified.

[0027] Specifically, the preparation unit 14 calculates, as a feature, metadata indicating a change in each data set when each component is applied to each data set. For example, as shown in Fig. 6, the preparation unit 14 calculates, as metadata, a change in the number of data items with NaN values ​​included in a data set after applying the component, i.e., an increase, no change, or a decrease, relative to the number of data items with NaN values ​​included in the data set before applying the pre-processing component.

[0028] The preparation unit 14 may calculate other metadata such as, for example, a change in the total number of columns before and after application of component preprocessing to the dataset, the type of the changed column, etc. The preparation unit 14 may also calculate a change in the total number of instances, a frequency of column values, a maximum value of the changed column, a minimum value of the changed column, a standard deviation of the changed column, a ratio of the median to the maximum value, a ratio of the median to the minimum value, a frequency of 0 in the changed column, etc.

[0029] The preparation unit 14 calculates metadata for each part for each data set, as shown in FIG. 7. Then, the preparation unit 14 aggregates the metadata of each data set for each part, as shown in FIG. 8, to generate aggregated metadata. Specifically, the preparation unit 14 generates aggregated metadata by horizontally connecting the metadata for each data set for each part. In the example of FIG. 8, there are N data sets. Furthermore, the number of columns in data set 1 is M1, the number of columns in data set 2 is M2, ..., and the number of columns in data set N is MN. Furthermore, when calculating X types of metadata, the metadata for each column is metadata md1, metadata md2, ..., metadata mdX.

[0030] The preparation unit 14 also assigns the subset ID of the subset into which each part is classified as a correct label to the aggregated metadata of each part to generate training data. The preparation unit 14 then trains the classification model 38 so that the classification result of the subset output when the aggregated metadata is input to the classification model 38 matches the correct label.

[0031] The acquisition unit 16 acquires a target part for additional judgment, and acquires a subset of the target part. The target part is an example of a "first part" in the disclosed technology. Specifically, as shown in FIG. 9, the acquisition unit 16 calculates metadata for the target part in the same manner as the processing by the preparation unit 14 described above, and generates aggregated metadata. In FIG. 9, the part ID for the target part is set to "new". The acquisition unit 16 inputs the generated aggregated metadata to the classification model 38 trained by the preparation unit 14, and acquires a classification result for the subset of the target part.

[0032] The identification unit 18 identifies, from among the best pipelines stored in the performance DB 36, one or more pipelines that include parts of the same subset as the subset of the target part as a pipeline to be evaluated. The pipeline to be evaluated is an example of the "first one or more machine learning pipelines" of the disclosed technology. Specifically, the identification unit 18 refers to the subset DB 34 to identify parts included in the same subset as the subset of the target part acquired by the acquisition unit 16. Then, the identification unit 18 searches the best pipelines stored in the performance DB 36 as candidates to be evaluated, and identifies the best pipeline of the evaluation candidate that includes the identified parts as the best pipeline to be evaluated.

[0033] For example, assume that subset C is acquired as a subset of the target parts. In this case, as shown in Fig. 10, the identification unit 18 identifies, among the best pipelines that are candidates for evaluation targets, the best pipeline for dataset 1 that includes parts of subset C as the evaluation target. Furthermore, the identification unit 18 does not identify, as the evaluation target, the best pipeline for dataset 2 that does not include parts of subset C, i.e., excludes it from the evaluation target.

[0034] The generation unit 20 generates a replacement pipeline in which, for each pipeline to be evaluated, a part in the same subset as the target part is changed to the target part. The replacement pipeline is an example of the "second one or more machine learning pipelines" of the disclosed technology. In the example of FIG. 10, the generation unit 20 generates a replacement pipeline by changing the part "P5(C)" included in the best pipeline for dataset 1 to the target part "Pnew(C)".

[0035] The determination unit 22 determines whether or not to add the target part to the search range of AutoML based on a result of comparing the performance of each of the pipelines to be evaluated with the performance of each of the replacement pipelines. Specifically, the determination unit 22 evaluates the performance of the replacement pipeline, compares it with the performance of the pipeline to be evaluated, i.e., the pipeline before replacement, and determines that the target part is to be added to the search range of AutoML if the performance has improved. The determination unit 22 determines that the target part is not to be added to the search range of AutoML if the performance has not improved. In the case where a plurality of best pipelines to be evaluated are identified, the determination unit 22 may determine that the target part is to be added to the search range of AutoML, for example, if the performance of any one of the best pipelines to be evaluated improves. The determination unit 22 outputs a determination result of whether or not the target part can be added to the search range of AutoML.

[0036] The machine learning pipeline component determination device 10 may be realized by, for example, a computer 40 shown in FIG. 11. The computer 40 includes a CPU (Central Processing Unit) 41, a GPU (Graphics Processing Unit) 42, a memory 43 as a temporary storage area, and a non-volatile storage device 44. The computer 40 also includes an input / output device 45 such as an input device and a display device, and an R / W (Read / Write) device 46 that controls reading and writing of data to a storage medium 49. The computer 40 also includes a communication I / F (Interface) 47 that is connected to a network such as the Internet. The CPU 41, the GPU 42, the memory 43, the storage device 44, the input / output device 45, the R / W device 46, and the communication I / F 47 are connected to each other via a bus 48.

[0037] The storage device 44 is, for example, a hard disk drive (HDD), a solid state drive (SSD), a flash memory, etc. The storage device 44 as a storage medium stores a machine learning pipeline component determination program 50 for causing the computer 40 to function as the machine learning pipeline component determination device 10. The machine learning pipeline component determination program 50 has a preparation process control instruction 54, an acquisition process control instruction 56, an identification process control instruction 58, a generation process control instruction 60, and a determination process control instruction 62. The storage device 44 also has an information storage area 70 in which information constituting each of the subset DB 34, the performance DB 36, and the classification model 38 is stored.

[0038] The CPU 41 reads out the machine learning pipeline part determination program 50 from the storage device 44, expands it in the memory 43, and sequentially executes the control instructions of the machine learning pipeline part determination program 50. The CPU 41 operates as the preparation unit 14 shown in FIG. 2 by executing the preparation process control instruction 54. The CPU 41 also operates as the acquisition unit 16 shown in FIG. 2 by executing the acquisition process control instruction 56. The CPU 41 also operates as the identification unit 18 shown in FIG. 2 by executing the identification process control instruction 58. The CPU 41 also operates as the generation unit 20 shown in FIG. 2 by executing the generation process control instruction 60. The CPU 41 also operates as the determination unit 22 shown in FIG. 2 by executing the determination process control instruction 62. The CPU 41 also reads out information from the information storage area 70, and expands each of the subset DB 34, the performance DB 36, and the classification model 38 in the memory 43. As a result, the computer 40 that has executed the machine learning pipeline part determination program 50 functions as the machine learning pipeline part determination device 10. The CPU 41 that executes the program is hardware. Also, part of the program may be executed by the GPU .

[0039] The functions realized by the machine learning pipeline component determination program 50 may be realized, for example, by a semiconductor integrated circuit, more specifically, an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or the like.

[0040] Next, the operation of the machine learning pipeline part determination device 10 according to the present embodiment will be described. First, the machine learning pipeline part determination device 10 executes the preparation process shown in Fig. 12. Then, the machine learning pipeline part determination device 10 executes the determination process shown in Fig. 13. Note that the preparation process and the determination process are an example of a machine learning pipeline part determination method of the disclosed technology.

[0041] The preparation process will be described with reference to FIG.

[0042] In step S10, the preparation unit 14 acquires the data set set 30 and the part set 32. Next, in step S12, the preparation unit 14 classifies each part included in the part set 32 ​​into one of a plurality of subsets, and stores the part ID and the subset ID in association with each other in the subset DB 34.

[0043] Next, in step S14, the preparation unit 14 executes AutoML for each of the multiple datasets included in the dataset set 30 to identify the best pipeline and its performance. Then, the preparation unit 14 stores the identified best pipeline and its performance in the performance DB 36 in association with each other for each dataset.

[0044] Next, in step S16, the preparation unit 14 calculates changes in each data set when each part is applied to each data set as metadata. Next, in step S18, the preparation unit 14 aggregates the metadata of each data set for each part to generate aggregated metadata, and assigns the subset ID of the subset into which each part is classified as a correct answer label to the aggregated metadata of each part to generate training data. Next, in step S20, the preparation unit 14 trains the classification model 38 using the training data, and the preparation process ends.

[0045] Next, the determination process will be described with reference to FIG.

[0046] In step S30, the acquisition unit 16 acquires target parts for additional determination. Next, in step S32, the acquisition unit 16 calculates metadata for the target parts and generates aggregated metadata. Next, in step S34, the acquisition unit 16 inputs the generated aggregated metadata to the trained classification model 38 to obtain a classification result for a subset of the target parts.

[0047] Next, in step S36, the identification unit 18 identifies, as a pipeline to be evaluated, one or more pipelines that include parts of the same subset as the subset of the target part, from among the best pipelines stored in the performance DB 36. Next, in step S38, the generation unit 20 generates, for each of the pipelines to be evaluated, a replacement pipeline in which parts of the same subset as the target part are replaced with the target part.

[0048] Next, in step S40, the determination unit 22 evaluates the performance of the replaced pipeline. Next, in step S42, the determination unit 22 compares the performance of the pipeline before and after the replacement and determines whether the performance has improved. If the performance has improved, the process proceeds to step S44, and if the performance has not improved, the process proceeds to step S46.

[0049] In step S44, the determination unit 22 determines that the target part should be added to the search range of AutoML. On the other hand, in step S46, the determination unit 22 determines that the target part should not be added to the search range of AutoML. Next, in step S48, the determination unit 22 outputs a determination result as to whether or not the target part should be added to the search range of AutoML, and the determination process ends.

[0050] As described above, the machine learning pipeline part determination device according to the present embodiment obtains a subset into which the target part of the addition determination is classified. In addition, the machine learning pipeline part determination device identifies a pipeline to be evaluated that includes a part of the same subset as the target part among multiple pipelines output by AutoML for multiple datasets. In addition, the machine learning pipeline part determination device generates a replacement pipeline in which the parts of the same subset are changed to target parts for each pipeline to be evaluated. Then, the machine learning pipeline part determination device determines whether or not to add the target part to the search range of AutoML based on the result of comparing the performance of each pipeline to be evaluated with the performance of each replacement pipeline. This makes it possible to reduce the calculation cost for determining whether or not a new part can be added to the search range of automatic machine learning.

[0051] Specifically, as shown in Fig. 10, in the above-mentioned reference method, the pipelines of all combinations of parts included in the search range before and after the addition of the target part are evaluated. On the other hand, in this embodiment, among the best pipelines for each data set, the pipelines including parts of the same subset as the target part and the replacement pipelines in which the parts of the pipelines are replaced with the target parts are evaluated. In this way, in this embodiment, the evaluation targets for determining whether or not the target part can be added are narrowed down, so that the calculation cost can be reduced.

[0052] In the above embodiment, the subset into which the target part is classified is obtained using a classification model trained in advance, but the present invention is not limited to this. For example, each part is vectorized using the aggregated metadata as described above, and each part is clustered for each subset in a vector space. Then, a subset corresponding to a cluster having a closest distance between the vector indicated by the aggregated metadata of the target part and the center of each cluster may be obtained as the subset into which the target part is classified.

[0053] In the above embodiment, the machine learning pipeline component determination program is stored (installed) in advance in the storage device, but is not limited to this. The program according to the disclosed technology may be provided in a form stored in a storage medium such as a CD-ROM, a DVD-ROM, or a USB memory.

[0054] Regarding the above embodiment, the following supplementary notes are further disclosed.

[0055] (Appendix 1) Obtaining a type of a first part; A program for generating a machine learning pipeline including a part selected from a plurality of parts according to a task identifies a first one or more machine learning pipelines including a part of the same type as the first part among a plurality of machine learning pipelines output for a plurality of datasets; generating, for each of the first one or more machine learning pipelines, a second one or more machine learning pipelines in which the components of the same type are replaced with the first components; determining whether to add the first component to the plurality of components based on a comparison of a performance of each of the first one or more machine learning pipelines with a performance of each of the second one or more machine learning pipelines; A machine learning pipeline component determination program that causes a computer to execute processing.

[0056] (Appendix 2) acquiring the type of the first part includes classifying the plurality of parts into a plurality of types and acquiring the type of the first part by classifying the plurality of types into one of the plurality of types. 2. The machine learning pipeline component determination program according to claim 1.

[0057] (Appendix 3) inputting the features of the first part into a machine learning model that has been trained to output the type of the part when the features of the part are input, using training data that associates the features of each of the plurality of parts with the type into which the part is classified, thereby acquiring the type of the first part; 3. The machine learning pipeline component determination program according to claim 2.

[0058] (Appendix 4) The component is a component that performs pre-processing on a data set, a change in the data set when the part is applied to the data set is used as the feature; The machine learning pipeline component determination program according to claim 3.

[0059] (Appendix 5) The plurality of machine learning pipelines are the machine learning pipelines with the highest performance for each of the plurality of datasets. 5. The machine learning pipeline component determination program according to claim 1,

[0060] (Appendix 6) Obtaining a type of a first part; A program for generating a machine learning pipeline including a part selected from a plurality of parts according to a task identifies a first one or more machine learning pipelines including a part of the same type as the first part among a plurality of machine learning pipelines output for a plurality of datasets; generating, for each of the first one or more machine learning pipelines, a second one or more machine learning pipelines in which the components of the same type are replaced with the first components; determining whether to add the first component to the plurality of components based on a comparison of a performance of each of the first one or more machine learning pipelines with a performance of each of the second one or more machine learning pipelines; A machine learning pipeline component determination method characterized in that processing is executed by a computer.

[0061] (Appendix 7) acquiring the type of the first part includes classifying the plurality of parts into a plurality of types and acquiring the type of the first part by classifying the plurality of types into one of the plurality of types. The machine learning pipeline component determination method according to claim 6,

[0062] (Appendix 8) inputting the features of the first part into a machine learning model that has been trained to output the type of the part when the features of the part are input, using training data that associates the features of each of the plurality of parts with the type into which the part is classified, thereby acquiring the type of the first part; The machine learning pipeline component determination method according to claim 7,

[0063] (Appendix 9) The component is a component that performs pre-processing on a data set, a change in the data set when the part is applied to the data set is used as the feature; The machine learning pipeline component determination method according to claim 8,

[0064] (Appendix 10) The plurality of machine learning pipelines are the machine learning pipelines with the highest performance for each of the plurality of datasets. 10. The machine learning pipeline component determination method according to any one of Supplementary Note 6 to Supplementary Note 9,

[0065] (Appendix 11) Obtaining a type of a first part; A program for generating a machine learning pipeline including a part selected from a plurality of parts according to a task identifies a first one or more machine learning pipelines including a part of the same type as the first part among a plurality of machine learning pipelines output for a plurality of datasets; generating, for each of the first one or more machine learning pipelines, a second one or more machine learning pipelines in which the components of the same type are replaced with the first components; determining whether to add the first component to the plurality of components based on a comparison of a performance of each of the first one or more machine learning pipelines with a performance of each of the second one or more machine learning pipelines; A machine learning pipeline component determination device having a control unit that executes processing.

[0066] (Appendix 12) acquiring the type of the first part includes classifying the plurality of parts into a plurality of types and acquiring the type of the first part by classifying the plurality of types into one of the plurality of types. 12. The machine learning pipeline component determination device according to claim 11,

[0067] (Appendix 13) inputting the features of the first part into a machine learning model that has been trained to output the type of the part when the features of the part are input, using training data that associates the features of each of the plurality of parts with the type into which the part is classified, thereby acquiring the type of the first part; 13. The machine learning pipeline component determination device according to claim 12.

[0068] (Appendix 14) The component is a component that performs pre-processing on a data set, a change in the data set when the part is applied to the data set is used as the feature; 14. The machine learning pipeline component determination device according to claim 13.

[0069] (Appendix 15) The plurality of machine learning pipelines are the machine learning pipelines with the highest performance for each of the plurality of datasets. The machine learning pipeline component determination device according to any one of Supplementary Note 11 to Supplementary Note 14,

[0070] (Appendix 16) Obtaining a type of a first part; A program for generating a machine learning pipeline including a part selected from a plurality of parts according to a task identifies a first one or more machine learning pipelines including a part of the same type as the first part among a plurality of machine learning pipelines output for a plurality of datasets; generating, for each of the first one or more machine learning pipelines, a second one or more machine learning pipelines in which the components of the same type are replaced with the first components; determining whether to add the first component to the plurality of components based on a comparison of a performance of each of the first one or more machine learning pipelines with a performance of each of the second one or more machine learning pipelines; A non-transitory storage medium storing a machine learning pipeline component determination program that causes a computer to execute processing. [Explanation of symbols]

[0071] 10 Machine learning pipeline component judgement device 12 Control section 14 Preparation Department 16 Acquisition Department 18 Specific section 20 Generation part 22 Judgment section 30 Data Set Collection 32 Parts Collection 34 Subset DB 36 Performance DB 38 Classification Models 40 Computer 41 CPU 42 GPU 43 Memory 44 Storage device 45 I / O Devices 46 R / W device 47 Communication I / F 48 Bus 49 Storage medium 50 Machine Learning Pipeline Part Judgment Program 54 Preparation Process Control Instructions 56 Acquisition Process Control Instructions 58 Specific Process Control Order 60 Generation process control instructions 62 Decision process control instructions 70 Information storage area

Claims

1. Obtaining a type of a first part; A program for generating a machine learning pipeline including a part selected from a plurality of parts according to a task identifies a first one or more machine learning pipelines including a part of the same type as the first part among a plurality of machine learning pipelines output for a plurality of datasets; generating, for each of the first one or more machine learning pipelines, a second one or more machine learning pipelines in which the components of the same type are replaced with the first components; determining whether to add the first component to the plurality of components based on a comparison of a performance of each of the first one or more machine learning pipelines with a performance of each of the second one or more machine learning pipelines; A machine learning pipeline component determination program that causes a computer to execute processing.

2. and acquiring the type of the first part by classifying the plurality of parts into a plurality of types and classifying the first part into one of the plurality of types. The machine learning pipeline component determination program according to claim 1 .

3. inputting the features of the first part into a machine learning model that has been trained to output the type of the part when the features of the part are input, using training data that associates the features of each of the plurality of parts with the type into which the part is classified, thereby acquiring the type of the first part; The machine learning pipeline component determination program according to claim 2 .

4. The component is a component that performs pre-processing on a data set, a change in the data set when the part is applied to the data set is used as the feature; The machine learning pipeline component determination program according to claim 3 .

5. The plurality of machine learning pipelines are the machine learning pipelines with the highest performance for each of the plurality of datasets. The machine learning pipeline component determination program according to any one of claims 1 to 4.

6. Obtaining a type of a first part; A program for generating a machine learning pipeline including a part selected from a plurality of parts according to a task identifies a first one or more machine learning pipelines including a part of the same type as the first part among a plurality of machine learning pipelines output for a plurality of datasets; generating, for each of the first one or more machine learning pipelines, a second one or more machine learning pipelines in which the components of the same type are replaced with the first components; determining whether to add the first component to the plurality of components based on a comparison of a performance of each of the first one or more machine learning pipelines with a performance of each of the second one or more machine learning pipelines; A machine learning pipeline component determination method characterized in that processing is executed by a computer.

7. Obtaining a type of a first part; A program for generating a machine learning pipeline including a part selected from a plurality of parts according to a task identifies a first one or more machine learning pipelines including a part of the same type as the first part among a plurality of machine learning pipelines output for a plurality of datasets; generating, for each of the first one or more machine learning pipelines, a second one or more machine learning pipelines in which the components of the same type are replaced with the first components; determining whether to add the first component to the plurality of components based on a comparison of a performance of each of the first one or more machine learning pipelines with a performance of each of the second one or more machine learning pipelines; A machine learning pipeline component determination device having a control unit that executes processing.

Images