Information processing system, information processing method, and program

Abstract

This information processing system comprises: an acquisition unit that acquires teacher data including a prescribed explanatory variable and a correct answer label; a candidate generation unit that generates candidate data including an explanatory variable satisfying a prescribed constraint condition; a dimension reduction unit that generates a plurality of feature data in which each of the teacher data and the candidate data is dimensionally reduced; a selection unit that selects candidate data on the basis of the degree of similarity to the teacher data, the degree of similarity being calculated using the feature data; and an output unit that outputs the selected candidate data.

Description

Information processing system, information processing method and program

[0001] The present disclosure relates to an information processing system, an information processing method, and a program.

[0002] There are known techniques for improving the prediction accuracy of machine learning models using a technique called active learning, etc. For example, Patent Literature 1 discloses a learning model generation system that generates multiple trained models by inputting training data into candidate models and performing machine learning, classifies the trained models into multiple groups based on the similarity of inference results output by each of the trained models, and sets trained models belonging to a group selected based on an index generated for each group as candidate models.

[0003] Supervised learning, a type of machine learning technique, uses training data in which correct values ​​of the objective variable (also called "correct labels") are assigned to one or more explanatory variables. In order to build a machine learning model with high generalization performance using supervised learning, a large amount of training data that covers the data distribution that can be predicted is required.

[0004] To create training data, it is necessary to perform annotation, which assigns correct labels to candidate explanatory variables. Annotation requires obtaining the correct values ​​of the objective variables for the explanatory variables, which requires a great deal of effort and cost. For example, to create training data for a physical property prediction model, it is necessary to conduct experiments using material samples under various experimental conditions and measure the physical property values ​​that are to be used as objective variables.

[0005] Active learning is a method for prioritizing the selection or generation of effective data to improve the predictive accuracy of a machine learning model and adding it to the training data. In active learning, a machine learning model is first generated based on available training data. Next, unlabeled data (hereinafter also referred to as "candidate data") that has not been assigned a correct answer label is input into the trained model to perform inference. Furthermore, based on the inference results for the candidate data, candidate data that is difficult for the trained model to infer is selected as an annotation target.

[0006] Next, the correct label for the selected candidate data is obtained, and the labeled data (hereinafter also referred to as "additional data") in which the correct label is assigned to the candidate data is added to the training data. Furthermore, the additional data is input into the trained model to perform re-training, and test data is input into the trained model to evaluate the generalization performance. Then, the addition of training data and re-training are repeated until the generalization performance of the trained model reaches a desired level.

[0007] Japanese Patent Application Laid-Open No. 2021-179859

[0008] However, the conventional techniques leave room for more efficient generation of training data. For example, because the conventional techniques use the inference accuracy of a trained model as an index, appropriate training data may not be generated if sufficient training data is not available.

[0009] For example, in active learning, there is a conventional technology that selects candidate data as an annotation target based on the reliability of predicted values ​​of a trained model. However, if the amount of trained training data is insufficient for the data distribution that can be predicted, there may be a wide range of low reliability predictions, making it difficult to select or generate candidate data appropriately. Furthermore, when the number of explanatory variables is large, more training data is required to improve prediction accuracy, which increases the possibility that the trained training data is insufficient.

[0010] In particular, in physical property prediction models, there are many types of features that can be used as explanatory variables, and because experiments are required to obtain correct labels, the amount of available training data is often small. Therefore, it is difficult to efficiently generate appropriate candidate data using conventional active learning methods that are based solely on the reliability of predicted values.

[0011] One aspect of the present disclosure aims to efficiently generate data used to train a machine learning model.

[0012] An information processing system according to a first aspect of the present disclosure includes an acquisition unit that acquires teacher data including predetermined explanatory variables and correct labels; a candidate generation unit that generates candidate data including explanatory variables that satisfy predetermined constraints; a dimension reduction unit that generates a plurality of feature data by reducing the dimensions of the teacher data and the candidate data; a selection unit that selects candidate data based on the similarity with the teacher data calculated using the feature data; and an output unit that outputs the selected candidate data.

[0013] An information processing system according to a second aspect of the present disclosure is the information processing system according to the first aspect, wherein the feature data is a latent variable calculated by partial least squares.

[0014] An information processing system according to a third aspect of the present disclosure is the information processing system according to the second aspect, wherein the similarity is a Euclidean distance between the latent variables.

[0015] An information processing system according to a fourth aspect of the present disclosure is an information processing system according to any one of the first to third aspects, wherein the selection unit is configured to select candidate data further based on the reliability of a predicted value obtained by a trained model that has learned the teacher data.

[0016] An information processing system according to a fifth aspect of the present disclosure is an information processing system according to the fourth aspect, further comprising an evaluation unit configured to calculate an evaluation value of candidate data based on similarity and reliability, and a selection unit configured to select candidate data based on the evaluation value.

[0017] An information processing system according to a sixth aspect of the present disclosure is the information processing system according to the fifth aspect, wherein the candidate generation unit is configured to optimize the candidate data based on the evaluation value.

[0018] An information processing system according to a seventh aspect of the present disclosure is an information processing system according to any of the first to sixth aspects, further comprising a model generation unit configured to generate a trained model based on training data.

[0019] An information processing system according to an eighth aspect of the present disclosure is an information processing system according to the seventh aspect, further comprising a model update unit configured to update the trained model based on additional data in which a correct answer label is assigned to the candidate data selected by the selection unit.

[0020] An information processing system according to a ninth aspect of the present disclosure is an information processing system according to any one of the first to eighth aspects, wherein the explanatory variables include experimental conditions related to the substance, and the correct label includes a physical property value related to the substance.

[0021] An information processing system according to a tenth aspect of the present disclosure is the information processing system according to the ninth aspect, wherein the substance includes at least one of a composition including a polymer, a low molecular weight compound, or an inorganic material.

[0022] An information processing method according to an eleventh aspect of the present disclosure includes a computer executing the steps of acquiring teacher data including predetermined explanatory variables and correct labels, generating candidate data including explanatory variables that satisfy predetermined constraints, generating a plurality of feature data by reducing the dimensions of the teacher data and the candidate data, selecting candidate data based on the similarity to the teacher data calculated using the feature data, and outputting the selected candidate data.

[0023] A program according to a twelfth aspect of the present disclosure causes a computer to execute the steps of acquiring teacher data including predetermined explanatory variables and correct labels, generating candidate data including explanatory variables that satisfy predetermined constraints, generating a plurality of feature data by reducing the dimensions of the teacher data and the candidate data, selecting candidate data based on the similarity to the teacher data calculated using the feature data, and outputting the selected candidate data.

[0024] According to one aspect of the present disclosure, data used for training a machine learning model can be efficiently generated.

[0025] FIG. 1 is a block diagram showing an example of the overall configuration of a machine learning system. FIG. 2 is a block diagram showing an example of the hardware configuration of a computer. FIG. 3 is a block diagram showing an example of the functional configuration of a model learning device. FIG. 4 is a diagram showing an example of teacher data. FIG. 5 is a block diagram showing an example of the functional configuration of a data generation device. FIG. 6 is a sequence diagram showing an example of a machine learning method. FIG. 7 is a flowchart showing an example of data generation processing. FIG. 8 is a diagram showing an example of evaluation results. FIG. 9 is a diagram showing an example of evaluation results. FIG. 10 is a diagram showing an example of experimental results related to a modified example.

[0026] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. In this specification and drawings, components having substantially the same functional configurations are designated by the same reference numerals, and redundant description will be omitted.

[0027] [Embodiment] One embodiment of the present disclosure is an information processing system that builds a machine learning model. Hereinafter, the information processing system according to this embodiment will be referred to as a "machine learning system." In this embodiment, the machine learning system has a function of building a machine learning model through active learning.

[0028] In this embodiment, the machine learning model may be a prediction model that predicts physical property values ​​related to a substance. Hereinafter, the machine learning model according to this embodiment will be referred to as a "physical property prediction model." The substance may include at least one of a composition containing a polymer, a low molecular weight compound, or an inorganic material. As an example, the substance may include a rubber composition, latex, a cycloolefin resin, etc.

[0029] Here, compositions containing polymers also include rubber compositions. Examples of rubber compositions include specialty rubbers, general-purpose rubbers, or semi-general-purpose rubbers. Special rubbers are synthetic rubbers with excellent liquid resistance and heat resistance. Examples of specialty rubbers include hydrogenated nitrile rubber, nitrile rubber, acrylic rubber, fluororubber, silicone rubber, epichlorohydrin rubber, etc. General-purpose rubbers or semi-general-purpose rubbers are synthetic rubbers that do not fall under the category of specialty rubbers. Examples of general-purpose rubbers include styrene butadiene rubber, butadiene rubber, isoprene rubber, etc. Examples of semi-general-purpose rubbers include ethylene propylene diene rubber, chloroprene rubber, etc.

[0030] The machine learning model may be any type of machine learning model. Examples of the type of machine learning model include Gaussian Process Regression (GPR), Partial Least Squares Regression (PLS), neural networks, random forests, and gradient boosting decision trees. Examples of the gradient boosting decision trees include Light Gradient Boosting Machine (LightGBM) and eXtreme Gradient Boosting (XGBoost).

[0031] This embodiment aims to efficiently generate data used for training a machine learning model. To this end, this embodiment generates multiple pieces of feature data by reducing the dimensions of each of the training data and candidate data, and selects candidate data based on the similarity to the training data calculated using the feature data.

[0032] According to one aspect, the present embodiment evaluates the similarity with existing training data based on dimension-reduced feature data, thereby enabling efficient generation of data used for training a machine learning model. According to another aspect, the present embodiment efficiently generates data that can improve the prediction accuracy of a machine learning model, thereby enabling efficient construction of a highly accurate machine learning model.

[0033] <Overall Configuration> The overall configuration of the machine learning system in this embodiment will be described with reference to Fig. 1. Fig. 1 is a block diagram showing an example of the overall configuration of the machine learning system.

[0034] 1, the machine learning system 1000 includes a model learning device 10, a data generating device 20, and a terminal device 50. The model learning device 10, the data generating device 20, and the terminal device 50 are connected to each other so as to be able to communicate data with each other via a communication network N such as a local area network (LAN) or the Internet.

[0035] The model learning device 10 is an example of an information processing device such as a personal computer, workstation, or server that constructs a physical property prediction model, which is an example of a machine learning model. The model learning device 10 generates a physical property prediction model based on teacher data in response to a learning request from the terminal device 50. The model learning device 10 updates the learned physical property prediction model based on additional data in response to an update request from the terminal device 50.

[0036] The data generating device 20 is an example of an information processing device such as a personal computer, workstation, or server that generates candidate data to be used for updating a physical property prediction model. The data generating device 20 generates multiple pieces of candidate data in response to a generation request from the terminal device 50. The data generating device 20 evaluates each of the multiple pieces of candidate data and selects candidate data based on the evaluation results. The data generating device 20 transmits the generation results including the selected candidate data to the terminal device 50.

[0037] The terminal device 50 is an example of an information processing terminal such as a personal computer, smartphone, or tablet terminal operated by a user of the machine learning system 1000. The terminal device 50 transmits a learning request for a physical property prediction model to the model learning device 10. The terminal device 50 transmits a generation request for candidate data to the data generating device 20. The terminal device 50 receives the generation results of candidate data from the data generating device 20 and presents them to the user. The terminal device 50 transmits an update request for the physical property prediction model to the model learning device 10.

[0038] The overall configuration of the machine learning system 1000 shown in FIG. 1 is an example, and various system configuration examples are possible depending on the application and purpose. For example, the machine learning system 1000 may include multiple units of one or more of the model learning device 10, the data generating device 20, and the terminal device 50. For example, the model learning device 10 or the data generating device 20 may be implemented by multiple computers or may be implemented as a cloud computing service. For example, the model learning device 10 and the data generating device 20 may be implemented by a single server that combines the functions of both. For example, the model learning device 10, the data generating device 20, and the terminal device 50 may be implemented by a standalone computer. The device divisions shown in FIG. 1, such as the model learning device 10, the data generating device 20, and the terminal device 50, are an example.

[0039] <Hardware Configuration> The hardware configuration of the machine learning system 1000 will be described with reference to Fig. 2. The model learning device 10, data generating device 20, and terminal device 50 included in the machine learning system 1000 are realized by, for example, a computer. Fig. 2 is a block diagram showing an example of the hardware configuration of a computer.

[0040] 2, the computer 500 includes a CPU (Central Processing Unit) 501, a ROM (Read Only Memory) 502, a RAM (Random Access Memory) 503, a HDD (Hard Disk Drive) 504, an input device 505, a display device 506, a communication I / F (Interface) 507, and an external I / F 508. The CPU 501, the ROM 502, and the RAM 503 form a so-called computer. The hardware components of the computer 500 are connected to each other via a bus line 509. The input device 505 and the display device 506 may be connected to the external I / F 508 for use.

[0041] The CPU 501 is a computing device that reads programs and data from a storage device such as the ROM 502 or the HDD 504 onto the RAM 503 and executes processing to realize overall control and functions of the computer 500 .

[0042] The ROM 502 is an example of a non-volatile semiconductor memory (storage device) that can retain programs and data even when the power is turned off. The ROM 502 functions as a main storage device that stores various programs, data, etc. required for the CPU 501 to execute various programs installed in the HDD 504. Specifically, the ROM 502 stores boot programs such as a Basic Input Output System (BIOS) and an Extensible Firmware Interface (EFI) that are executed when the computer 500 starts up, as well as data such as OS (Operating System) settings and network settings.

[0043] The RAM 503 is an example of a volatile semiconductor memory (storage device) in which programs and data are erased when the power is turned off. The RAM 503 is, for example, a dynamic random access memory (DRAM) or a static random access memory (SRAM). The RAM 503 provides a working area in which various programs installed in the HDD 504 are expanded when executed by the CPU 501.

[0044] The HDD 504 is an example of a non-volatile storage device that stores programs and data. The programs and data stored in the HDD 504 include an OS, which is basic software that controls the entire computer 500, and applications that provide various functions on the OS. Note that the computer 500 may use a storage device that uses flash memory as a storage medium (e.g., an SSD (Solid State Drive)) instead of the HDD 504.

[0045] The input device 505 includes a touch panel, operation keys and buttons, a keyboard and mouse, a microphone for inputting sound data such as voice, and the like, which are used by the user to input various signals.

[0046] The display device 506 is composed of a display such as a liquid crystal display or organic electroluminescence (EL) display for displaying a screen, a speaker for outputting sound data such as voice, and the like.

[0047] The communication I / F 507 is an interface that connects to a communication network and enables the computer 500 to perform data communication.

[0048] The external I / F 508 is an interface with external devices, such as a drive device 510.

[0049] The drive device 510 is a device for loading a recording medium 511. The recording medium 511 here includes media that record information optically, electrically, or magnetically, such as CD-ROMs, flexible disks, and magneto-optical disks. The recording medium 511 may also include semiconductor memories that record information electrically, such as ROMs and flash memories. This allows the computer 500 to read from and / or write to the recording medium 511 via the external I / F 508.

[0050] The various programs to be installed in the HDD 504 are installed, for example, by setting the distributed recording medium 511 in a drive device 510 connected to the external I / F 508 and reading the various programs recorded on the recording medium 511 by the drive device 510. Alternatively, the various programs to be installed in the HDD 504 may be installed by being downloaded via the communication I / F 507 from a network different from the communication network.

[0051] <Functional Configuration> The functional configuration of the machine learning system 1000 will be described with reference to Figures 3 to 5. Although arrows are shown in Figures 3 and 5, the direction of communication in the machine learning system 1000 is not limited to the directions indicated by the arrows.

[0052] <Model Learning Device> Fig. 3 is a block diagram showing an example of the functional configuration of a model learning device 10. As shown in Fig. 3, the model learning device 10 includes a data storage unit 101, a model storage unit 102, a request receiving unit 110, a model generation unit 120, and a model update unit 130.

[0053] The data storage unit 101 and the model storage unit 102 are realized by the HDD 504 shown in FIG.

[0054] The request receiving unit 110, the model generating unit 120, and the model updating unit 130 are realized by the processing that the CPU 501 executes by a program loaded from the HDD 504 shown in FIG. 2 onto the RAM 503.

[0055] The data storage unit 101 stores teacher data and additional data. The teacher data may be stored in advance. The teacher data may be included in a learning request received by the request receiving unit 110 and stored in the data storage unit 101 by the request receiving unit 110. The additional data may be included in an addition request received by the request receiving unit 110 and stored in the data storage unit 101 by the request receiving unit 110.

[0056] A trained physical property prediction model is stored in the model storage unit 102. The trained physical property prediction model may be stored in the model storage unit 102 by the model generation unit 120. The trained physical property prediction model may be updated by the model update unit 130.

[0057] The request receiving unit 110 receives a learning request or an update request. The request receiving unit 110 may receive a learning request or an update request from the terminal device 50. The request receiving unit 110 may receive a learning request or an update request from another device such as the data generating device 20.

[0058] The learning request is information or a signal requesting learning of a physical property prediction model. The learning request may include training data to be used for training the physical property prediction model. The training data may include correct values ​​(i.e., correct labels) of one or more predetermined explanatory variables and target variables.

[0059] 4 is a diagram showing an example of training data. As shown in FIG. 4, the training data has data items indicating an ID, an explanatory variable, and a target variable. The ID is identification information that identifies each piece of data included in the training data.

[0060] The explanatory variables may include, for example, the type of substance, the composition of the substance, the production conditions of the substance, or the measurement conditions of the physical property values. The explanatory variables may include, for example, the main raw material type, the additive type, the additive amount, the vulcanizing agent, the vulcanization accelerator, the preparation temperature, the preparation time, the measurement temperature, and the measurement humidity. The main raw material type, the additive type, the additive amount, the vulcanizing agent, and the vulcanization accelerator are examples of the composition. The preparation temperature and the preparation time are examples of the production conditions. The measurement temperature and the measurement humidity are examples of the measurement conditions. The number of explanatory variables is not limited, and may be, for example, 3 to 20.

[0061] The objective variable may include, for example, tensile strength. A specific correct value is set for the objective variable of the training data. The correct value may be an actual measurement value obtained through an experiment, a calculated value obtained through a simulation, or a predicted value obtained through another prediction model.

[0062] The explanatory variables and the objective variables may include one-dimensional numerical values ​​(e.g., real numbers), characters (e.g., categorical variables), image data, or time-series data. When the explanatory variables or objective variables are image data or time-series data, they may be converted into feature quantities extracted from the data or into identifiable information.

[0063] The update request is information or a signal requesting an update of a trained physical property prediction model. The update request may include additional data used to update the physical property prediction model. The additional data has the same data items as the training data. The additional data has values ​​of at least one or more of the explanatory variables and the objective variables that are different from those of the training data.

[0064] The teacher data and the additional data may be electronic files in which the data is electronically recorded. The electronic files may be written in a predetermined format. Examples of the predetermined format may include a Comma Separated Value (CSV) format, an Excel (registered trademark) format, an Extensible Markup Language (XML) format, or a JavaScript Object Notation (JSON) format.

[0065] The model generation unit 120 generates a physical property prediction model. The model generation unit 120 may generate the physical property prediction model by learning training data read from the data storage unit 101. The model generation unit 120 may perform preprocessing on the training data. The preprocessing may include any information processing used in machine learning. For example, the preprocessing may include data cleansing, scaling, encoding, or the like. The model generation unit 120 may store the learned physical property prediction model in the model storage unit 102.

[0066] The model updating unit 130 updates the trained physical property prediction model. The model updating unit 130 may update the trained physical property prediction model based on additional data read from the data storage unit 101. The model updating unit 130 may update the physical property prediction model by additionally learning the additional data. The model updating unit 130 may generate a new physical property prediction model by re-learning the teacher data to which the additional data has been added. Note that the updating of the physical property prediction model may be, for example, re-learning, additional learning, transfer learning, fine tuning, or the like. The model updating unit 130 may store the updated physical property prediction model in the model storage unit 102 as a trained physical property prediction model.

[0067] <Data Generating Device> Fig. 5 is a block diagram showing an example of the functional configuration of a data generating device 20. As shown in Fig. 5, the data generating device 20 includes a request receiving unit 210, an acquiring unit 220, a candidate generating unit 230, a predicting unit 240, a dimensionality reducing unit 250, an evaluating unit 260, a selecting unit 270, and an output unit 280.

[0068] The request receiving unit 210, the acquisition unit 220, the candidate generation unit 230, the prediction unit 240, the dimension reduction unit 250, the evaluation unit 260, the selection unit 270, and the output unit 280 are realized by processing that is executed by the CPU 501 using a program expanded from the HDD 504 shown in FIG. 2 onto the RAM 503.

[0069] The request receiving unit 210 receives a generation request. The request receiving unit 210 may receive a generation request from the terminal device 50. The request receiving unit 110 may receive a generation request from another device such as the model learning device 10.

[0070] The generation request is information or a signal requesting the generation of candidate data. The generation request may include generation parameters used to generate the candidate data. The generation parameters may include, for example, constraints on explanatory variables, the number of generation cycles, and the number of candidate data to be generated.

[0071] The constraint conditions for explanatory variables are conditions indicating constraints that the explanatory variables must satisfy. For example, the constraint conditions may include upper and lower limits and a variation range (step size) for the explanatory variables, which are numerical values. The variation range may be determined based on a controllable quantity in an experiment. For example, the constraint conditions may include options for the explanatory variables, which are categorical variables.

[0072] The constraint conditions may be set in advance by the user. The constraint conditions may be determined based on training data. The constraint conditions are not limited to conditions for a single explanatory variable, but may also be conditions that depend on other explanatory variables. As an example, a condition that depends on other explanatory variables may be a condition for a calculation result (e.g., a sum, etc.) using multiple explanatory variables.

[0073] In this embodiment, multiple candidate data may be repeatedly generated. Hereinafter, the process of generating candidate data once is referred to as a "generation cycle." A generation cycle includes the process of generating multiple candidate data and evaluating each of the generated candidate data. The number of generation cycles is the number of times a generation cycle is repeated. The number of candidate data to be generated is the number of candidate data generated in one generation cycle.

[0074] The generation request may include information indicating a physical property prediction model for which candidate data is to be generated. The information indicating the physical property prediction model may be, for example, identification information for identifying the physical property prediction model, information indicating a device in which the physical property prediction model is stored, or information indicating the type of physical property prediction model. For example, the acquisition unit 220 may acquire the physical property prediction model indicated in the information indicating the physical property prediction model. The generation request may include a trained physical property prediction model. The generation request may include training data used to train the physical property prediction model. The generation request may include information indicating the model learning device 10 from which the data generation device 20 acquires the physical property prediction model and training data.

[0075] The acquisition unit 220 acquires teacher data used in learning the physical property prediction model. The acquisition unit 220 may read out teacher data stored in the data storage unit 101 of the model learning device 10. The acquisition unit 220 may request teacher data from the model learning device 10 and acquire the teacher data received from the model learning device 10. The acquisition unit 220 may receive the teacher data included in the generation request from the request receiving unit 210.

[0076] The acquisition unit 220 may acquire a learned physical property prediction model. The acquisition unit 220 may read out a physical property prediction model stored in the model storage unit 102 of the model learning device 10. The acquisition unit 220 may request a physical property prediction model from the model learning device 10 and acquire the physical property prediction model received from the model learning device 10. The acquisition unit 220 may receive the physical property prediction model included in the generation request from the request receiving unit 210.

[0077] The candidate generation unit 230 generates candidate data. The candidate data includes explanatory variables that satisfy constraints included in the generation parameters. The candidate data has the same data items as the teacher data and additional data. The candidate data differs from the teacher data and additional data in that no objective variable is set (in other words, no correct answer label is assigned).

[0078] The candidate generation unit 230 may generate a plurality of candidate data. The candidate generation unit 230 may generate the number of candidate data to be generated that is included in the generation parameters. The candidate generation unit 230 may generate a plurality of candidate data by randomly setting the values ​​of each explanatory variable. Note that at this time, for explanatory variables with constraints, the candidate generation unit 230 may randomly set the values ​​of the explanatory variables within a range that satisfies the constraints. The candidate generation unit 230 may generate candidate data by optimizing the candidate data using a predetermined optimization method. For example, the predetermined optimization method may be a black-box optimization method such as Bayesian optimization or a genetic algorithm.

[0079] When optimizing candidate data, the candidate generator 230 may generate candidate data that is likely to have an improved evaluation value based on the evaluation values ​​of candidate data previously generated. The evaluation values ​​of the candidate data may be calculated by the evaluator 260.

[0080] The candidate generator 230 may use different methods for generating candidate data depending on the number of times a generation cycle is executed. For example, in the early stages of the iterative process, the candidate generator 230 may generate multiple pieces of candidate data in which the values ​​of each explanatory variable are randomly set. Furthermore, in the middle stages of the iterative process and thereafter, the candidate generator 230 may generate candidate data using a predetermined optimization method. As an example, the candidate generator 230 may randomly generate candidate data from the first generation cycle through the twentieth generation cycle, and then optimize the candidate data from the 21st generation cycle onward using a black-box optimization method such as Bayesian optimization or a genetic algorithm.

[0081] The prediction unit 240 acquires a predicted value by the physical property prediction model for each piece of candidate data generated by the candidate generation unit 230. The prediction unit 240 may predict a dependent variable based on the physical property prediction model. The prediction unit 240 may predict a dependent variable by inputting explanatory variables included in the candidate data into the trained physical property prediction model.

[0082] The prediction unit 240 may acquire a predicted value by the physical property prediction model from the model learning device 10. The prediction unit 240 may transmit a prediction request including candidate data to the model learning device 10 and receive a predicted value from the model learning device 10. In this case, upon receiving the prediction request, the model learning device 10 may input explanatory variables included in the candidate data into the trained physical property prediction model and transmit a predicted value of the objective variable output by the physical property prediction model to the data generating device 20.

[0083] The dimension reduction unit 250 generates a plurality of feature data by reducing the dimensions of the teacher data acquired by the acquisition unit 220 and the candidate data generated by the candidate generation unit 230. The dimension reduction unit 250 may reduce the dimensions of only a portion of the teacher data and the candidate data. The dimension reduction unit 250 may also reduce the dimensions of all of the teacher data and the candidate data.

[0084] The feature data may be, for example, latent variables calculated by partial least squares. In other words, the dimension reduction unit 250 may convert each of the teacher data and the candidate data into latent variables according to partial least squares. The number of dimensions of the latent variables may be determined arbitrarily, but may be three, for example.

[0085] The dimension reduction method is not limited to partial least squares, and may be, for example, principal component analysis (PCA), multi-dimensional scaling (MDS), t-distributed stochastic neighbor embedding (t-SNE), or the like.

[0086] The evaluation unit 260 calculates an evaluation value for each piece of candidate data generated by the candidate generation unit 230. The evaluation unit 260 may calculate the evaluation value based on the similarity between the candidate data and the teacher data. The evaluation unit 260 may calculate the evaluation value based on the reliability of the predicted value by the physical property prediction model. As an example, the evaluation unit 260 may calculate the product of the reliability and the similarity as the evaluation value. The evaluation unit 260 may calculate the evaluation value by weighting the reliability or the similarity.

[0087] The evaluation unit 260 may calculate the similarity between each piece of candidate data and each piece of teacher data using the feature data generated by the dimension reduction unit 250. The evaluation unit 260 may select the highest similarity between each piece of candidate data and each piece of teacher data as the similarity between the candidate data and the teacher data.

[0088] For example, when the feature data are latent variables calculated by partial least squares, the similarity may be the Euclidean distance between the latent variables. In other words, the evaluation unit 260 may calculate the Euclidean distance between each latent variable generated based on the candidate data generated by the candidate generation unit 230 and each latent variable generated based on the teacher data. The evaluation unit 260 may determine the shortest Euclidean distance between the latent variable based on the candidate data and each latent variable based on the teacher data as the similarity between the candidate data and the teacher data.

[0089] The evaluation unit 260 may calculate the reliability of the predicted value based on the predicted value acquired by the prediction unit 240. For example, when the physical property prediction model is a Gaussian process regression model, the reliability of the predicted value may be the standard deviation of the probability distribution output as a prediction result by the physical property prediction model.

[0090] In this embodiment, the evaluation unit 260 may calculate an evaluation value indicating that the lower the similarity to the teacher data, the higher the evaluation. Furthermore, the evaluation unit 260 may calculate an evaluation value indicating that the lower the reliability of the predicted value, the higher the evaluation. A low similarity to the teacher data indicates that the candidate data is not similar to any of the previously trained teacher data. Therefore, by learning candidate data with low similarity to the teacher data, the interpolation region of the physical property prediction model is likely to be expanded and generalization performance is likely to be improved. Furthermore, a low reliability of the predicted value indicates that the amount of teacher data is insufficient in the region. Therefore, by learning candidate data with low reliability of the predicted value, the prediction accuracy of the region adjacent to the candidate data is likely to be improved.

[0091] The Euclidean distance, which is an example of similarity, indicates that the larger the value, the lower the similarity, and the smaller the value, the higher the similarity. Furthermore, the standard deviation, which is an example of reliability, indicates that the larger the value, the lower the reliability, and the smaller the value, the higher the reliability. Therefore, when the reliability is the standard deviation and the similarity is the Euclidean distance, the larger the product of the reliability and similarity, the higher the evaluation, and the smaller the value, the lower the evaluation.

[0092] The evaluation unit 260 may calculate the evaluation value based on the predicted value of the physical property prediction model in addition to the reliability and similarity. For example, the evaluation unit 260 may calculate the evaluation value such that the better the predicted value of the objective variable, the higher the evaluation. When improving the prediction accuracy of the physical property prediction model, it may be desirable to improve the prediction accuracy in a region where the objective variable has a good value. By preferentially selecting candidate data with a good predicted value of the objective variable, the prediction accuracy in a region where the objective variable has a good value is more likely to be improved. A good value of the objective variable refers, for example, to a value of the objective variable that is the same as the value of the objective variable possessed by the substance the experimenter wishes to produce, or a value within a predetermined value. Alternatively, it refers to a value of the objective variable that is as large or small as possible. By preferentially selecting candidate data with a good predicted value of the objective variable, the experimenter can attempt to produce a substance having a desired physical property value while improving the prediction accuracy of the physical property prediction model.

[0093] The constraints may include a condition that the larger the predicted value of the objective variable, the higher the evaluation value of the candidate data. The constraints may also include a condition that the smaller the predicted value of the objective variable, the higher the evaluation value of the candidate data. In this case, the evaluation unit 260 may calculate the evaluation value based on the predicted value of the physical property prediction model and the constraints, in addition to the reliability and similarity.

[0094] The evaluation unit 260 stores the evaluation value in association with the candidate data. The evaluation unit 260 may store the evaluation value in a storage device such as the RAM 503 or HDD 504 of the data generating device 20. The evaluation unit 260 may store a pair of information (e.g., an ID) that can identify the candidate data and the evaluation value in the storage device.

[0095] The selection unit 270 selects candidate data to be output from among the candidate data generated by the candidate generation unit 230 based on the evaluation value calculated by the evaluation unit 260. The selection unit 270 selects one or more highly evaluated candidate data based on the evaluation value. For example, the selection unit 270 may select the candidate data with the highest evaluation. For example, if the reliability is the standard deviation and the similarity is the Euclidean distance, the selection unit 270 may select the candidate data with the largest evaluation value, which is the product of the reliability and the similarity.

[0096] The selection unit 270 may select multiple candidate data. The selection unit 270 may sort the candidate data in descending order of evaluation based on the evaluation value, and select a predetermined number of candidate data from the highest evaluation. The selection unit 270 may select candidate data whose evaluation value is equal to or greater than a predetermined threshold. The number of candidate data to be selected may be determined arbitrarily, and may be, for example, about 10.

[0097] The selection unit 270 may perform the following process to simultaneously improve the prediction accuracy of the property prediction model and produce a substance having a target property value. The selection unit 270 may select a predetermined number of candidate data items in descending order of evaluation, and further select candidate data items with a favorable predicted value for the objective variable. Specifically, the selection unit 270 may select, from the selected predetermined number of candidate data items, candidate data items including a predicted value for the objective variable closest to the objective value and candidate data items including a predicted value for the objective variable furthest from the objective value. The selection unit 270 may also select, from the selected predetermined number of candidate data items, multiple candidate data items whose predicted values ​​differ by a predetermined value. By having an experimenter conduct an experiment using the candidate data items selected by the selection unit 270, the experimenter can conduct an experiment under multiple experimental conditions whose predicted values ​​are different from each other and whose difference from the objective value is within a predetermined value. This allows the machine learning system 1000 to improve the prediction accuracy of the property prediction model while suggesting to the experimenter experimental conditions that may produce a substance having a target value.

[0098] The selection unit 270 may select a predetermined number of candidate data items in descending order of evaluation, and further select a predetermined number of candidate data items from the predetermined number of candidate data items in descending order of the candidate data items containing predicted values ​​of the objective variable whose values ​​are closest to the objective value. This allows the machine learning system 1000 to improve the prediction accuracy of the property prediction model while suggesting to the experimenter experimental conditions under which a substance having the objective value may be produced.

[0099] Furthermore, the selection unit 270 may select a predetermined number of candidate data items in descending order of evaluation, and further select a predetermined number of candidate data items from among the predetermined number of candidate data items in descending order of the predicted values ​​of the objective variables. Alternatively, the selection unit 270 may select a predetermined number of candidate data items in descending order of evaluation, and further select a predetermined number of candidate data items from among the predetermined number of candidate data items in descending order of the predicted values ​​of the objective variables.

[0100] The information indicating the selection criteria for candidate data by the selection unit 270 may be set at any timing by the user of the machine learning system 1000. The information indicating the selection criteria indicates a method for selecting candidate data by the selection unit 270. The terminal device 50 may display the generation result of candidate data output by the output unit 280 of the data generation device 20 described below, and may further accept input of information indicating the selection criteria. The data generation device 20 receives the information indicating the selection criteria from the terminal device 50 to which the information indicating the selection criteria has been input. The selection unit 270 may reselect candidate data based on the information indicating the selection criteria received by the data generation device 20, and the output unit 280 may re-output the generation result of candidate data.

[0101] The output unit 280 outputs the generation results of the candidate data. The generation results include the candidate data selected by the selection unit 270. The output unit 280 may display the generation results of the candidate data on the display device 506 of the data generating device 20. The output unit 280 may transmit the generation results of the candidate data to the terminal device 50. The output unit 280 may transmit the generation results of the candidate data to another device such as the model learning device 10.

[0102] When receiving the generation result from the data generating device 20, the terminal device 50 may present the candidate data included in the generation result to the user. For example, the terminal device 50 may display a screen including the candidate data on the display device 506.

[0103] <Processing Procedure> The machine learning method executed by the machine learning system 1000 will be described with reference to Fig. 6. Fig. 6 is a sequence diagram showing an example of the machine learning method.

[0104] In step S1, a user of the machine learning system 1000 performs a learning request operation on the terminal device 50 to request learning of a physical property prediction model. In the learning request operation, as an example, a request for teacher data is displayed on the display device 506 of the terminal device 50, and the user performs an operation in response to the request displayed on the display device 506. The user's operation may be an operation of inputting electronic data in which teacher data is recorded into the terminal device 50. As a result, the user inputs the electronic data in which teacher data is recorded into the terminal device 50. In response to the user's learning request operation, the terminal device 50 generates a learning request including the input teacher data and transmits it to the model learning device 10.

[0105] In step S2, the model learning device 10 receives a learning request from the terminal device 50. The request receiving unit 110 of the model learning device 10 receives the learning request received by the model learning device 10. The request receiving unit 110 stores the teacher data included in the learning request in the data storage unit 101. The request receiving unit 110 notifies the model generation unit 120 that the learning request has been received.

[0106] In response to a notification from the request receiving unit 110, the model generation unit 120 of the model learning device 10 reads out teacher data from the data storage unit 101. The model generation unit 120 generates a physical property prediction model by learning the read out teacher data. The model generation unit 120 stores the learned physical property prediction model in the model storage unit 102. The model learning device 10 requests the terminal device 50 to display a generation request screen. The terminal device 50 displays the generation request screen on the display device 506.

[0107] In step S3, the user of the machine learning system 1000 performs a generation request operation on the terminal device 50 to request the generation of candidate data. In the generation request operation, as an example, a message requesting generation parameters may be displayed on the display device 506 of the terminal device 50, and the user may perform an operation in response to the request displayed on the display device 506. The operation by the user may be an operation to input the generation parameters into the terminal device 50. As a result, the user inputs the generation parameters into the terminal device 50. In response to the generation request operation by the user, the terminal device 50 generates a generation request including the input generation parameters and transmits the request to the data generator 20.

[0108] In step S4, the data generating device 20 receives a generation request from the terminal device 50. The request receiving unit 210 of the data generating device 20 receives the generation request received by the data generating device 20. The request receiving unit 210 sends the generation request to the acquisition unit 220 and the candidate generating unit 230.

[0109] The acquisition unit 220 of the data generating device 20 receives a generation request from the request receiving unit 210. The acquisition unit 220 acquires teacher data and a physical property prediction model from the model learning device 10 indicated in the generation request. The acquisition unit 220 sends the physical property prediction model to the prediction unit 240. The acquisition unit 220 sends the teacher data to the dimension reduction unit 250.

[0110] In step S5, the candidate generator 230 of the data generator 20 receives the generation request from the request receiver 210. The candidate generator 230 acquires generation parameters from the generation request. The candidate generator 230 generates candidate data based on the generation parameters.

[0111] <Data Generation Process> The data generation process (step S5 in FIG. 6) executed by the data generation device 20 will be described with reference to FIG. 7. FIG. 7 is a flowchart showing an example of the data generation process.

[0112] In step S5-1, the candidate generator 230 generates multiple pieces of candidate data based on the number of generations included in the generation parameters. In the first generation cycle, the candidate generator 230 randomly generates multiple pieces of candidate data. The candidate generator 230 sends the generated multiple pieces of candidate data to the prediction unit 240 and the dimension reduction unit 250.

[0113] In step S5-2, the prediction unit 240 receives candidate data from the candidate generation unit 230. The prediction unit 240 receives a physical property prediction model from the acquisition unit 220. The prediction unit 240 predicts a dependent variable for each candidate data based on the physical property prediction model. The prediction unit 240 sends the predicted value of the dependent variable for each candidate data to the evaluation unit 260.

[0114] In step S5-3, the evaluation section 260 receives the predicted value for each candidate data from the prediction section 240. The evaluation section 260 calculates the reliability of the predicted value for each candidate data based on the predicted value.

[0115] In step S5-4, the dimension reduction unit 250 receives candidate data from the candidate generation unit 230. The dimension reduction unit 250 receives teacher data from the acquisition unit 220. The dimension reduction unit 250 reduces the dimensions of each of the candidate data and teacher data. As a result, multiple pieces of feature data are generated by reducing the dimensions of each of the candidate data and teacher data. The dimension reduction unit 250 sends the generated multiple pieces of feature data to the evaluation unit 260.

[0116] In step S5-5, the evaluation unit 260 receives multiple pieces of feature data from the dimension reduction unit 250. The evaluation unit 260 calculates the similarity between each piece of candidate data and each piece of teacher data based on the multiple pieces of feature data. The evaluation unit 260 selects the candidate data with the highest similarity between each piece of candidate data and each piece of teacher data.

[0117] In step S5-6, the evaluation unit 260 calculates an evaluation value for each piece of candidate data based on the reliability calculated in step S5-3 and the similarity calculated in step S5-5. Specifically, the evaluation unit 260 calculates the product of the reliability calculated in step S5-3 and the similarity calculated in step S5-5 as the evaluation value. The evaluation unit 260 associates the calculated evaluation value with the candidate data and stores it in the storage device.

[0118] In step S5-7, the evaluation unit 260 determines whether or not to terminate the repetitive process. Specifically, the evaluation unit 260 determines whether or not the number of times the generation cycle has been executed is equal to or greater than the number of generation cycles included in the generation parameters. If the number of times the generation cycle has been executed is equal to or greater than the number of generation cycles, the evaluation unit 260 determines to terminate the repetitive process. On the other hand, if the number of times the generation cycle has been executed is less than the number of generation cycles, the evaluation unit 260 determines not to terminate the repetitive process.

[0119] If it is determined that the repetitive processing should be ended (YES), the evaluation unit 260 ends the data generation processing. The evaluation unit 260 requests the selection unit 270 to select candidate data. In response to the request from the evaluation unit 260, the selection unit 270 executes the processing of step S6 in FIG. 6, which will be described later. On the other hand, if it is determined that the repetitive processing should not be ended (NO), the evaluation unit 260 returns the processing to step S5-1.

[0120] When the process returns to step S5-1, the candidate generator 230 generates new candidate data for the next generation cycle. If the number of executions of the generation cycle is equal to or less than a predetermined threshold (for example, if the number of executions of the generation cycle is 20 or less), the candidate generator 230 randomly generates new candidate data in step S5-1. On the other hand, if the number of executions of the generation cycle exceeds the predetermined threshold (for example, if the number of executions of the generation cycle is 21 or more), the candidate generator 230 generates new candidate data by optimizing the candidate data using a predetermined optimization method in step S5-1.

[0121] Thereafter, the data generating device 20 executes the processes from step S5-2 to step S5-7 on the new candidate data. In this manner, the data generating device 20 repeatedly executes steps S5-1 to S5-7 until the number of executions of the generation cycle is equal to or greater than the number of generation cycles.

[0122] Returning to FIG. 6 , in step S6, the selection unit 270 of the data generating device 20 selects candidate data to be output from the candidate data generated in step S5. The selection unit 270 reads out the evaluation value of each piece of candidate data from the storage device. Based on the read evaluation values, the selection unit 270 selects one or more highly rated candidate data. The selection unit 270 sends the selected candidate data to the output unit 280.

[0123] In step S7, the output unit 280 of the data generating device 20 receives the candidate data from the selection unit 270. The output unit 280 generates a generation result including the received candidate data. The output unit 280 transmits the generation result to the terminal device 50.

[0124] In step S8, the terminal device 50 receives the generation result from the data generating device 20. The terminal device 50 presents the candidate data included in the generation result to the user. For example, the terminal device 50 displays a screen including the candidate data on the display device 506.

[0125] In step S9, the user of the machine learning system 1000 refers to the candidate data presented by the terminal device 50. The user obtains a correct label based on the candidate data. When multiple candidate data are presented, the user may select one or more candidate data from which to obtain a correct label.

[0126] For example, a user may conduct an experiment according to experimental conditions indicated by explanatory variables included in the candidate data displayed on the display device 506 as a generation result, measure physical property values ​​serving as the objective variable, and obtain a correct label for the objective variable from the measurement results. It is also possible that an experiment may be conducted under conditions different from the experimental conditions indicated by the explanatory variables. The machine learning system 1000 may accept corrections to experimental values ​​from values ​​in the candidate data for each explanatory variable, such as experimental conditions and measurement conditions, in the operation of assigning correct labels, as described below. The machine learning system 1000 may update the physical property prediction model using additional data to which correct labels have been assigned. Alternatively, a user may calculate or predict physical property values ​​according to experimental conditions based on a simulation or another prediction model, and obtain the calculated or predicted results as the correct label for the objective variable.

[0127] When the user acquires the correct label, the user performs an operation on the terminal device 50 to assign the correct label to the candidate data. The terminal device 50 assigns the correct label to the candidate data in response to the user's operation. As a result, additional data is generated that includes the explanatory variables included in the candidate data and the correct label assigned by the user. The terminal device 50 may output electronic data recording the additional data.

[0128] In step S10, a user of the machine learning system 1000 performs an update request operation on the terminal device 50 to request an update of the physical property prediction model. The update request operation may, for example, be an operation of inputting electronic data in which additional data is recorded onto a screen displayed on the display device 506 of the terminal device 50. In response to the update request operation by the user, the terminal device 50 generates an update request including the input additional data and transmits the update request to the model learning device 10.

[0129] In step S11, the model learning device 10 receives an update request from the terminal device 50. The request receiving unit 110 of the model learning device 10 receives the update request received by the model learning device 10. The request receiving unit 110 stores the additional data included in the update request in the data storage unit 101.

[0130] The model update unit 130 of the model learning device 10 reads additional data from the data storage unit 101. The model update unit 130 updates the physical property prediction model by additionally learning the read additional data. The model update unit 130 stores the updated physical property prediction model in the model storage unit 102 as a learned physical property prediction model. In other words, the model update unit 130 overwrites the learned physical property prediction model stored in the model storage unit 102 with the updated physical property prediction model.

[0131] <Evaluation Results> The results of evaluating the prediction accuracy of the physical property prediction model will be described with reference to Fig. 8 and Fig. 9. Fig. 8 and Fig. 9 are diagrams showing examples of the evaluation results. Fig. 8 and Fig. 9 show evaluation results of the prediction accuracy when training data obtained by the machine learning system 1000 was added to a physical property prediction model that predicts the tensile strength of polyisoprene latex. In this evaluation, training data was used that included the compounding conditions, processing conditions, and evaluation conditions of polyisoprene latex as explanatory variables, and the tensile strength of polyisoprene latex as a response variable.

[0132] 8 and 9 are graphs showing the transition of prediction accuracy with an increase in the number of experiments, with the horizontal axis representing the number of experiments and the vertical axis representing the prediction accuracy. The prediction accuracy is calculated using the coefficient of determination R ２ The shaded areas in Figures 8 and 9 indicate the ranges where experiments were conducted based on the experimental conditions output by this embodiment. The remaining areas indicate the ranges where experiments were conducted based on the experimenter's hypotheses, as in the conventional case. Each black circle shown in Figures 8 and 9 is shown at a position corresponding to the prediction accuracy of the physical property prediction model trained using experimental data ranging from 0 to the number of experiments corresponding to the black circle, and the number of experiments of the training data used for training.

[0133] FIG. 8 shows a dashed line indicating a prediction accuracy of 0.86 and a dashed line indicating a prediction accuracy of 0.92. In the range (a) of FIG. 8 , experiments were conducted based on the experimenter's hypotheses, etc., as in the conventional case, for the number of experiments from 0 to 647. The prediction accuracy of the physical property prediction model trained using experimental data from the number of experiments from 0 to 647 was 0.86. In the range (b) of FIG. 8 , experiments were conducted based on the experimental conditions output by this embodiment for the number of experiments from 648 to 683. As a result, the prediction accuracy of the physical property prediction model trained using experimental data from the number of experiments from 0 to 683 was 0.92. The prediction accuracy of the physical property prediction model increased from 0.86 to 0.92 with the addition of additional data from the number of experiments from 648 to 683.

[0134] In the range (c) of Fig. 8, experiments were conducted based on the experimenter's hypotheses, etc., as in the past, for the number of experiments from 684 to 830. As shown in Fig. 8, the prediction accuracy of the property prediction model trained using experimental data from the number of experiments from 0 to 830 was comparable to (slightly decreased from) that of the property prediction model trained using experimental data from the number of experiments from 0 to 683.

[0135] In the range (d) of Fig. 8 , experiments were conducted based on the experimental conditions output by this embodiment for the number of experiments from 831 to 854. As shown in Fig. 8 , the prediction accuracy of the property prediction model trained using experimental data for the number of experiments from 0 to 854 was comparable to that of the property prediction model trained using experimental data for the number of experiments from 0 to 683.

[0136] When the machine learning system 1000 updated the physical property prediction model using an additional 36 pieces of data in range (b) of Figure 8, the prediction accuracy significantly improved from 0.86 to 0.92. Meanwhile, the prediction accuracy of the physical property prediction model updated by the machine learning system 1000 using data in ranges (a) to (c) of Figure 8 was comparable to the prediction accuracy of the physical property prediction model updated by the machine learning system 1000 using data in ranges (a) to (b) of Figure 8. This demonstrates that the prediction accuracy of the physical property prediction model of the machine learning system 1000 significantly improved by using 36 pieces of data as additional data, which are the results of experiments performed in accordance with the output results of the machine learning system 1000. The machine learning system 1000 was able to efficiently generate data used for training the machine learning model.

[0137] Furthermore, the prediction accuracy of the physical property prediction model updated by the machine learning system 1000 using data in range (a) to range (d) of Figure 8 was comparable to the prediction accuracy of the physical property prediction model using data in range (a) to range (b) of Figure 8. It can be seen that the prediction accuracy of the physical property prediction model of the machine learning system 1000 was sufficiently improved by additional data (range (b) of Figure 8) from the results of an experiment conducted in response to the first output result of the machine learning system 1000. In this way, the machine learning system 1000 can improve the prediction accuracy of the physical property prediction model even if the number of times additional data is added using the machine learning system 1000 is small.

[0138] 9 is an approximate straight line that approximates the transition of prediction accuracy at three points where the number of experiments was 400, 550, and 647 within the range where experiments were conducted based on the experimenter's hypotheses, etc., as in the past. This dashed line shows the transition of prediction accuracy of the physical property prediction model within the range where experiments were conducted based on hypotheses, etc., in an attempt to improve the prediction accuracy of the physical property prediction model.

[0139] On the other hand, the dashed-dotted lines shown in Figure 9 are approximate lines that approximate the transition of prediction accuracy at three points where the number of experiments was 648, 663, and 683 within the range in which experiments were conducted based on the experimental conditions output by this embodiment. The steeper the slope of these approximate lines, the greater the improvement in prediction accuracy with fewer experiments. In other words, a steeper slope of the approximate line means that prediction accuracy has been improved efficiently. Figure 9 shows that the slope of the dashed-dotted line is steeper than that of the dotted line, indicating that conducting experiments based on the experimental conditions output by this embodiment significantly improved prediction accuracy.

[0140] The evaluation results show that the machine learning system 1000 according to the embodiment outputs experimental conditions that are effective in improving prediction accuracy. The evaluation results also show that the machine learning system 1000 according to the embodiment can efficiently improve the prediction accuracy of a machine learning model with a small number of experiments.

[0141] The machine learning system 1000 may use multiple types of machine learning models. For example, the machine learning system 1000 executes the processes from step S1 to step S5 of FIG. 6 using each of the various physical property prediction models used by the machine learning system 1000. In step S5, the data generating device 20 calculates, for each piece of candidate data, an evaluation value for the candidate data generated by each physical property prediction model using the same method regardless of the type of physical property prediction model. In step S6, the selection unit 270 of the data generating device 20 selects one or more highly evaluated candidate data from all of the candidate data generated using each of the multiple types of physical property prediction models based on the evaluation value. The machine learning system 1000 then executes the processes from step S7 onward in the same manner as in FIG. 6.

[0142] <Modification> The machine learning system 1000 according to the above embodiment may be combined with another machine learning method. The other machine learning method may be an optimization method that recommends optimal experimental conditions based on machine learning. In this modification, an example will be described in which, in a system that recommends experimental conditions based on Bayesian optimization for the purpose of improving the peel strength of an acrylic resin, a physical property prediction model is constructed using the machine learning system 1000 prior to Bayesian optimization.

[0143] The physical property prediction model according to this modification will be described with reference to FIG. 10 . FIG. 10 is a diagram showing an example of experimental results according to the modification. In this experiment, before using the physical property prediction model by Bayesian optimization, training data was created multiple times using the machine learning system 1000. FIG. 10 shows an example of the results of measuring the peel strength of an acrylic resin produced with the aim of improving the peel strength of the acrylic resin.

[0144] Figure 10 is a graph showing the peel strength values ​​for each experiment, with the horizontal axis representing the number of experiments and the vertical axis representing the peel strength. In the range (a) of Figure 10, experiments were conducted based on the experimenter's hypotheses, as in the past, and the peel strength of the acrylic resin obtained through the experiments is shown (white circles). In the ranges (b), (d), and (f) of Figure 10, experiments were conducted based on experimental conditions obtained by the machine learning system 1000, and the peel strength of the acrylic resin obtained through the experiments is shown (shaded circles). In the ranges (c), (e), and (g) of Figure 10, experiments were conducted based on experimental conditions output from a physical property prediction model using Bayesian optimization, and the peel strength of the acrylic resin obtained through the experiments is shown (black triangles).

[0145] In the ranges (c), (e), and (g) of FIG. 10 , a property prediction model different from that used by the machine learning system 1000 was used. Specifically, the property prediction model used in the ranges (c), (e), and (g) of FIG. 10 was generated by Bayesian optimization using training data including experimental data obtained from experiments based on the experimenter's hypotheses and experimental data obtained from experiments based on the experimental conditions obtained by the machine learning system 1000. Bayesian optimization is suitable for data-driven maximization or minimization of a target property. Note that the property prediction models generated by Bayesian optimization from the second iteration onward used the experimental data obtained in previous experiments as training data.

[0146] 10 shows a dashed line indicating that the peel strength is 10.6 N / m and a dashed line indicating that it is 12.3 N / m. In this experiment, in the property prediction model of the machine learning system 1000 trained using experimental data from up to 80 experiments (i.e., only experimental data based on the experimenter's hypotheses, etc.), the coefficient of determination R ２ On the other hand, in the physical property prediction model of the machine learning system 1000 trained using experimental data for up to 149 experiments (i.e., experimental data including experimental data based on the experimenter's hypotheses, experimental data obtained by the machine learning system 1000, and experimental data performed according to the output of the physical property prediction model by Bayesian optimization), the coefficient of determination R ２ became 0.70. In this way, the machine learning system 1000 can improve the prediction accuracy of the property prediction model.

[0147] In addition, the machine learning system 1000 can provide training data that can be used to obtain the desired physical property value by updating the physical property prediction model of the machine learning system 1000 using data from a physical property prediction model based on Bayesian optimization that is suitable for predicting the desired physical property value (peel strength in Figure 10).

[0148] Furthermore, a Bayesian optimization physical property prediction model using data obtained by the machine learning system 1000 as training data was able to recommend experimental conditions capable of producing an acrylic resin with a high peel strength of 12.3 N / m. The maximum peel strength of the acrylic resin obtained in experiments conducted based on the experimenter's hypotheses was 10.6 N / m. Figure 10 shows that by utilizing the data obtained by the machine learning system 1000, an acrylic resin with a high peel strength that could not be obtained in experiments conducted based on the experimenter's hypotheses can be produced. In this way, the use of the machine learning system 1000 can improve the prediction accuracy of the physical property prediction model.

[0149] As described with reference to FIG. 10 , data obtained using the machine learning system 1000 may be provided for training a model suitable for predicting a target physical property value. An apparatus having a physical property prediction model trained using the provided data acquires information indicating the target physical property value, inputs the acquired information indicating the target physical property value into the physical property prediction model, and displays the output result of the physical property prediction model on a predetermined display device. The output result of the physical property prediction model is, for example, a high Expectation Improvement (EI) score. In this way, the machine learning system 1000 may provide the generated data to an apparatus or the like that generates another physical property prediction model.

[0150] <Effects of the embodiment> A machine learning system 1000 according to an embodiment of the present disclosure acquires training data including predetermined explanatory variables and correct labels, generates candidate data including explanatory variables that satisfy predetermined constraints, generates multiple pieces of feature data by reducing the dimensions of the training data and the candidate data, and selects candidate data based on the similarity to the training data calculated using the feature data.

[0151] In one aspect, according to this embodiment, the similarity with the training data is calculated based on feature data obtained by reducing the dimensions of the training data and candidate data, thereby enabling efficient generation of data to be used for training a machine learning model.

[0152] The feature data may be latent variables calculated by partial least squares. The similarity may be the Euclidean distance between the latent variables. In one aspect, this embodiment can robustly generate appropriate data by avoiding the influence of multicollinearity, noise, and the like.

[0153] The machine learning system 1000 may select candidate data further based on the reliability of a predicted value by a trained model that has trained the training data. The machine learning system 1000 may calculate an evaluation value of the candidate data based on the similarity and the reliability. The machine learning system 1000 may select candidate data based on the evaluation value. In one aspect, according to this embodiment, candidate data can be appropriately selected based on the similarity to the training data and the reliability of the predicted value.

[0154] The machine learning system 1000 may optimize the candidate data based on the evaluation value. According to one aspect, this embodiment can generate candidate data that is likely to improve prediction accuracy based on the evaluation results of other candidate data.

[0155] The machine learning system 1000 may generate a trained model based on training data. The machine learning system 1000 may update the trained model based on additional data in which selected candidate data is assigned a correct answer label. According to one aspect, the present embodiment updates the machine learning model based on data that can improve the prediction accuracy of the machine learning model, thereby efficiently improving the prediction accuracy of the machine learning model.

[0156] The explanatory variables may include experimental conditions related to the substance. The ground truth labels may include physical property values ​​related to the substance. The substance may include at least one of a composition including a polymer, a low molecular weight compound, or an inorganic material. In one aspect, according to this embodiment, experimental conditions can be generated to obtain data that can improve the prediction accuracy of a machine learning model that predicts physical property values ​​of various substances.

[0157] [Supplementary Note] Each function of the above-described embodiments can be realized by one or more processing circuits. Here, the term "processing circuit" in this specification includes a processor programmed to execute each function by software, such as a CPU (Central Processing Unit) or GPU (Graphics Processing Unit) implemented by an electronic circuit, as well as devices such as an ASIC (Application Specific Integrated Circuit), DSP (Digital Signal Processor), FPGA (Field Programmable Gate Array), and conventional circuit modules designed to execute each of the above-described functions.

[0158] Although the embodiments of the present disclosure have been described in detail above, the embodiments disclosed herein are illustrative in all respects and are not limiting. The embodiments can be modified and improved in various ways without departing from the scope and spirit of the appended claims. The matters described in the above embodiments can be configured in other ways as long as they are not inconsistent, and can be combined as long as they are not inconsistent.

[0159] This application claims priority from Japanese Patent Application No. 2024-96438, filed with the Japan Patent Office on June 14, 2024, the entire contents of which are incorporated herein by reference.

[0160] 10: Model learning device 20: Data generation device 50: Terminal device 101: Data storage unit 102: Model storage unit 110: Request reception unit 120: Model generation unit 130: Model update unit 210: Request reception unit 220: Acquisition unit 230: Candidate generation unit 240: Prediction unit 250: Dimension reduction unit 260: Evaluation unit 270: Selection unit 280: Output unit 1000: Machine learning system

Claims

1. An information processing system comprising: an acquisition unit configured to acquire teacher data including predetermined explanatory variables and correct labels; a candidate generation unit configured to generate candidate data including explanatory variables that satisfy predetermined constraints; a dimension reduction unit configured to generate a plurality of feature data by reducing the dimensions of the teacher data and the candidate data; a selection unit configured to select the candidate data based on the similarity to the teacher data calculated using the feature data; and an output unit configured to output the selected candidate data.

2. The information processing system according to claim 1, wherein the feature data is a latent variable calculated by partial least squares.

3. The information processing system according to claim 2, wherein the similarity is a Euclidean distance between the latent variables.

4. An information processing system according to any one of claims 1 to 3, wherein the selection unit is configured to select the candidate data further based on the reliability of a predicted value by a trained model that has trained the training data.

5. The information processing system according to claim 4, further comprising an evaluation unit configured to calculate an evaluation value of the candidate data based on the similarity and the reliability, and the selection unit configured to select the candidate data based on the evaluation value.

6. The information processing system according to claim 5, wherein the candidate generation unit is configured to optimize the candidate data based on the evaluation value.

7. An information processing system according to any one of claims 1 to 6, further comprising a model generation unit configured to generate a trained model based on the training data.

8. The information processing system according to claim 7, further comprising a model update unit configured to update the trained model based on additional data to which a correct answer label has been assigned to the candidate data selected by the selection unit.

9. An information processing system according to any one of claims 1 to 8, wherein the explanatory variables include experimental conditions related to a substance, and the correct labels include physical property values ​​related to the substance.

10. The information processing system according to claim 9, wherein the substance includes at least one of a composition including a polymer, a low molecular weight compound, or an inorganic material.

11. An information processing method in which a computer executes the following steps: acquiring training data including predetermined explanatory variables and correct labels; generating candidate data including explanatory variables that satisfy predetermined constraints; generating a plurality of feature data by reducing the dimensions of the training data and the candidate data; selecting the candidate data based on the similarity with the training data calculated using the feature data; and outputting the selected candidate data.

12. A program for causing a computer to execute the following steps: acquiring training data including predetermined explanatory variables and correct labels; generating candidate data including explanatory variables that satisfy predetermined constraints; generating a plurality of feature data by reducing the dimensions of the training data and the candidate data; selecting the candidate data based on the similarity with the training data calculated using the feature data; and outputting the selected candidate data.

Images