Model creation method, program, and information processing device

Abstract

And the prediction precision of the model is improved. The present disclosure provides a model creation method for creating, by an information processing apparatus, a model in which input data included in learning data is associated with output data, the model creation method performing: a first exclusion process of excluding learning data in which the input data is an abnormal value from a learning data set; a second exclusion process of excluding learning data in which the input data and the output data are abnormal values from the learning data set; a first model creation process of creating a first model using the learning data set from which the learning data has been excluded in the first exclusion process; a second model creation process of creating a second model using the learning data set from which the learning data has been excluded in the second exclusion process; and performing model selection processing, wherein the model with better prediction precision in the first model and the second model is adopted.

Description

Technical Field

[0001] This invention relates to a model creation method, program, and information processing device. Background Technology

[0002] Materials informatics is known for using information science techniques such as statistical analysis, machine learning, and data mining to improve the efficiency of materials development. One aspect of materials informatics is composition search, which aims to find the optimal composition of materials.

[0003] In component search, in order to create a more accurate model, outliers are separated or excluded from the learning data (for example, see Patent Documents 1-3). Patent Document 1 discloses a technique that obtains a dataset in a predetermined format consisting of a group of multiple test conditions and material information of a target substance created based on the test conditions, performs statistical processing on the dataset, and, based on the results, infers recommended test conditions for optimizing the material information and the corresponding material information of the target substance.

[0004] Patent document 2 discloses a technique in which a data evaluation model, constructed by machine learning on the collected data, evaluates the input collected data and automatically selects training data from the evaluated data to build a learning model based on the evaluation results.

[0005] Patent document 3 discloses a technique that selects data from data stored in a data storage unit for updating an existing learning model used in machine learning, creates a new learning model based on the selected data, and updates the existing learning model to the new learning model.

[0006] <Prior art documents>

[0007] <Patent Documents>

[0008] Patent Document 1: Japanese Patent Application Publication No. 2021-193623

[0009] Patent Document 2: Japanese Patent Application Publication No. 2021-107970

[0010] Patent Document 3: Japanese Patent Application Publication No. 2021-157654 Summary of the Invention

[0011] <Problem to be solved by this invention>

[0012] However, traditional techniques suffer from insufficient outlier removal, hindering improvements in model prediction accuracy. For example, with a large number of explanatory variables and a sparse dataset, the search space expands significantly, making it difficult to appropriately identify outliers using traditional methods. Furthermore, since such datasets are not normally distributed, it is difficult to use methods such as standard deviation to identify outliers.

[0013] This invention provides a technique to improve the prediction accuracy of a model.

[0014] <Methods for solving problems>

[0015] This disclosure provides a model creation method, in which an information processing device creates a model that associates input data and output data contained in learning data. The model creation method performs the following processes: a first exclusion process, excluding learning data from the learning dataset where the input data is an outlier; a second exclusion process, excluding learning data from the learning dataset where both the input data and the output data are outliers; a first model creation process, using the learning dataset from which the learning data was excluded in the first exclusion process to create a first model; a second model creation process, using the learning dataset from which the learning data was excluded in the second exclusion process to create a second model; and a model selection process, employing the model with better prediction accuracy between the first model and the second model.

[0016] According to the second disclosure, in the first disclosure, in the first exclusion process, any one of a plurality of different methods is used to exclude outliers from the input data.

[0017] According to the third disclosure, in the second disclosure, in the first exclusion process, any one of at least two different first methods or second methods is used to exclude outliers from the input data.

[0018] According to the fourth disclosure, in any of the first to third disclosures, in the second exclusion process, outliers in the input data are excluded using any one of a plurality of different methods, and in the case that the output data is a single variable, outliers in the output data are excluded using a predetermined method.

[0019] According to the fifth disclosure, in the fourth disclosure, in the second exclusion process, outliers of the input data are excluded using any one of at least two different third or fourth methods, and in the case that the output data is a single variable, outliers of the output data are excluded using a fifth method. In the second exclusion process, outliers of the input data are excluded using the third method and outliers of the output data are excluded using the fifth method, or, outliers of the input data are excluded using the fourth method and outliers of the output data are excluded using the fifth method.

[0020] According to the 6th disclosure, in any of the 1st to 3rd disclosures, in the second exclusion process, any one of a plurality of different methods is used to exclude outliers of the input data, and in the case that the output data is multivariable, any one of a plurality of different methods is used to exclude outliers of the output data.

[0021] According to the 7th disclosure, in the 6th disclosure, in the second exclusion process, outliers of the input data are excluded using any one of at least two different third or fourth methods, and in the case that the output data is multivariable, outliers of the output data are excluded using any one of at least two different sixth or seventh methods. In the second exclusion process, outliers of the input data are excluded using the third method and outliers of the output data are excluded using the sixth method, or outliers of the input data are excluded using the fourth method and outliers of the output data are excluded using the sixth method, or outliers of the input data are excluded using the third method and outliers of the output data are excluded using the seventh method, or outliers of the input data are excluded using the fourth method and outliers of the output data are excluded using the seventh method.

[0022] According to the 8th disclosure, in the 3rd disclosure, the first method is a method for detecting outliers based on the distance of each data point from the data used as a reference, using quantiles, and the second method is a method for detecting outliers using the local outlier factor method.

[0023] According to the 9th disclosure, in the 5th disclosure, the third method is a method for detecting outliers based on the distance of each data point from the baseline data using quantiles, the fourth method is a method for detecting outliers using the local outlier factor method, and the fifth method is a method for detecting outliers using quantiles.

[0024] According to the 10th disclosure, in the 7th disclosure, the third method and the sixth method are methods for detecting outliers based on the distance of each data point from the baseline data using quantiles, and the fourth method and the seventh method are methods for detecting outliers using the local anomaly factor method.

[0025] According to the 11th disclosure, in the 8th disclosure, in the first exclusion process, the quantile, which is a hyperparameter of the first method, is determined by grid search, or the number of neighboring points, which is a hyperparameter of the second method, is determined by grid search.

[0026] According to the 12th disclosure, in the 9th disclosure, in the second exclusion process, at least one of the quantiles as hyperparameters of the third method or as hyperparameters of the fifth method is determined by grid search, or at least one of the number of neighboring points as hyperparameters of the fourth method or as hyperparameters of the fifth method is determined by grid search.

[0027] According to the 13th disclosure, in the 10th disclosure, in the second exclusion process, at least one of the quantiles as hyperparameters of the third method or the quantiles as hyperparameters of the sixth method is determined by grid search; or at least one of the quantiles as hyperparameters of the third method or the number of neighboring points as hyperparameters of the seventh method is determined by grid search; or at least one of the number of neighboring points as hyperparameters of the fourth method or the quantiles as hyperparameters of the sixth method is determined by grid search; or at least one of the number of neighboring points as hyperparameters of the fourth method or the number of neighboring points as hyperparameters of the seventh method is determined by grid search.

[0028] The 14th disclosure provides a program for an information processing apparatus used to create a model that associates input data and output data contained in learning data to perform the following processes: a first exclusion process, excluding learning data from the learning dataset where the input data is an outlier; a second exclusion process, excluding learning data from the learning dataset where both the input data and the output data are outliers; a first model creation process, using the learning dataset from which the learning data was excluded in the first exclusion process to create a first model; a second model creation process, using the learning dataset from which the learning data was excluded in the second exclusion process to create a second model; and a model selection process, employing the model with better prediction accuracy between the first model and the second model.

[0029] The 15th disclosure provides an information processing apparatus for creating a model that associates input data and output data contained in learning data. The apparatus includes: an exclusion unit that excludes learning data from the learning dataset where the input data is an outlier, and a learning dataset that excludes both the input data and the output data from the learning dataset where both are outliers; a model creation unit that uses the learning dataset from which the learning data containing the input data has been excluded from being outliers to create a first model, and uses the learning dataset from which both the input data and the output data have been excluded from being outliers to create a second model; and a model selection unit that selects the model with better prediction accuracy between the first model and the second model.

[0030] <The Effects of the Invention>

[0031] This disclosure can improve the prediction accuracy of the model. Attached Figure Description

[0032] [ Figure 1A The graph compares and illustrates outliers caused by measurement errors with outliers measured on the prototype.

[0033] [ Figure 1B The graph compares and illustrates outliers caused by measurement errors with outliers measured on the prototype.

[0034] [ Figure 2 [ ] is a diagram illustrating the process by which an information processing device creates a model by eliminating outliers from a learning dataset.

[0035] [ Figure 3A This is an example of a system architecture diagram for a data processing system.

[0036] [ Figure 3B [Illustration 1] is a diagram of an information processing apparatus that illustrates an example of creating a model by excluding outliers.

[0037] [ Figure 4 [Illustration 1] is a diagram showing an example of the hardware structure of a server device and an information processing device.

[0038] [ Figure 5 [Illustration 1] is a diagram illustrating an example of the functional structure of an information processing device.

[0039] [ Figure 6 The figure illustrates an outlier detection method based on quartiles.

[0040] [ Figure 7 The figure illustrates the outlier detection method based on the Leave One Out approach.

[0041] [ Figure 8AThe diagram illustrates the methods for detecting outliers in explanatory and target variables.

[0042] [ Figure 8B ] is in Figure 8A The diagram illustrates the method for eliminating outliers detected in the sample.

[0043] [ Figure 9 This is an example of a flowchart illustrating the overall process of model creation.

[0044] [ Figure 10 ] is in Figure 9 Here is an example of a flowchart illustrating the process of excluding outliers in the input data and creating the model in step S2.

[0045] [ Figure 11 ] is in Figure 9 Here is an example of a flowchart illustrating the process of excluding outliers in the input and output data and creating the model in step S4.

[0046] [ Figure 12 ] is in Figure 9 Here is an example of a flowchart illustrating the process of excluding input data as outliers and creating a model in step S2.

[0047] [ Figure 13 ] is in Figure 12 An example of a flowchart illustrating the process of performing a grid search for the optimal quantile in step S215.

[0048] [ Figure 14 ] is in Figure 12 Here is an example of a flowchart illustrating the process of performing a grid search for the optimal number of neighboring points in step S218.

[0049] [ Figure 15 ] is in Figure 9 Here is an example of a flowchart illustrating the process of excluding learning data where both input and output data are outliers and creating a model in step S4.

[0050] [ Figure 16 ] is in Figure 15 An example of a flowchart illustrating the grid search process for the optimal quantile in step S416.

[0051] [ Figure 17 ] is in Figure 15 An example flowchart illustrating the process of performing a grid search for the optimal number of neighboring points in step S419.

[0052] [ Figure 18 ] is in Figure 15 Here is an example of a flowchart illustrating the process of excluding learning data where both input and output data are outliers and creating a model in step S421.

[0053] [ Figure 19 ] is in Figure 18 An example of a flowchart illustrating the process of performing a grid search for the optimal quantiles for the input and output data in step S4215A.

[0054] [ Figure 20 ] is in Figure 18 Here is an example of a flowchart illustrating the process of performing a grid search for the optimal quantile on the input data side and the optimal number of neighboring points on the output data side in step S4215B.

[0055] [ Figure 21 ] is in Figure 18 Here is an example of a flowchart illustrating the process of performing a grid search for the optimal number of nearest neighbors on the input data side and the optimal quantile on the output data side in step S4215C.

[0056] [ Figure 22 ] is in Figure 18 Here is an example of a flowchart illustrating the process of performing a grid search for the optimal number of neighboring points for the input and output data in step S4215D.

[0057] [ Figure 23A [Illustration 1] is an example of a diagram illustrating a method for excluding outliers from input data in the first exclusion process.

[0058] [ Figure 23B [Illustration 1] is an example of a diagram illustrating a combination of methods for excluding outliers in the input and output data during the second exclusion process. Detailed Implementation

[0059] The following description, as an example of a method for implementing the present invention, illustrates a model creation method performed by an information processing apparatus and an information processing apparatus.

[0060] <Supplementary Notes on Comparative Techniques>

[0061] In composition search techniques that search for optimal material compositions, a model is created to search for compositions with preferred properties using a training dataset. Material developers sometimes use compositions significantly different from previous methods to create prototypes and measure the properties of these prototypes to confirm whether they possess preferred characteristics. However, in many cases, no improvement in properties is observed, and sometimes no further search is conducted around that composition. In such cases, the composition of the prototype may become an outlier.

[0062] However, the data measured for the characteristics of the prototype are different from measurement errors. It is impossible to determine whether to exclude them when the data is acquired, and the judgment is often made after the development process has progressed. Therefore, omissions or forgetting to exclude them may occur.

[0063] Figure 1A , Figure 1B This is a diagram comparing and illustrating the outlier 101 caused by a measurement error with the outlier 102 measured on the prototype. For Figure 1A The outlier 101 shown is caused by a measurement error. Developers may mark the measurement data as "measurement error" when measuring features, or exclude it from the accumulated data on the spot.

[0064] In contrast, for Figure 1B The outlier 102 measured on the prototype shown corresponds to the coordinates of a data point relative to the composition (the ratio of materials 1 and 2). Although experiments were conducted with compositions significantly different from previous ones, for example, no improvement in properties was sometimes observed. If no improvement in properties is found, further searching around that composition will not be conducted. Furthermore, measurement data for this prototype should be excluded if it reduces the accuracy of the model or is detrimental to the search for components with desirable properties. As mentioned above, since outlier 102 is not caused by measurement errors and is not excluded from accumulated data on the spot, it is difficult to exclude it during measurement. Therefore, outlier 102 may be missed or forgotten to be excluded as the development process progresses.

[0065] Furthermore, when detecting outliers with the types and amounts of materials to be mixed as explanatory variables, a large number of explanatory variables exist. However, it is rare to perform experiments uniformly by changing the types and amounts of materials equally, and the training dataset tends to become sparse. Therefore, it is sometimes difficult to apply general outlier detection methods such as outlier detection based on standard deviation or quartiles. Additionally, there is a need for a method (an effective method) that can detect outliers even with large training datasets.

[0066] Therefore, when the information processing apparatus of this disclosure creates a model for component search using machine learning or the like, it excludes outliers from the learning dataset in the following manner.

[0067] Figure 2 This diagram illustrates the process by which an information processing device creates a model by eliminating outliers from a learning dataset.

[0068] In S100, the information processing device measures the distance of the explanatory variables (input data) of the multivariate dataset from the reference data and excludes the learning data containing input data that are outliers in the distance distribution. Outlier detection methods can utilize quartiles, etc. Alternatively, the information processing device can also use other methods such as the local outlier factor method to detect outliers.

[0069] In S200, the information processing device excludes learning data in which the input data is an outlier in the distance distribution and is also an outlier with respect to the target variable (output data).

[0070] In S300, the information processing device creates a model using the learning dataset that excluded the learning data in step S100, and uses the learning dataset that excluded the learning data in step S200 to create a model with better accuracy.

[0071] It should be noted that the processing of steps S100 and S200 can be performed either in any order or in parallel.

[0072] As described above, in the information processing apparatus of this embodiment, outliers can be easily eliminated even when general outlier detection methods are difficult to apply due to the sparsity of the learning dataset (explanatory variables). Therefore, it is easy to eliminate data measured for prototypes, etc.

[0073] • Because it can exclude learning data where the target variable and explanatory variables are outliers, it can also detect situations where measurement is stopped due to the deterioration of characteristics caused by experimental mixing (since it is a prototype, the explanatory variables are outliers, and the target variable is also an outlier due to the deterioration of characteristics).

[0074] • In information processing devices, since models are created and accuracy is verified by excluding outliers, outliers of explanatory variables can be excluded when they are detrimental to the model's predictive accuracy.

[0075] Since models are created by excluding learning data where only the explanatory variable is an outlier, and models are created by excluding learning data where both the explanatory and target variables are outliers, and models with better accuracy are used, it is possible to use models created by excluding outliers from the perspective of accuracy.

[0076] Furthermore, since the number of models created is not as large as when applying the Leave One Out method comprehensively to a large training dataset, outliers can be detected even when the training dataset has a large amount of data.

[0077] <About Terminology>

[0078] A learning dataset is a collection of data needed for machine learning, specifically a set of learning data consisting of a combination of input and output data. A learning dataset is also called a training dataset.

[0079] Outliers are observations that deviate significantly from other values. In this implementation, outliers may also include data that reduces model accuracy or hinders the search for desirable properties.

[0080] A model is a mechanism (program, file, etc.) in machine learning that derives output data from input data. Input and output data are correlated through the model. Models are sometimes also referred to as AI or artificial intelligence.

[0081] <System Architecture Example>

[0082] Figure 3A This is an example of a system architecture diagram for a data processing system 100. Figure 3A The data processing system 100 is a client-server system and includes a server device 10 and an information processing device 30. However, the information processing device 30 may be a general-purpose computer and is sometimes not included in the data processing system 100.

[0083] Server device 10 and information processing device 30 are connected in a communicative manner via a wide area network N1 such as the Internet. Server device 10 can be located in a cloud or data center, or it can be located locally. Server device 10 can be a web server that returns processing results in response to requests from information processing device 30. A server is a computer or software that provides information or processing results to requests from clients.

[0084] Server device 10 is one or more information processing devices. Server device 10 creates a model for component search based on learning data specified by developer 9 through information processing device 30. One feature of server device 10 is that it creates the model after appropriately excluding outliers from the learning data.

[0085] Server device 10 may have a data storage device pre-stored with learning data. Alternatively, server device 10 may obtain learning data from a data server or NAS (Network Attached Storage). Alternatively, developer 9 may operate information processing device 30 to send learning data to server device 10.

[0086] Server device 10 can support cloud computing. Cloud computing refers to a form of resource utilization on a network without being aware of specific hardware resources. Therefore, server device 10 does not need to be housed in a single enclosure or constituted as a set of devices. Server device 10 can distribute its functions to multiple information processing devices, or it can enable multiple information processing devices to each have all the functions and switch between them through load balancing and other methods.

[0087] The information processing device 30 is installed in facilities such as enterprises, research institutes, development departments, or factories, and is connected to network N2. Network N2 can be a LAN, Wi-Fi (registered trademark), wide area Ethernet (registered trademark), 4G, 5G, 6G, or other mobile phone networks.

[0088] Information processing device 30 is a machine learning or general-purpose computer used by developer 9. Here, developer 9 is the person who uses server device 10 to create a model based on learning data. Users of information processing device 30 may include not only the model creator but also the person undertaking the model creation task. Additionally, users of information processing device 30 may be component searchers.

[0089] A local application, specifically designed for a web browser or server device 10, operates within the information processing device 30. When the information processing device 30 executes a web browser, both the information processing device 30 and the server device 10 execute a web application. The web application operates through the coordination of a program based on a programming language (e.g., JavaScript) that works on the web browser and a program on the web server (server device 10) side. When the web application is executed, model creation can be performed by the server device 10 or by the information processing device 30 that receives the web application.

[0090] An application that is not executed unless installed in the information processing device 30 is called a local application. Similarly, in this embodiment, the application executed by the information processing device 30 can be either a web application or a local application. In this case, the process of creating a model by excluding outliers from the learning data can be performed by the server device 10 or by the information processing device 30 through a local application.

[0091] In addition, in this embodiment, such as Figure 3B As shown, the information processing device 30 can create a model by excluding outliers individually, rather than as... Figure 3A The client-server approach is shown. In this case, the local application that creates the model by excluding the model from the learning data operates in the information processing device 30. However, in this case, the information processing device 30 can also obtain the learning data from a data server on the network. Therefore, in Figure 3B In this configuration, the information processing device 30 is preferably also capable of being connected to a network.

[0092] Information processing device 30 may be, for example, a desktop PC, laptop PC, smartphone, PDA (Personal Digital Assistant), tablet terminal, etc., used by developer 9. Alternatively, information processing device 30 may be a device used for working with a web browser or local applications.

[0093] Additionally, developer 9 can appropriately differentiate the use. Figure 3A The shape and Figure 3B Its form.

[0094] • Abnormal values ​​are excluded by the information processing device 30, and a model is created by the server device 10.

[0095] • Outliers are excluded by server device 10, and a model is created by information processing device 30.

[0096] It should be noted that, in this embodiment, unless otherwise stated, it will be based on Figure 3B The form (processed separately by the information processing device 30) will be described.

[0097] <Hardware Structure Example>

[0098] Reference Figure 4 The hardware structure of the server device 10 and the information processing device 30 included in the data processing system 100 according to this embodiment will be described.

[0099] <<Information Processing System and Information Processing Device 30>>

[0100] Figure 4 This is a diagram illustrating an example of the hardware structure of the server device 10 and the information processing device 30 according to this embodiment. Figure 4 As shown, the server device 10 and the information processing device 30 are constructed from a computer 500 and include a CPU 501, ROM 502, RAM 503, HD (Hard Disk) 504, HDD (Hard Disk Drive) controller 505, display 506, external device connection I / F (Interface) 508, network I / F 509, bus 510, keyboard 511, indicating device 512, optical driver 514, and media I / F 516.

[0101] The CPU 501 controls the operation of the entire server device 10 and information processing device 30. The ROM 502 stores programs such as IPL used to drive the CPU 501. The RAM 503 serves as the working area for the CPU 501. The HDD 504 stores various data, including programs. The HDD controller 505 controls the reading and writing of various data to the HDD 504 according to the control of the CPU 501. The display 506 displays various information such as cursor, menu, window, characters, or images. The external device connection I / F 508 is an interface for connecting various external devices. In this case, the external device is, for example, a USB (Universal Serial Bus) memory or a printer. The network I / F 509 is an interface for data communication using network N2. The bus 510 is used for electrical connections such as… Figure 4 The CPU501 shown includes the address bus or data bus of each component.

[0102] Additionally, the keyboard 511 is an input unit with multiple keys for inputting characters, numbers, or various instructions. The indicator device 512 is an input unit for selecting and executing various instructions, selecting processing objects, moving the cursor, etc. The optical driver 514 controls the reading or writing of various data to an optical storage medium 513, which is an example of a removable recording medium. It should be noted that the optical storage medium 513 can be a CD, DVD, Blu-ray (registered trademark), etc. The media I / F 516 controls the reading or writing (storage) of data to a recording medium 515, such as flash memory.

[0103] <About Features>

[0104] Next, use Figure 5 The functional structure of the information processing device 30 according to this embodiment will be divided into modules for explanation. Figure 5 This is a diagram illustrating an example of the functional structure of the information processing apparatus 30 according to this embodiment. It should be noted that, although... Figure 5 The lines between functional modules indicate their relationship to each other, but do not restrict processing to proceed in the order connected by the lines.

[0105] The information processing device 30 includes a display control unit 31, an operation receiving unit 32, a measurement error data detection unit 33, a distance calculation unit 34, a grid search unit 35, a quantile outlier detection unit 36, a local outlier detection unit 37, an exclusion unit 38, a model creation unit 39, an accuracy calculation unit 40, a verification unit 41, and a model selection unit 42. These components are activated by instructions from the CPU 501 according to a program executed in the RAM 503. Figure 4 The unit that performs a function or plays a role by operating any one of the components shown.

[0106] In addition, the information processing device 30 has a learning data storage unit 49. The learning data storage unit 49 consists of... Figure 4 The HD504, RAM503, etc., shown are constructed. A learning dataset is stored in the learning data storage unit 49. The learning dataset is assumed to be divided into input data and output data. In the case of a multiple regression model, the input data is called the explanatory variable, and the output data is called the target variable. Although the terms explanatory variable and target variable are sometimes used in this embodiment, machine learning methods are not limited to multiple regression models. The learning data storage unit 49 can be located on a network accessible by the information processing device 30.

[0107] The display control unit 31 interprets the screen information stored in the local application and displays the screen information on the display 506. The operation receiving unit 32 receives various operations from the developer 9 in the various screens displayed on the display 506.

[0108] The measurement error data detection unit 33 acquires the learning dataset stored in the learning data storage unit 49 and detects measurement error data contained in the learning dataset. Measurement error data is assumed to be assigned a predetermined flag.

[0109] The distance calculation unit 34 calculates the distance between the learning data (which can be either the input data or the output data) and the reference data that is set as the reference data, in order to exclude outliers, for example, by means of quartiles.

[0110] The grid search unit 35 prepares all combinations of the specified hyperparameters. In this embodiment, the hyperparameters are either the quantile Q or the number of nearest neighbors N. It should be noted that a grid search is not mandatory.

[0111] The quantile outlier detection unit 36 ​​detects outliers from the learning data, for example, based on quartiles (which can be either the input or output data). Alternatively, when performing a grid search, the quantile outlier detection unit 36 ​​detects outliers from the learning data based on the quantile Q determined by the grid search unit 35.

[0112] The local outlier detection unit 37, for example, sets the number of neighboring points N to 3 and detects outliers (which can be either input or output data) from the learning dataset. Furthermore, when performing a grid search, the local outlier detection unit 37 detects outliers from the learning dataset based on the number of neighboring points N determined by the grid search unit 35. A local outlier is an outlier detected based on the density of data in space. Local density is the reciprocal of the average distance between the local point and its k nearest neighbors. Assuming there is an outlier point A, its local density is low because its local density is relatively large compared to its k nearest neighbors. Conversely, the local density is high because the k non-outlier points are relatively close to their neighbors. Therefore, in the case of outlier point A, its local density is low, but the local density of its k nearest neighbors is high. For non-outlier points, their local density is the same as the local density of their neighbors. Thus, if the ratio of the local density of a point to the local density of its neighbors differs by a threshold, it is judged as an outlier.

[0113] The exclusion unit 38 excludes learning data containing input data from the learning dataset, specifically outliers detected by the quantile outlier detection unit 36 ​​or the local outlier detection unit 37. Additionally, the exclusion unit 38 excludes learning data containing both input and output data, specifically outliers detected by the quantile outlier detection unit 36 ​​or the local outlier detection unit 37. It should be noted that exclusion does not require deleting the learning data itself; it only requires that it not be used in the model creation.

[0114] The model creation unit 39 uses a learning dataset that has excluded outliers to create a model for component search. Various machine learning methods can be used to create the model. Machine learning is a technique used to enable computers to learn like humans; it refers to the technique where a computer autonomously creates algorithms for data recognition and other judgments from pre-acquired learning data, and applies these algorithms to predict new data. The learning method used for machine learning can be any of supervised learning, unsupervised learning, or reinforcement learning (supervised learning is mainly used in this embodiment). Furthermore, it can be a combination of these learning methods; there is no limitation on the learning method used for machine learning. Machine learning methods include perceptrons, deep learning, support vector machines, logistic regression, Naive Bayes, decision trees, random forests, etc., and are not limited to the methods described in this embodiment.

[0115] For example, deep learning is an algorithm that, after predicting output data based on input data, adjusts the weights of a neural network using backpropagation to reduce errors compared to teacher data. In the case of component search, the developer prepares a learning dataset containing multiple learning datasets, using the component amounts of each material as input data and the material properties (strength properties, physical properties, electrical properties, corrosion resistance, etc.) as output data. The model creation unit 39 learns the correspondence between the input data and the output data.

[0116] The accuracy calculation unit 40 calculates the accuracy of the model created by the model creation unit 39. The learning dataset is pre-divided into L parts, and a model is created using the training data after removing one part (referred to as the test data). The accuracy calculation unit 40 inputs the excluded test data into the created model and compares its output with the output data from the test data, thereby calculating the accuracy of the created model.

[0117] In the case where the grid search unit 35 uses different quantiles or nearest neighbors to exclude outliers, a model is created for each learning dataset that excludes different outliers. The accuracy calculation unit 40 calculates the accuracy for each model. Furthermore, even when cross-validation is performed, the accuracy calculation unit 40 calculates the accuracy for each model.

[0118] The validation unit 41 creates L models based on the learning dataset using methods such as cross-validation, and averages the output accuracy of each model. By averaging, the differences in model accuracy caused by differences in the learning data are reduced.

[0119] The model selection unit 42 selects the model with better accuracy from those models created by excluding learning data where only the input data is an outlier and those created by excluding learning data where both the input and output data are outliers.

[0120] Outlier Detection Based on Quartiles

[0121] Reference Figure 6 The detection of outliers based on quartiles, which is a comparative technique used in this embodiment and is partially utilized, will be further explained. Figure 6 This diagram illustrates a method for detecting outliers (IQR) based on quartiles. Quartiles are a detection method that divides data into four equal quartiles (Q1, Q2, Q3) arranged in ascending order. Data points that are less than 1.5 times the IQR(Q3-Q1) relative to Q1, or greater than 1.5 times the IQR(Q3-Q1) relative to Q3, are considered outliers (IQR).

[0122] This method refers to detecting outliers based on the distribution of the data. However, even if the training data is excluded using this method, it may not be possible to exclude outliers that reduce the model's prediction accuracy.

[0123] Outlier Detection Based on the Leave One Out Method

[0124] Reference Figure 7 Further explanation is provided regarding the detection of outliers based on the Leave One Out method, which is a comparative technique used in this embodiment. Figure 7 This diagram illustrates an outlier detection method based on the Leave One Out approach. The Leave One Out method is a technique that excludes one outlier from the training data as an unknown and uses the remaining training data to build the model. Figure 7 There are 8 training datasets. Therefore, 8 models are created, and the accuracy of each model is compared when inputting unknown data 112. Unknown data 112 with poor accuracy is excluded as outliers. The difference between this and cross-validation is that the Leave One Out method involves a much larger number of data splits.

[0125] This method can be described as a way to improve the predictive accuracy of machine learning models. However, when the Leave One Out method is applied comprehensively to a large training dataset, the number of models to be created increases, which can lead to increased computational costs, and outliers in the components (explanatory variables) may not be eliminated.

[0126] <Summary of the outlier removal method in this embodiment>

[0127] Reference Figure 8A , Figure 8B The outline of the outlier removal method in this embodiment will be described. Figure 8A , Figure 8B This diagram illustrates outliers in the explanatory variables, outliers in the target variable, and methods for eliminating outliers. First, Figure 8A This diagram illustrates the method for detecting outliers in explanatory and target variables. The quantile outlier detection unit 36 ​​or the local outlier detection unit 37 detects outliers in the explanatory variables based on the distribution of the input data used as explanatory variables. Figure 8A Two outliers, 121 and 122, were detected from the explanatory variables (components).

[0128] As mentioned above, in a component system (explanatory variables), it is sometimes difficult to detect "prototypes (challenge components, components used for comparison)" as outliers. This is because there are many explanatory variables and the training dataset is sparse, resulting in a large data expansion in the search space, which is not normally distributed. In this case, the following approach is effective.

[0129] 1. Set a reference component and calculate the distance to that component (the definition of distance is arbitrary).

[0130] 2. Let the larger outliers in the distance distribution be equal to the outliers of the explanatory variables.

[0131] The quantiles used to identify outliers or the number of neighboring points in local outliers, N, can also be automatically determined as a hyperparameter through grid search.

[0132] Next, the quantile outlier detection unit 36 ​​or the local outlier detection unit 37 detects outliers in the distribution of the output data, which is the target variable. This is to extract cases where subsequent searches are stopped due to deterioration of characteristics. In this embodiment, since outliers are not excluded even if only the target variable is an outlier, cases where subsequent searches are stopped due to outliers and deterioration of characteristics can be extracted even if the explanatory variables are also outliers.

[0133] Two outliers, 123 and 124, were detected in the target variable. Excluding outliers based on the target variable is expected to improve the model's accuracy.

[0134] Figure 8B It is in Figure 8A The diagram illustrates the method for eliminating outliers detected in the data. The elimination unit 38 eliminates learning data containing outliers 121 and 122 that are explanatory variables, and the model creation unit 39 creates a model. Similarly, the elimination unit 38 eliminates learning data containing outliers 121 and 123 that are both outliers in both the explanatory and target variables, and the model creation unit 39 creates a model. The accuracy calculation unit 40 calculates the accuracy of these models, and the model selection unit 42 selects the model with the higher accuracy.

[0135] It should be noted that when excluding Figure 8B When all outliers are excluded, the model's search region may be narrow (although accuracy is improved, the prediction of the extrapolated region is weaker). Therefore, it is preferable to exclude only outliers where the characteristic (target variable) is biased towards the worse side of the characteristic from the target value.

[0136] <Action Steps>

[0137] The following is for reference Figures 9-11 The process of creating a model for component search by excluding outliers is explained.

[0138] <Processing without using grid search>

[0139] first, Figure 9 This is a flowchart illustrating the overall process of model creation. Hereinafter, the explanatory variables will be referred to as "input data," and the target variable as "output data."

[0140] First, the measurement error data detection unit 33 excludes learning data generated by measurement errors, such as those with added labels, from the learning dataset (S1).

[0141] The information processing device 30 performs a first exclusion process to remove outliers from the input data. Additionally, the information processing device 30 performs a first model creation process (S2) to create a model (an example of a first model) based on a learning dataset from which learning data containing outliers has been excluded. Details will be provided in [reference needed]. Figure 10 Please provide an explanation.

[0142] Next, the verification unit 41 performs the processing in step S2 multiple times to perform cross-validation (S3). That is, the verification unit 41 creates multiple models (an example of multiple first models) by changing the test data not used for model creation, based on the learning dataset after excluding the learning data containing input data as outliers. The verification unit 41 calculates the accuracy of each model calculated by the accuracy calculation unit 40.

[0143] Next, the information processing device 30 performs a second exclusion process to remove learning data where both the input and output data are outliers. The information processing device 30 then performs a second model creation process (S4) to create a model (an example of a second model) based on the learning dataset from which the learning data where both input and output data were excluded as outliers. Details will be provided in [reference needed]. Figure 11 Please provide an explanation.

[0144] The validation unit 41 performs the processing in step S4 multiple times to conduct cross-validation (S5). That is, the validation unit 41 creates multiple models (an example of multiple second models) by changing the test data not used for model creation, based on the learning dataset after excluding the learning data where the input data and output data are outliers, and calculates the accuracy of each model.

[0145] The model selection unit 42 performs model selection processing (S6) to select the model with better prediction accuracy from the models selected in step S2 and step S4. Prediction accuracy refers to the accuracy of the model with respect to data other than the training data. Known indicators of the accuracy of regression models include MAE (mean squared absolute error), MAPE (mean squared percentage error), WAPE (weighted absolute percentage error), MSE (mean squared error), or RMSE (root mean squared error).

[0146] Figure 10 It is in Figure 9 The flowchart in step S2, which involves excluding outliers from the input data and creating a model, is explained.

[0147] First, the input data is normalized to a value within a defined range of 0 to 1, or normalized to a value with a mean of "0" and a variance of "1" (S21).

[0148] Next, the distance calculation unit 34 acquires the input data used as a reference (S22). The input data used as a reference can be preset by the developer. Alternatively, the input data representing the median of the output data (target value) can be automatically selected. Since the output data represents the median, it is also expected that its input data is distributed around the center of multiple other input data.

[0149] Next, the distance calculation unit 34 calculates the distance from the reference input data for each input data (S23). The distance can be a distance capable of detecting outliers, such as Euclidean distance or Mahalanobis distance.

[0150] The quantile outlier detection unit 36 ​​detects outliers, for example, based on the distance in step S23 (an example of the first method), and the exclusion unit 38 excludes learning data containing input data that are outliers from the learning dataset (S24). Quintiles, etc., can be used instead of quartiles.

[0151] Instead of the processing in steps S23 and S24 described above, the local outlier detection unit 37 can detect outliers using the local outlier factor method (an example of the second method).

[0152] Then, the model creation unit 39 uses a learning dataset that focuses on the input data and excludes outliers to create a model (S25).

[0153] Figure 11 It is in Figure 9 The flowchart in step S4, which describes the process of excluding outliers in the input and output data and creating the model, is explained.

[0154] First, determine whether the output data is univariate (S41). Univariate means that the output data has one element, while multivariate means that the output data has multiple elements.

[0155] In the case of a single variable (as is the case in S41), the quantile outlier detection unit 36 ​​detects outliers in the output data based on quartiles, for example (an example of the fifth method) (S42). Quintiles, etc., can also be used instead of quartiles.

[0156] In the case of multiple variables (not in S41), the local outlier detection unit 37, for example, sets the number of neighbors to 3 and detects outliers in the output data based on the local outlier factor method (an example of the seventh method) (S43). A number of neighbors of 4, or a non-neighbor number of 3, can also be used. In the case of multiple variables (not in S41), the quantile outlier detection unit 36 ​​calculates the distance from the baseline data for each data point and, based on this distance, detects outliers, for example, based on quartiles (an example of the sixth method).

[0157] The exclusion unit 38 excludes learning data from the learning dataset where both the output and input data are outliers (S44). Then, the model creation unit 39 creates a model using the learning dataset that has excluded outliers while focusing on the input and output data.

[0158] <Processing the input data using grid search>

[0159] Next, refer to Figures 12-14 This section explains the process of using grid search to determine the optimal quantiles for the input data and to exclude outliers. The overall process of model creation is described. Figure 9 same.

[0160] Figure 12 It is in Figure 9 The flowchart in step S2, which describes the process of excluding input data as outliers and creating a model, is provided.

[0161] First, the input data is normalized to a value within a defined range of 0 to 1, or normalized to a value with a mean of "0" and a variance of "1" (S211).

[0162] Next, the mesh search unit 35 selects any one of the outlier detection methods (distance measurement from the reference, local anomaly factor method) (S212). The selection method can be preset by the developer, or both can be used and the method with better model accuracy can be selected.

[0163] When a distance measurement from a reference is selected (an example of the first method), the distance calculation unit 34 acquires the input data used as the reference (S213). The input data used as the reference can be preset by the developer. Alternatively, the input data representing the median value of the output data (target value) can be automatically selected.

[0164] Next, the grid search unit 35 determines the candidate quantiles q[1], q[2], q[3] ...... q[f] (S214). The candidate quantiles q[1], q[2], q[3] ...... q[f] are, for example, natural numbers such as 2, 3, 4, 5, etc.

[0165] The grid search unit 35 performs a grid search on the candidate quantiles to find the optimal quantile (S215). See details below. Figure 13 Please provide an explanation.

[0166] The model creation unit 39 determines the model with the best accuracy among those created using different quantiles (S216). The accuracy metric can be the same as in step S6.

[0167] When the local anomaly factor method is selected (an example of the second method), the grid search unit 35 determines the candidate n[1], n[2], n[3] ...... n[f] of the number of neighboring points (S217). The candidate n[1], n[2], n[3] ...... n[f] of the number of neighboring points are, for example, natural numbers such as 1, 2, 3, 4, 5, etc.

[0168] The grid search unit 35 performs a grid search using a candidate pair of neighbor number counts to find the optimal neighbor number count (S218). See details below. Figure 14 Please provide an explanation.

[0169] The model creation unit 39 determines the model with the best accuracy among those created using different numbers of neighboring points (S219). The accuracy metric can be the same as in step S6.

[0170] It should be noted that, in Figure 12 In the processing, a grid search can be performed on either the quantile or the number of neighboring points.

[0171] Figure 13 It is in Figure 12 The flowchart in step S215 illustrates the grid search process for the optimal quantile.

[0172] The grid search unit 35 sets the quantile candidates q[1], q[2], q[3] ...... q[f] as quantile Q, and repeats the following process (S2151) for all candidates.

[0173] The distance calculation unit 34 calculates the distance from the reference input data for each input data (S2152). The distance can be Euclidean distance, Mahalanobis distance, or other distances that can detect outliers.

[0174] Quantile outlier detection unit 36 ​​detects outliers based on quantile Q based on the distance in step S2152 (S2153).

[0175] The exclusion unit 38 excludes the learning data with the outlier (input data), and the model creation unit 39 creates the model (S2154).

[0176] Calculate the accuracy of the model created by the accuracy calculation unit 40 (S2155). The accuracy index can be the same as in step S6.

[0177] As described above, models are created for each quantile candidate q[1], q[2], q[3], ..., q[f], and the accuracy of each model is calculated. (The process returns to...) Figure 12 In step S216, the model creation unit 39 determines the model with the best accuracy (the optimal quantile).

[0178] Figure 14 It is in Figure 12 The flowchart in step S218 illustrates the process of performing a grid search for the optimal number of neighboring points.

[0179] The grid search unit 35 sets the candidate n[1], n[2], n[3] ...... n[f] of the optimal number of neighboring points to the number of neighboring points N, and repeats the following process (S2181) for all candidates.

[0180] The local outlier detection unit 37 detects outliers based on the number of neighboring points N (S2182).

[0181] The exclusion unit 38 excludes the learning data with the outlier (input data), and the model creation unit 39 creates the model (S2183).

[0182] The accuracy calculation unit 40 calculates the accuracy of the created model (S2184). The accuracy index can be the same as in step S6.

[0183] As described above, models are created for candidate neighbor numbers n[1], n[2], n[3], ..., n[f], and the accuracy of each model is calculated. (The process returns to...) Figure 12 In step S219, the model creation unit 39 determines the model with the best accuracy (the optimal number of neighboring points).

[0184] Due to the passage Figure 13 or Figure 14 The processing in the middle creates an optimal model using the best quantiles or the best number of nearest neighbors, so the processing proceeds to... Figure 9 Step S3 in the process is used to perform cross-validation. It should be noted that the processing in step S4 can be combined with... Figure 11 The processing described in the previous section is the same, but a grid search can also be performed on the output data as described below.

[0185] <Processing using grid search for input and output data>

[0186] Next, refer to Figures 15-17This section explains how to use grid search to determine the optimal quantiles or number of nearest neighbors for the input and output data, and how to exclude outliers. The entire model creation process is described. Figure 9 same.

[0187] Figure 15 It is in Figure 9 The flowchart in step S4, which describes the process of excluding learning data where both input and output data are outliers and creating a model, is provided.

[0188] First, determine whether the output data is a single variable (S411).

[0189] When the output data is a single variable (as in S411), the input data is normalized to a value within a defined range of 0 to 1, or normalized to a value with a mean of "0" and a variance of "1" (S412).

[0190] Next, the mesh search unit 35 selects any one of the outlier detection methods (distance measurement from the reference, local anomaly factor method) (S413). The selection method can be preset by the developer, or both can be used and the method with better model accuracy can be selected.

[0191] When a distance measurement from a reference is selected (an example of the third method), the distance calculation unit 34 acquires the input data used as the reference (S414). The input data used as the reference can be preset by the developer. Alternatively, the input data representing the median value can be automatically selected as the output data (target value).

[0192] Next, the grid search unit 35 determines the candidate quantiles q[1], q[2], q[3] ...... q[f] (S415). The candidate quantiles q[1], q[2], q[3] ...... q[f] are, for example, natural numbers such as 2, 3, 4, 5, etc.

[0193] The grid search unit 35 performs a grid search on the candidate quantiles to find the optimal quantile (S416). For details, please refer to [reference needed]. Figure 16 Please provide an explanation.

[0194] The model creation unit 39 determines the model with the best accuracy among those created using different quantiles (S417). The accuracy metric can be the same as in step S6.

[0195] In the case of selecting the local outlier factor method (an example of the fourth method), the grid search unit 35 determines the candidate n[1], n[2], n[3] ...... n[f] of the number of neighboring points (S418). The candidate n[1], n[2], n[3] ...... n[f] of the number of neighboring points are, for example, natural numbers such as 1, 2, 3, 4, 5, etc.

[0196] The grid search unit 35 performs a grid search using a candidate pair of neighbor number counts to find the optimal neighbor number count (S419). For details, please refer to [reference needed]. Figure 17 Please provide an explanation.

[0197] The model creation unit 39 determines the model with the best accuracy among those created using different numbers of neighboring points (S420). The accuracy metric can be the same as in step S6.

[0198] When it is determined in step S411 that the output data is multivariate (if not in S411), the processing flow is the same as in the case of univariate data (S421). However, as a method for detecting outliers in the output data, distance measurement from the reference and the local outlier factor method can be selected.

[0199] It should be noted that, in Figure 15 In the processing, a grid search can be performed on either the quantile or the number of neighboring points.

[0200] Figure 16 It is in Figure 15 The flowchart in step S416 illustrates the grid search process for the optimal quantile.

[0201] The grid search unit 35 sets the quantile candidates q[1], q[2], q[3] ...... q[f] as quantile Q, and repeats the following process (S4161) for all candidates.

[0202] The distance calculation unit 34 calculates the distance from the reference input data for each input data (S4162). The distance can be Euclidean distance, Mahalanobis distance, or other distances that can detect outliers.

[0203] Quantile outlier detection unit 36 ​​detects outliers based on quantile Q based on the distance in step S4162 (S4163).

[0204] Similarly, the quantile outlier detection unit 36 ​​detects outliers based on quantile Q for the output data (univariate) (S4164).

[0205] The exclusion unit 38 excludes learning data in which both input and output data are outliers, and the model creation unit 39 creates a model (S4165).

[0206] Calculate the accuracy of the model created by the accuracy calculation unit 40 (S4166). The accuracy index can be the same as in step S6.

[0207] As described above, models are created for each quantile candidate q[1], q[2], q[3], ..., q[f], and the accuracy of each model is calculated. (The process returns to...) Figure 15 In step S417, the model creation unit 39 determines the model with the highest accuracy (the optimal quantile).

[0208] Figure 17 It is in Figure 15 The flowchart in step S419 illustrates the process of performing a grid search for the optimal number of neighboring points.

[0209] The grid search unit 35 sets the quantile candidates q[1], q[2], q[3], ..., q[f] as the quantile Q, and sets the neighbor number candidates n[1], n[2], n[3], ..., n[f] as the neighbor number N (S4191). The grid search unit 35 repeats the following processing for all combinations of quantile candidates and neighbor number candidates (S4192, S4193).

[0210] The local outlier detection unit 37 detects outliers in the input data when the number of neighboring points is N (S4194).

[0211] Quantile outlier detection unit 36 ​​detects outliers based on quantile Q in the output data (S4195).

[0212] The exclusion unit 38 excludes learning data in which both input and output data are outliers, and the model creation unit 39 creates a model (S4196).

[0213] The accuracy calculation unit 40 calculates the accuracy of the model created given the number of neighboring points N and the quantile Q (S4197). The accuracy index can be the same as in step S6.

[0214] Models are created for all combinations of candidates based on the number of nearest neighbors and the quantiles, and the accuracy of each model is calculated. (The process then returns to...) Figure 15 In step S420, the model creation unit 39 determines the model with the highest accuracy (optimal quantiles, optimal number of neighboring points).

[0215] pass Figure 16 or Figure 17 The processing utilizes optimal quantiles and the optimal number of nearest neighbors to exclude outliers and create an optimal model. The processing then proceeds to... Figure 9Step S5 involves cross-validation.

[0216] <<Case where the output data has multiple variables>>

[0217] Figure 18 It is in Figure 15 The flowchart example illustrates the process of excluding learning data where both input and output data are outliers and creating the model in step S421 (output data is multivariate). Figure 18 In the middle, sometimes it mainly refers to and Figure 15 The differences between steps S412 to S420 will be explained.

[0218] exist Figure 18 In the processing, in step S4212, the mesh search unit 35 selects an outlier detection method (distance measurement from the reference, local anomaly factor method) for the input data and output data respectively (S4212). Therefore, the processing branches after step S4212 are the following four cases.

[0219] A. "Input data: distance from the benchmark (an example of the third method), Output data: distance from the benchmark (an example of the sixth method)"

[0220] B. Input data: distance from the baseline (an example of the third method); Output data: local anomaly factor method (an example of the seventh method)

[0221] C. Input data: Local anomaly factor method (an example of the fourth method); Output data: Distance from the baseline (an example of the sixth method)

[0222] D. Input data: Local anomaly factor method (an example of the fourth method), Output data: Local anomaly factor method (an example of the seventh method)

[0223] A. When "Input data: distance from the reference, output data: distance from the reference" is selected, the distance calculation unit 34 obtains the input data and output data as the reference (S4213A).

[0224] Next, the grid search unit 35 determines the candidate quantiles q1[1], q1[2], q1[3] ......q1[f] on the input data side and the candidate quantiles q2[1], q2[2], q2[3] ......q2[f] on the output data side (S4214A).

[0225] The grid search unit 35 performs a grid search (S4215A) on the optimal combination from all combinations of candidate quantiles on the input data side and candidate quantiles on the output data side. For details, please refer to [reference needed]. Figure 19Please provide an explanation.

[0226] The model creation unit 39 determines the model with the highest accuracy among those created by excluding input data whose quantiles on the input data side are judged as outliers and output data whose quantiles on the output data side are judged as outliers (S4216A). The accuracy metric can be the same as in step S6.

[0227] B. When "Input data: distance from the reference, output data: local anomaly factor method" is selected, the distance calculation unit 34 obtains the input data as the reference (S4213B).

[0228] Next, the grid search unit 35 determines the candidate q[1], q[2], q[3] ...... q[f] of the quantiles on the input data side and the candidate n[1], n[2], n[3] ...... n[f] of the number of neighboring points on the output data side (S4214B).

[0229] The grid search unit 35 performs a grid search (S4215B) on the optimal combination from all combinations of candidate quantiles on the input data side and candidate numbers of neighboring points on the output data side. Details will be provided in [reference needed]. Figure 20 Please provide an explanation.

[0230] The model creation unit 39 determines the model with the highest accuracy from the models created by excluding input data that is judged as outlier based on the quantiles of the input data and output data that is judged as outlier based on the number of neighboring points of the output data (S4216B). The accuracy metric can be the same as in step S6.

[0231] C. When "Input data: Local anomaly factor method, Output data: Distance from the reference" is selected, the distance calculation unit 34 obtains the output data as the reference (S4213C).

[0232] Next, the grid search unit 35 determines the candidate n[1], n[2], n[3]...... n[f] of the number of neighboring points on the input data side and the candidate q[1], q[2], q[3]...... q[f] of the quantiles on the output data side (S4214C).

[0233] The grid search unit 35 performs a grid search (S4215C) on the optimal combination among all combinations of the candidate number of neighboring points on the input data side and the candidate quantiles on the output data side. For details, please refer to [reference needed]. Figure 21 Please provide an explanation.

[0234] The model creation unit 39 determines the model with the highest accuracy from models created by excluding input data that is judged as outlier based on the number of neighboring points on the input data side and output data that is judged as outlier based on the quantiles on the output data side (S4216C). The accuracy metric can be the same as in step S6.

[0235] D. When “Input data: Local anomaly factor method, Output data: Local anomaly factor method” is selected, the grid search unit 35 determines the candidate n1[1], n1[2], n1[3] ...... n1[f] of the neighboring points on the input data side and the candidate n2[1], n2[2], n2[3] ...... n2[f] of the neighboring points on the output data side (S4214D).

[0236] Next, the grid search unit 35 performs a grid search (S4215D) on the optimal combination among all candidate combinations of the number of neighboring points on the input data side and the number of neighboring points on the output data side. Details will be provided in [reference needed]. Figure 22 Please provide an explanation.

[0237] The model creation unit 39 determines the model with the highest accuracy from models created by excluding input data where the number of neighboring points on the input data side is judged as outlier and output data where the number of neighboring points on the output data side is judged as outlier (S4216D). The accuracy metric can be the same as in step S6.

[0238] It should be noted that, in Figure 18 Mesh search can be omitted in steps S4215A, S4215B, S4215C, and S4215D, or it can be performed in any one or more steps.

[0239] Figure 19 It is in Figure 18 The flowchart in step S4215A illustrates the process of performing a grid search for the optimal quantiles based on the input and output data.

[0240] The grid search unit 35 sets the candidate quantiles q1[1], q1[2], q1[3], ..., q1[f] on the input data side as quantile Q1, and sets the candidate quantiles q2[1], q2[2], q2[3], ..., q2[f] on the output data side as quantile Q2 (S42151A). The grid search unit 35 repeats the following processing for all combinations of Q1 and Q2 (S42152A, S42153A).

[0241] The distance calculation unit 34 calculates the distance from the reference data for each input data (S42154A). The distance can be Euclidean distance, Mahalanobis distance, or other distances that can detect outliers.

[0242] Quantile outlier detection unit 36 ​​detects outliers based on quantile Q1 based on the distance in step S42154A (S42155A).

[0243] Steps S42156A and S42157A can also be the same.

[0244] The exclusion unit 38 excludes learning data in which both input and output data are outliers, and the model creation unit 39 creates a model (S42158A).

[0245] The accuracy calculation unit 40 calculates the accuracy of the model created under the conditions of quantiles Q1 and Q2 (S42159A). The accuracy index can be the same as in step S6.

[0246] As described above, models are created for all combinations of candidate quantiles on the input data side and candidate quantiles on the output data side, and the accuracy of each model is calculated. Since the processing returns to... Figure 18 In step S4216A, the model creation unit 39 determines the model with the best accuracy (the best quantiles for the input and output data).

[0247] Figure 20 It is in Figure 18 The flowchart in step S4215B illustrates the process of performing a grid search for the optimal quantile on the input data side and the optimal number of neighboring points on the output data side.

[0248] The grid search unit 35 sets the candidate quantiles q[1], q[2], q[3], ..., q[f] on the input data side as quantile Q, and sets the candidate number of neighboring points n[1], n[2], n[3], ..., n[f] on the output data side as the number of neighboring points N (S42151B). The grid search unit 35 repeats the following processing for all combinations of Q and N (S42152B, S42153B).

[0249] The distance calculation unit 34 calculates the distance from the reference data for each input data (S42154B). The distance can be Euclidean distance, Mahalanobis distance, or other distances that can detect outliers.

[0250] Quantile outlier detection unit 36 ​​detects outliers based on quantile Q based on the distance in step S42154B (S42155B).

[0251] Similarly, the local outlier detection unit 37 uses the local outlier factor method to detect outliers based on the number of neighboring points N for the output data (multivariate) (S42156B).

[0252] The exclusion unit 38 excludes learning data in which both input and output data are outliers, and the model creation unit 39 creates a model (S42158B).

[0253] The accuracy calculation unit 40 calculates the accuracy of the model created given the quantile Q and the number of neighboring points N (S42159B). The accuracy index can be the same as in step S6.

[0254] As described above, models are created for all combinations of candidates for quantiles and candidates for the number of nearest neighbors, and the accuracy of each model is calculated. Since the processing returns to... Figure 18 In step S4216B, the model creation unit 39 determines the model with the best accuracy (the best quantile for the input data and the best number of neighboring points for the output data).

[0255] Figure 21 It is in Figure 18 The flowchart in step S4215C describes the grid search process for the optimal number of nearest neighbors on the input data side and the optimal quantile on the output data side.

[0256] Next, the grid search unit 35 sets the candidate numbers n[1], n[2], n[3], ..., n[f] of the number of neighboring points on the input data side to the number of neighboring points N, and sets the candidate numbers q[1], q[2], q[3], ..., q[f] of the quantiles on the output data side to the quantiles Q (S42151C). The grid search unit 35 repeats the following processing for all combinations of N and Q (S42152C, S42153C).

[0257] The local outlier detection unit 37 uses the local outlier factor method to detect outliers based on the number of neighboring points N for the input data (S42154C).

[0258] The distance calculation unit 34 calculates the distance from the reference data to each output data (S42155C). The distance can be Euclidean distance, Mahalanobis distance, or other distances that can detect outliers.

[0259] Quantile outlier detection unit 36 ​​detects outliers based on quantile Q based on the distance in step S42155C (S42156C).

[0260] The exclusion unit 38 excludes learning data in which both input and output data are outliers, and the model creation unit 39 creates a model (S42158C).

[0261] The accuracy calculation unit 40 calculates the accuracy of the model created given the number of neighboring points N and the quantile Q (S42159C). The accuracy index can be the same as in step S6.

[0262] As described above, models are created for all combinations of candidates based on the number of nearest neighbors and the quantiles, and the accuracy of each model is calculated. Since the processing returns to... Figure 18 In step S4216C, the model creation unit 39 determines the model with the best accuracy (the optimal number of neighboring points for the input data and the optimal quantiles for the output data).

[0263] Figure 22 It is in Figure 18 The flowchart in step S4215D illustrates the process of performing a grid search for the optimal number of neighboring points based on the input and output data.

[0264] The grid search unit 35 sets the candidate numbers of neighboring points n1[1], n1[2], n1[3] ...... n1[f] as the number of neighboring points N1, and sets the candidate numbers of neighboring points n2[1], n2[2], n2[3] ...... n2[f] as the number of neighboring points N2 (S42151D). The grid search unit 35 repeats the following processing for all combinations of N1 and N2 (S42152D, S42153D).

[0265] The local outlier detection unit 37 uses the local outlier factor method to detect outliers based on the number of neighboring points N1 for the input data (S42154D).

[0266] The local outlier detection unit 37 detects outliers based on the number of neighboring points N2 using the local outlier factor method (S42155D) for the output data.

[0267] The exclusion unit 38 excludes learning data in which both input and output data are outliers, and the model creation unit 39 creates a model (S42158D).

[0268] The accuracy calculation unit 40 calculates the accuracy of the model created with the number of neighboring points N1 and N2 (S42159D). The accuracy index can be the same as in step S6.

[0269] As described above, models are created for all candidate combinations of the number of neighboring points on the input data side and the number of neighboring points on the output data side, respectively, and the accuracy of each model is calculated. Since the processing returns to... Figure 18 In step S4216D, the model creation unit 39 determines the model with the best accuracy (the optimal number of neighboring points for the input and output data).

[0270] pass Figures 19-22 The processing of either of these factors involves removing outliers from both the optimal quantile or number of nearest neighbors on the input data side and the optimal quantile or number of nearest neighbors on the output data side to create the best model. The processing then proceeds to... Figure 9 Step S5 involves cross-validation.

[0271] <Supplementary Explanation Regarding Combinations of Methods for Excluding Outliers>

[0272] Figure 23A and Figure 23B Examples of combinations of various methods in the first exclusion process and the second exclusion process are shown. Figure 23A This illustrates a method for excluding outliers in the input data during the first exclusion process. Multiple methods for excluding outliers in the input data are available, and any one of them can be used. It should be noted that outlier exclusion based on quantiles (the first method) and outlier exclusion based on local anomaly factors (the second method) are merely examples.

[0273] Figure 23B This illustrates a combination of methods for excluding outliers in the input and output data during the second exclusion process. First, multiple methods for excluding outliers from the input data are prepared, and any one of them can be used arbitrarily.

[0274] When the output data is a single variable, this embodiment exemplifies outlier exclusion based on quantiles as an example of a method for excluding outliers in the output data. It should be noted that outlier exclusion based on quantiles (the third method) is merely an example. Therefore, outliers in both the input and output data can be excluded by combining any of the third or fourth methods with the fifth method. Furthermore, when the output data is a single variable, multiple methods for excluding outliers can be prepared.

[0275] In cases where the output data is multivariate, this embodiment exemplifies outlier removal methods based on quantiles and outlier removal based on local anomaly factors as examples of methods for removing outliers from the output data. It should be noted that outlier removal based on quantiles (the sixth method) and outlier removal based on local anomaly factors (the seventh method) are merely examples. Therefore, outliers in both the input and output data can be removed by combining any of the third or fourth methods with any of the sixth or seventh methods.

[0276] <Main Effects>

[0277] In the information processing apparatus of this embodiment, outliers can be easily eliminated even when general outlier detection methods are difficult to apply due to the sparsity of the learning dataset (explanatory variables). Therefore, it is easy to eliminate data measured for prototypes, etc.

[0278] • Because it can exclude learning data where the target variable and explanatory variables are outliers, it can also detect situations where measurement is stopped due to the deterioration of characteristics caused by experimental mixing (since it is a prototype, the explanatory variables are outliers, and the target variable is also an outlier due to the deterioration of characteristics).

[0279] • In information processing devices, since models are created and accuracy is verified by excluding outliers, outliers of explanatory variables can be excluded when they are detrimental to the model's predictive accuracy.

[0280] Since models are created by excluding learning data where only the explanatory variable is an outlier, and models are created by excluding learning data where both the explanatory and target variables are outliers, and models with better accuracy are used, it is possible to use models created by excluding outliers from the perspective of accuracy.

[0281] Additionally, because the number of models created is not that large, outliers can be detected even when the amount of data in the training dataset is large.

[0282] <Other Application Examples>

[0283] While the above embodiments have described the best mode for carrying out the invention, the invention is not limited to these embodiments, and various modifications and substitutions may be made without departing from the spirit of the invention.

[0284] For example, in this embodiment, an example of detecting outliers from a learning dataset of a model primarily used for component search has been described, but it is not limited to this type of materials informatics; the learning dataset for excluding outliers may include input and output data used for machine learning.

[0285] Furthermore, using quartiles and local outlier methods to exclude outliers is just one example. For univariate cases, standard deviation, the Smirnov-Grubbs test, and Hotelling's theory can be used. For multivariate cases, standard deviation, cluster analysis, and Hotelling's T2 method can be used to detect outliers. In other words, the methods for detecting outliers are not limited to two (preferably at least two), and any of several different methods can be used.

[0286] Alternatively, in a client-server configuration, the user can connect the information processing device 30 to the server device 10 and request the creation of a model. The server device 10 then creates the model and sends it to the information processing device 30. The server device 10 can store the model internally. In this case, when interpretable variables are sent from the information processing device 30, the server device 10 can return the target variables to the information processing device 30. The information processing device 30 can then appropriately execute the web application provided by the server device 10.

[0287] Furthermore, to facilitate understanding of the processing performed by the information processing device 30 and the server device 10, the processes are divided according to their main functions as follows: Figure 5 The structural examples shown are provided. This invention is not limited to the way processing units are divided or named. The processing of the information processing device 30 and the server device 10 can also be divided into more processing units according to the processing content. Alternatively, it can be divided into a single processing unit comprising more processing steps.

[0288] Each function of the above-described embodiments can be implemented by one or more processing circuits. Here, "processing circuit" in this specification refers to a processor, such as a processor implemented by electronic circuits, that is programmed to perform the functions using software, including processing circuits of devices such as ASICs (Application Specific Integrated Circuits), DSPs (Digital Signal Processors), FPGAs (Field Programmable Gate Arrays), or conventional circuit modules designed to perform the above-described functions.

[0289] This application claims priority to Japanese Patent Application No. 2023-134755, filed with the Japan Patent Office on August 22, 2023, the entire contents of which are incorporated herein by reference.

[0290] Symbol Explanation

[0291] 10 server devices

[0292] 30 Information processing devices

[0293] 100 Data Processing System.

Claims

1. A model creation method, wherein an information processing device creates a model that associates input data and output data contained in learning data, the model creation method performing the following processing: The first exclusion process involves excluding learning data from the learning dataset where the input data is an outlier. The second exclusion process involves excluding learning data from the learning dataset where the input data and output data are outliers. The first model creation process uses the learning dataset in which the learning data was excluded in the first exclusion process to create a first model; The second model creation process uses the learning dataset in which the learning data was excluded in the second exclusion process to create a second model; as well as For model selection, the model with better prediction accuracy between the first model and the second model is selected.

2. The model creation method according to claim 1, wherein, In the first exclusion process, any one of a plurality of different methods is used to exclude outliers from the input data.

3. The model creation method according to claim 2, wherein, In the first exclusion process, outliers in the input data are excluded using either of at least two different first or second methods.

4. The model creation method according to any one of claims 1 to 3, wherein, In the second exclusion process, any one of several different methods is used to exclude outliers from the input data. When the output data is a single variable, outliers in the output data are excluded using a predetermined method.

5. The model creation method according to claim 4, wherein, In the second exclusion process, outliers in the input data are excluded using at least two different third or fourth methods. If the output data is a single variable, a fifth method is used to exclude outliers in the output data. In the second exclusion process, The third method is used to exclude outliers from the input data, and the fifth method is used to exclude outliers from the output data, or... The fourth method is used to exclude outliers from the input data, and the fifth method is used to exclude outliers from the output data.

6. The model creation method according to any one of claims 1 to 3, wherein, In the second exclusion process, any one of several different methods is used to exclude outliers from the input data. When the output data is multivariate, any one of several different methods is used to exclude outliers from the output data.

7. The model creation method according to claim 6, wherein, In the second exclusion process, outliers in the input data are excluded using at least two different third or fourth methods. If the output data is multivariate, outliers in the output data are excluded using at least two different sixth or seventh methods. In the second exclusion process, Use the third method to exclude outliers from the input data, use the sixth method to exclude outliers from the output data, or... Use the fourth method to exclude outliers from the input data, use the sixth method to exclude outliers from the output data, or... Use the third method to exclude outliers from the input data, use the seventh method to exclude outliers from the output data, or... The fourth method is used to exclude outliers from the input data, and the seventh method is used to exclude outliers from the output data.

8. The model creation method according to claim 3, wherein, The first method is a method of detecting outliers based on the distance of each data point from the baseline data point, using quantiles. The second method is a method for detecting outliers using the local anomaly factor method.

9. The model creation method according to claim 5, wherein, The third method is a method for detecting outliers based on the distance of each data point from the baseline data point, using quantiles. The fourth method is a method for detecting outliers using the local anomaly factor method. The fifth method is a method of detecting outliers using quantiles.

10. The model creation method according to claim 7, wherein, The third and sixth methods are outlier detection methods based on the distance of each data point from the baseline data point, using quantiles. The fourth and seventh methods are methods for detecting outliers using the local anomaly factor method.

11. The model creation method according to claim 8, wherein, In the first exclusion process The quantiles, which are hyperparameters of the first method, are determined by grid search, or... The number of neighboring points, which is a hyperparameter of the second method, is determined by grid search.

12. The model creation method according to claim 9, wherein, In the second exclusion process, At least one of the quantiles, either as hyperparameters of the third method or as hyperparameters of the fifth method, is determined by grid search. The grid search determines at least one of the number of neighboring points as a hyperparameter of the fourth method or the quantile as a hyperparameter of the fifth method.

13. The model creation method according to claim 10, wherein, In the second exclusion process, At least one of the quantiles, either as hyperparameters of the third method or as hyperparameters of the sixth method, is determined by grid search. The quantile, which is a hyperparameter of the third method, or the number of neighboring points, which is a hyperparameter of the seventh method, is determined by grid search. Determine at least one of the following: the number of neighboring points as a hyperparameter of the fourth method or the quantiles as a hyperparameter of the sixth method, through grid search. The number of neighboring points, either a hyperparameter of the fourth method or a hyperparameter of the seventh method, is determined by grid search.

14. A program for an information processing apparatus used to create a model that associates input data contained in learning data with output data to perform the following processing: The first exclusion process involves excluding learning data from the learning dataset where the input data is an outlier. The second exclusion process involves excluding learning data from the learning dataset where the input data and output data are outliers. The first model creation process uses the learning dataset in which the learning data was excluded in the first exclusion process to create a first model; The second model creation process uses the learning dataset in which the learning data was excluded in the second exclusion process to create a second model; as well as For model selection, the model with better prediction accuracy between the first model and the second model is selected.

15. An information processing apparatus for creating a model that associates input data contained in learning data with output data, the information processing apparatus comprising: The exclusion unit excludes learning data from the learning dataset where the input data is an outlier, and excludes learning data from the learning dataset where both the input data and the output data are outliers. The model creation unit creates a first model using the learning dataset from which the input data is excluded as outlier, and creates a second model using the learning dataset from which both the input data and the output data are excluded as outliers. as well as The model selection unit selects the model with better prediction accuracy from the first model and the second model.

Images