Evaluation device, inferring device, evaluation method, program, and non-transitory computer readable medium

Abstract

To evaluate generalization performance of an NNP that learns a first principle calculation of various chemical structures.SOLUTION: An evaluation device includes: an inferring part for inputting an atomic structure to an already trained model for inferring a physical property value of an atomic structure from the atomic structure and successively propagating it, and acquiring an inferred result; and an evaluation part for comparing the inferred result with the physical property values acquired by executing a quantum chemical calculation to the atomic structure and evaluating the inferred result. The inferring part acquires each inferred result for the atomic structures that belong to multiple domains. The evaluation part evaluates each of the atomic structures that belong to the multiple domains by using the physical property values. Datasets of the atomic structures and the physical property values that the inferring part and the evaluation part use for inference and evaluation are not used for training the already trained model.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to an evaluation device, an inference device, an evaluation method, a program, and a non-transitory computer-readable medium. [Background technology]

[0002] To obtain the energy of molecules, crystals, etc., it is necessary to perform calculations using methods such as first-principles calculations. Research is being conducted on NNP (Neural Network Potential) as a method for realizing inferences based on the results obtained from first-principles calculations.

[0003] However, a common problem with neural network models is that NNP models that have been trained to achieve high accuracy are vulnerable to extrapolation problems. As a result, training a model to achieve high accuracy leads to a decrease in generalization performance, while training a model to improve generalization performance leads to greater accuracy issues. Therefore, metrics are needed to evaluate both the generalization performance and accuracy of NNPs, but no appropriate metrics have yet been proposed. While methods for evaluating accuracy and generalization performance have been reported for NNPs trained on limited domains, such as specific element combinations, no methods have been reported for accurately inferring various chemical structures from NNPs trained on results obtained through first-principles calculations of various chemical structures. [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] J. Behler, “Neural Network Potential-Energy Surfaces in Chemistry: A Tool for Large-Scale Simulations,” J. Phys. Chem. Chem. Phys., 2011, 13, 17930-17955 [Non-patent document 2] JS Smith, et. al., “ANI-1: an extensible neural network potential with DFT accuracy at force field computational cost,” Chem. Sci., 2017, 8, 3192-3203 [Non-patent document 3] CL Zitnick, et. al., “An Introduction to Electrocatalyst Design using Machine Learning for Renewable Energy Storage,” arXiv, 2010.09435, https: / / arxiv.org / abs / 2010.09435 Summary of the Invention [Problem to be solved by the invention]

[0005] The present disclosure provides an evaluation device for evaluating the generalization performance and accuracy of NNPs that have learned various chemical structures. [Means for solving the problem]

[0006] According to one embodiment, the evaluation device includes an inference unit that inputs an atomic structure to a trained model that infers physical property values ​​of the atomic structure from the atomic structure, performs forward propagation, and acquires an inference result, and an evaluation unit that compares and evaluates the inference result with physical property values ​​acquired by performing quantum chemical calculations on the atomic structure. The inference unit acquires the inference results for each of the atomic structures belonging to a plurality of domains. The evaluation unit performs evaluation using the physical property values ​​for each of the atomic structures belonging to the plurality of domains. The data set of the atomic structures and the physical property values ​​used by the inference unit and the evaluation unit for inference and evaluation is a data set that was not used in training the trained model.

[0007] The trained model evaluated by this evaluation device may be provided as a model for performing inference in an inference device.

[0008] The evaluation device may be provided in the training device, and the training device may perform model training while evaluating the accuracy and generalization performance of the model to be trained through evaluation by the evaluation device. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a block diagram schematically illustrating an evaluation device according to an embodiment. [Figure 2] 10 is a flowchart showing processing of an evaluation device according to an embodiment. [Figure 3] FIG. 10 is a diagram showing the relationship between domains and calculation conditions according to an embodiment. [Figure 4] FIG. 1 is a block diagram schematically illustrating an inference device according to an embodiment. [Figure 5] FIG. 1 is a block diagram illustrating a training device according to an embodiment. [Figure 6] FIG. 1 is a block diagram illustrating an example of a hardware implementation according to an embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0010] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of the present invention will be described with reference to the drawings. The drawings and the description of the embodiments are given by way of example only and are not intended to limit the present invention.

[0011] (First embodiment) FIG. 1 is a block diagram schematically illustrating an evaluation device according to an embodiment. The evaluation device 1 includes an input unit 100, a storage unit 102, an inference unit 104, an evaluation unit 106, and an output unit 108. The evaluation device 1 is a device that evaluates the generalization performance and accuracy of a trained model NN. In addition to the components shown in FIG. 1, components necessary for the operation of the evaluation device 1 are also provided as appropriate, although not shown.

[0012] The input unit 100 is an interface that accepts data input in the evaluation device 1. The evaluation device 1 accepts input of data required for evaluation via the input unit 100. The evaluation device 1 may also accept input of data related to a trained model NN to be evaluated, such as data related to hyperparameters and parameters, via the input unit 100.

[0013] The storage unit 102 stores data necessary for evaluation in the evaluation device 1. The storage unit 102 stores, for example, information input from the input unit 100, a program for operating the evaluation device 1, evaluation results, as well as information such as intermediate results required during calculation and parameters related to the trained model NN.

[0014] The inference unit 104 inputs data relating to the atomic structure input from the input unit 100 into the trained model NN, performs forward propagation, and outputs an inference result. This inference result is a physical property value of the input atomic structure. As an example, this physical property value is a quantity including energy (potential). The inference unit 104 outputs the inference result output from the trained model NN to the memory unit 102 or the evaluation unit 106.

[0015] A trained model NN is a neural network model that receives an atomic structure as input and outputs a physical property value. This trained model NN is, for example, a neural network model used in NNP, and is a model that infers energy from atomic structure. The trained model NN is a model trained by an appropriate machine learning method using the atomic structure and the energy value associated with the atomic structure.

[0016] The training of the trained model NN is performed using an atomic structure and physical property values ​​obtained for the atomic structure by first-principles calculation as a training data set, but this does not preclude the use of other first-principles methods. This first-principles calculation may be, for example, a calculation based on DFT (Density Function Theory), in which case, as described above, the physical property value may be energy. Therefore, when an atomic structure is input, the trained model NN is formed as a model that infers the calculation results of first-principles calculation (or quantum chemistry calculation), DFT, etc.

[0017] The inference unit 104 performs inference using a validation dataset, which is a dataset other than the datasets used in the training (the training dataset and the dataset for cross-validation check in the training). The validation dataset is obtained, for example, by performing DFT calculations on the atomic structure. As another example, data from a known database or the like that has not been used in the training can also be used. For example, data from a public database such as PubChem, Material Project, or ICSD, or a complex structure database, may be obtained and used as the validation dataset.

[0018] The evaluation unit 106 compares the result of inference by the inference unit 104 with the physical property values ​​in the validation dataset to evaluate the trained model NN. Specifically, the evaluation is performed using the inference result obtained by inputting a certain atomic structure to the inference unit 104 and the physical property values ​​associated with the atomic structure. The evaluation unit 106 compares the inference result with the physical property values, for example, calculates an absolute error, and evaluates the trained model NN.

[0019] The validation dataset includes multiple pieces of data belonging to multiple domains. Here, a domain refers to a chemical domain related to atomic structure. Examples of domains include, but are not limited to, crystalline structure, amorphous structure, diatomic molecule, molecular structure, surface structure, and adsorption structure.

[0020] To improve the accuracy of the evaluation of generalization performance, the energies of, for example, two atomic molecules, two adjacent molecules, macromolecules, adsorption structures, and amorphous structures generated by MD (Molecular Dynamics) simulations may be calculated using techniques such as DFT, and these may be incorporated as validation data sets.

[0021] When the domain has a crystal structure, data may be collected from the Material Project, or data may be collected by DFT calculations or the like.

[0022] When the domain has an amorphous structure, data may be collected by DFT calculations or the like after creating an amorphous state using NNP for the above crystal structure.

[0023] When the domain is a two-atom molecule, a molecular structure, or two adjacent molecules, the interatomic distance between the two atoms may be changed in various ways, and data may be collected by DFT calculations or the like.

[0024] When the domain is a surface structure, the surface structure may be generated using a function that cuts out the surface from the crystal structure, and data may be collected by DFT calculations or the like.

[0025] When the domain has an adsorption structure, a state in which molecules are adsorbed onto the surface structure may be created, and data may be collected by DFT calculations or the like.

[0026] In the above data collection, various appropriate parameter settings in appropriate calculation methods, software, etc., such as VASP (registered trademark), Gaussian (registered trademark), etc., can be used.

[0027] For example, when performing DFT calculations, functionals such as ωB3LYP, B3P86, ωB97X, ωB97X-D, APFD,HSE, TPSSh, MO6-2X, PBE, rPBE, revPBE, PBEsol, and PBE0 may be used, but other functionals may also be used. Using these functionals makes it possible to calculate the energy values ​​for the chemical structures listed above. It is also possible to add a Hubbard correction term, which is a function of the occupation matrix n or density matrix ρ, to these energy calculations. The basis functions used can also be included in the domain information.

[0028] The inference unit 104 obtains inference results from the atomic structure of a validation dataset that includes datasets belonging to these multiple domains, and the evaluation unit 106 evaluates the trained model NN by comparing the validation dataset with the inference results. Evaluation indices that can be used include energy, force, energy difference, adsorption energy, total interatomic distances and total atomic angles, or lattice constants. These indices may be used to comprehensively evaluate each mode. The inference results are compared with the validation dataset using these indices.

[0029] As a non-limiting example, the evaluation unit 106 may evaluate the mean absolute error (MAE) for each domain as an evaluation value. As other examples, the mean square error (MSE), the root mean square error (RMSE), and the coefficient of determination (R2) may be used.

[0030] The evaluation unit 106 evaluates the trained model NN based on the evaluation value calculated for each domain. For example, a threshold value for the evaluation value may be defined, and if the evaluation value for each domain is equal to or less than the threshold value (or equal to or more than the threshold value depending on the evaluation value), the trained model NN may be evaluated as having generalization ability.

[0031] The output unit 108 outputs the evaluation result of the evaluation unit 106. For example, if a certain trained model NN has generalization ability, the output unit 108 may output a result indicating that generalization ability is present. As another example, the output unit 108 may output the evaluation value calculated by the evaluation unit 106.

[0032] FIG. 2 is a flowchart showing the processing of the evaluation device 1 according to an embodiment.

[0033] First, the evaluation device 1 receives data related to a trained model NN to be evaluated and a validation data set via the input unit 100 (S100).

[0034] The inference unit 104 acquires multiple atomic structures belonging to a certain domain from a validation dataset and acquires an inferred value for each acquired data set based on a trained model NN (S102). If the trained model is a model based on NNP, the inference unit 104 acquires an inferred value of energy. To improve accuracy, the inferred value of force may also be acquired.

[0035] The evaluation unit 106 compares the inferred value for the atomic structure with the physical property value in the validation data set for the atomic structure to obtain an evaluation value (S104). The evaluation unit 106 calculates, for example, the MAE between the inferred value and the evaluation value for multiple atomic structures belonging to the domain. The calculated MAE is used as the evaluation value for this domain of the trained model NN.

[0036] The evaluation unit 106 determines whether an evaluation value has been acquired for the domain to be evaluated (S106). If there is a domain to be evaluated but for which an evaluation value has not been acquired (S106: NO), the evaluation unit 106 executes acquisition of an evaluation value for that domain (S102 to S104).

[0037] If there is no domain for which an evaluation value has not been obtained among the domains to be judged (S106: YES), the evaluation unit 106 evaluates the trained model NN using the evaluation value obtained for each domain (S108).

[0038] The evaluation unit 106 outputs the evaluation result via the output unit 108, and the evaluation device 1 ends the process (S110). By checking this output result, it becomes possible to evaluate the accuracy and generalization performance of the trained model NN.

[0039] In the above process, the processes of S102 to S104 can be processed in parallel. For example, for multiple validation data sets belonging to the same domain, inference values ​​can be obtained in parallel, followed by evaluation values. Furthermore, calculations in different domains can be executed in parallel. For example, an accelerator such as a GPU (Graphics Processing Unit) can be used for this execution.

[0040] When energy is used as the evaluation value, the evaluation unit 106 may evaluate the trained model NN based on whether the MAE in each domain is less than a predetermined threshold, for example, 0.05 eV. The evaluation of the trained model NN may be output as passing if the MAE in all domains being evaluated is less than a predetermined threshold. As another example, a generalization performance score may be calculated based on the MAE in each domain and output. As yet another example, the MAE of each domain may be output as the evaluation value.

[0041] In the case of energy, the above example is set to less than 0.05 eV as a non-limiting example. This value is an indicator of the degree of accuracy required for the error from the DFT calculation in order to calculate the ease of reaction progress (activation energy). Preferably, the predetermined threshold is 0.03 eV, and more preferably 0.02 eV. For example, if the error is 0.05 eV or more, it becomes difficult to accurately estimate the superiority or inferiority of the chemical reaction and various physical properties. Therefore, the predetermined threshold may be set to 0.05 eV.

[0042] Although the MAE is used above, the variance or standard deviation of the MAE may be calculated and used as the evaluation value. For example, based on this standard deviation, if the MAE value for each domain is less than 3σ, it may be judged as passing. Preferably, it may be 2.5σ, and more preferably, 2σ. Furthermore, the standard deviation itself may be compared with a predetermined threshold value.

[0043] The evaluation unit 106 can also use interatomic distances as the evaluation value. This evaluation value can be used, for example, when the domain can define the lattice constants of a molecular structure, a surface structure, an adsorption structure, or the like. In this case, the trained model NN is a model that infers all interatomic distances when the structure is optimized by, for example, DFT calculations. The inference unit 104 infers all interatomic distances using this trained model NN, and the evaluation unit 106 obtains all interatomic distances from a validation dataset, and determines the difference between the inferred value and this all interatomic distance as an error, and calculates the MAE from this error.

[0044] The evaluation unit 106 can also use the lattice constant as the evaluation value. This evaluation value can be used, for example, when the domain includes a crystalline structure. In this case, the trained model NN outputs, as an inferred value, a lattice constant obtained by optimizing the lattice structure using, for example, DFT calculations. The evaluation unit 106 obtains the lattice constant from the validation dataset, compares it with the result inferred by the inference unit 104, and calculates the MAE by taking the difference as an error.

[0045] In these cases, for example, whether the MAE value is within a predetermined distance, whether the MAE is within 3σ, or the like may be used as an evaluation.

[0046] The trained model NN may be a neural network model that can input calculation conditions determined for each domain along with the atomic structure. This trained model NN may be a model that infers energy, etc., when a DFT calculation is performed under the calculation conditions based on the calculation conditions input along with the atomic structure.

[0047] Figure 3 shows an example of a combination of DFT calculation conditions that can be input to a trained model NN. The domain corresponds to the above domain, and the calculation conditions include, for example, the exchange-correlation functional to be used.

[0048] The domains, from top to bottom, represent the structures of molecule, molecule, crystal, crystal, amorphous, amorphous, surface, surface, two atoms, adsorption, and adsorption. The description of calculation conditions defines functionals, basis functions, etc.

[0049] If the trained model NN can input such conditions, the evaluation device 1 can obtain a data set that meets each calculation condition from the validation data set in the evaluation of each domain, and perform inference by the inference unit 104 and evaluation by the evaluation unit 106 within this data set.

[0050] As described above, according to this embodiment, the accuracy of the trained model NN can be judged by the MAE itself, and the generalization performance can be evaluated between domains by using the evaluation value using this MAE.

[0051] (Second embodiment) In the first embodiment, the MAE is calculated for each domain, but the calculation of the MAE can be further subdivided. For example, the MAE may be calculated for each atomic structure that has a predetermined element among atomic structures that belong to a domain.

[0052] Specifically, a domain is first specified. In this domain, a dataset is extracted from the validation dataset for each element, and the MAE is calculated by inference using this dataset.

[0053] For example, an atomic structure containing a hydrogen atom in a domain is extracted from the validation data set, and the inference unit 104 infers an energy value. The evaluation unit 106 calculates the MAE of the atomic structure containing a hydrogen atom in this domain using the difference between this inferred value and the corresponding energy value in the validation data set. Similarly, the MAE of helium atoms, lithium atoms, etc. is calculated. The MAE may be calculated for all elements included in the validation data set, or a target element may be determined in advance and the MAE for the target element may be calculated.

[0054] The evaluation unit 106 performs a domain evaluation by statistically processing the multiple MAEs calculated for each domain. For example, as described above, the evaluation unit 106 may perform processing using variance or standard deviation, or may simply perform evaluation by comparing the average value of the MAEs with a predetermined threshold. As another example, each of the acquired MAEs may be compared with a predetermined threshold, such as 0.02 eV.

[0055] In this comparison, the smaller the granularity, the more accurate the comparison is for each atomic structure, and by further using the evaluation value for each domain, the generalization performance can also be evaluated. Therefore, according to this embodiment, it is possible to evaluate both individual accuracy and generalization performance.

[0056] (Third embodiment) Of course, the technical scope of the present disclosure also extends to an inference device 2 that performs inference using a trained model NN evaluated by the evaluation device 1 described above.

[0057] 4 is a block diagram schematically illustrating an inference device 2 according to one embodiment. The inference device 2 includes an input unit 200, a memory unit 202, an inference unit 204, and an output unit 206. The inference device 2 is, for example, a device that infers the energy of an arbitrary atomic structure in multiple domains using NNP. The overall configuration is similar to that of the evaluation device 1, and therefore detailed explanations are omitted.

[0058] The input unit 200 has an interface for inputting the atomic structure to be inferred. The memory unit 202 stores data required for the operation of the inference device 2. The inference unit 204 infers energy from the atomic structure using a trained model NN. The output unit 206 outputs the results of the inference by the inference unit 204.

[0059] Here, the trained model NN may be a model that has been evaluated as having high accuracy and generalization performance in the evaluations in the above-described embodiments, for example, a model that has passed a test in the evaluation device 1.

[0060] Such an inference device 2 can obtain appropriate energy from atomic structures because the accuracy and generalization performance of the trained model NN are guaranteed to a certain level.

[0061] 3, when a trained model NN is formed as a model to which calculation conditions can be input, the inference device 2 also acquires appropriate calculation conditions via the input unit 200. The inference unit 204 inputs the calculation conditions along with the atomic structure into the trained model NN, enabling the inference device 2 to output highly accurate energy inference results.

[0062] (Fourth embodiment) The evaluation device 1 described above can also be incorporated into a training device.

[0063] 5 is a block diagram showing a schematic diagram of a training device according to one embodiment. The training device 3 includes an input unit 300, a storage unit 302, an optimization unit 304, an evaluation device 1, and an output unit 306. As in the third embodiment, the overall configuration is similar to that of the evaluation device 1, and therefore detailed explanations are omitted.

[0064] The input unit 300 has an interface for inputting data necessary for training, etc. The storage unit 302 stores data necessary for the operation of the training device 3.

[0065] The optimizer 304 trains the model NN1 based on any suitable machine learning method, where the optimizer 304 performs training of the model NN1 using a training dataset that is different from the validation dataset.

[0066] The evaluation device 1 evaluates the model NN1 trained by the optimization unit 304 using the evaluation method described in the above-described embodiment. That is, the evaluation device 1 evaluates the accuracy and generalization performance of the model NN1. Based on the evaluation result, it is determined whether the optimization of the model NN1 in the training device 3 has been completed, and if the optimization is insufficient, the optimization unit 304 repeats the training of the model NN1 to optimize the model NN1.

[0067] This training may be repeated until the evaluation in the evaluation device 1 is successful.

[0068] The output unit 306 outputs the parameters of the model NN1 optimized by training to the outside or the storage unit 302, and the process ends.

[0069] In this way, the evaluation device 1 may be configured to be incorporated into the training device 3. By incorporating the evaluation device 1 into the training device 3, it becomes possible to train a model that achieves predetermined accuracy and predetermined generalization performance in training.

[0070] All of the above trained models may be concepts that include, for example, models that have been trained as described and then further distilled using a general method.

[0071] Some or all of the devices (evaluation device 1, inference device 2, or training device 3) in the above-described embodiments may be configured as hardware, or may be configured as software (program) information processing executed by a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), or the like. When configured as software information processing, software that realizes at least some of the functions of each device in the above-described embodiments may be stored on a non-transitory storage medium (non-transitory computer-readable medium) such as a flexible disk, a CD-ROM (Compact Disc-Read Only Memory), or a USB (Universal Serial Bus) memory, and the software information processing may be executed by loading the software into a computer. Alternatively, the software may be downloaded via a communication network. Furthermore, the software may be implemented in a circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array), thereby executing the information processing by hardware.

[0072] The type of storage medium that stores the software is not limited. The storage medium is not limited to removable media such as magnetic disks or optical disks, but may be fixed storage media such as hard disks or memory. The storage medium may be provided inside the computer or outside the computer.

[0073] 6 is a block diagram showing an example of the hardware configuration of each device (evaluation device 1, inference device 2, or training device 3) in the above-described embodiment. Each device may be realized as a computer 7 including, for example, a processor 71, a main storage device 72 (memory), an auxiliary storage device 73 (memory), a network interface 74, and a device interface 75, which are connected via a bus 76.

[0074] Although the computer 7 in FIG. 6 includes one of each component, it may include multiple of the same component. Also, while FIG. 6 shows one computer 7, the software may be installed on multiple computers, and each of the multiple computers may execute the same or different parts of the software. In this case, a distributed computing configuration may be used in which each computer communicates with the other computers via a network interface 74 or the like to execute the processing. That is, each device in the above-described embodiment (evaluation device 1, inference device 2, or training device 3) may be configured as a system in which one or more computers execute instructions stored in one or more storage devices to realize its functions. Furthermore, the system may be configured such that information transmitted from a terminal is processed by one or more computers provided on a cloud, and the processing results are transmitted to the terminal.

[0075] The various calculations of each device (evaluation device 1, inference device 2, or training device 3) in the above-described embodiments may be executed in parallel using one or more processors, or using multiple computers via a network. Furthermore, the various calculations may be distributed to multiple processing cores within a processor and executed in parallel. Furthermore, some or all of the processes, means, etc. of the present disclosure may be executed by at least one of a processor and a storage device provided on a cloud that can communicate with computer 7 via a network. Thus, each device in the above-described embodiments may be implemented in the form of parallel computing using one or more computers.

[0076] The processor 71 may be an electronic circuit (processing circuit, processing circuitry, CPU, GPU, FPGA, ASIC, etc.) including a computer control device and arithmetic device. The processor 71 may also be a semiconductor device including a dedicated processing circuit. The processor 71 is not limited to an electronic circuit using electronic logic elements, but may also be realized by an optical circuit using optical logic elements. The processor 71 may also include an arithmetic function based on quantum computing.

[0077] The processor 71 performs arithmetic processing based on data and software (programs) input from each device, etc. configured inside the computer 7, and can output the arithmetic results and control signals to each device, etc. The processor 71 may control each component constituting the computer 7 by executing the OS (Operating System) of the computer 7, applications, etc.

[0078] Each device (evaluation device 1, inference device 2, or training device 3) in the above-described embodiments may be realized by one or more processors 71. Here, the processor 71 may refer to one or more electronic circuits arranged on one chip, or may refer to one or more electronic circuits arranged on two or more chips or two or more devices. When multiple electronic circuits are used, the electronic circuits may communicate with each other via wire or wirelessly.

[0079] The main memory device 72 is a memory device that stores instructions executed by the processor 71 and various data, and information stored in the main memory device 72 is read by the processor 71. The auxiliary memory device 73 is a memory device other than the main memory device 72. Note that these memory devices refer to any electronic component capable of storing electronic information, and may be semiconductor memory. The semiconductor memory may be either volatile memory or non-volatile memory. The memory device for saving various data in each device (evaluation device 1, inference device 2, or training device 3) in the above-described embodiments may be realized by the main memory device 72 or the auxiliary memory device 73, or may be realized by an internal memory built into the processor 71. For example, the memory units 102, 202, and 302 in the above-described embodiments may be realized by the main memory device 72 or the auxiliary memory device 73.

[0080] Multiple processors may be connected (coupled) to one storage device (memory), or a single processor may be connected. Multiple storage devices (memories) may be connected (coupled) to one processor. When each device (evaluation device 1, inference device 2, or training device 3) in the above-described embodiments is configured with at least one storage device (memory) and multiple processors connected (coupled) to this at least one storage device (memory), it may include a configuration in which at least one of the multiple processors is connected (coupled) to at least one storage device (memory). This configuration may also be realized by storage devices (memories) and processors included in multiple computers. Furthermore, it may include a configuration in which the storage device (memory) is integrated with the processor (for example, a cache memory including an L1 cache and an L2 cache).

[0081] The network interface 74 is an interface for connecting to the communication network 8 wirelessly or via a wire. The network interface 74 may be an appropriate interface, such as one that conforms to an existing communication standard. Information may be exchanged with an external device 9A connected via the communication network 8 via the network interface 74. The communication network 8 may be any one of a WAN (Wide Area Network), a LAN (Local Area Network), a PAN (Personal Area Network), etc., or a combination thereof, as long as information is exchanged between the computer 7 and the external device 9A. An example of a WAN is the Internet, an example of a LAN is IEEE802.11 or Ethernet (registered trademark), and an example of a PAN is Bluetooth (registered trademark) or NFC (Near Field Communication), etc.

[0082] The device interface 75 is an interface such as a USB that directly connects to the external device 9B.

[0083] The external device 9A is a device connected to the computer 7 via a network. The external device 9B is a device connected directly to the computer 7.

[0084] For example, the external device 9A or the external device 9B may be an input device. The input device may be a device such as a camera, a microphone, a motion capture device, various sensors, a keyboard, a mouse, or a touch panel, and provides acquired information to the computer 7. Alternatively, the external device 9A or the external device 9B may be a device equipped with an input unit, a memory, and a processor, such as a personal computer, a tablet terminal, or a smartphone.

[0085] Furthermore, the external device 9A or the external device 9B may be, for example, an output device. The output device may be, for example, a display device such as an LCD (Liquid Crystal Display), a CRT (Cathode Ray Tube), a PDP (Plasma Display Panel), or an organic EL (Electro Luminescence) panel, or may be a speaker that outputs sound or the like. Alternatively, the output device may be a device including an output unit, a memory, and a processor, such as a personal computer, a tablet terminal, or a smartphone.

[0086] Furthermore, the external device 9A or the external device 9B may be a storage device (memory). For example, the external device 9A may be a network storage or the like, and the external device 9B may be a storage such as an HDD.

[0087] Furthermore, external device 9A or external device 9B may be a device having some of the functions of the components of each device (evaluation device 1, inference device 2, or training device 3) in the above-described embodiments. In other words, computer 7 may transmit or receive some or all of the processing results of external device 9A or external device 9B.

[0088] In this specification (including the claims), when the expression "at least one of a, b, and c" or "at least one of a, b, or c" (including similar expressions) is used, it includes any of a, b, c, ab, ac, bc, or abc. It may also include multiple instances of any element, such as aa, abb, aabbcc, etc. Furthermore, it also includes the addition of elements other than the enumerated elements (a, b, and c), such as having d, as in abcd.

[0089] In this specification (including the claims), when expressions such as "using data as input / based on / according to / in response to" (including similar expressions) are used, unless otherwise specified, this includes cases where various data itself is used as input, or where various data that has been processed in some way (e.g., noise-added, normalized, intermediate representation of various data, etc.) is used as input. Furthermore, when a statement is made that a result is obtained "based on / according to / in response to data," this includes cases where the result is obtained based solely on the data in question, as well as cases where the result is obtained as a result of being influenced by other data, factors, conditions, and / or states other than the data in question. Furthermore, when a statement is made that "data is output," this includes cases where various data itself is used as output, or where various data that has been processed in some way (e.g., noise-added, normalized, intermediate representation of various data, etc.) is output, unless otherwise specified.

[0090] When the terms "connected" and "coupled" are used in this specification (including the claims), they are intended as open-ended terms that encompass any of direct connection / coupling, indirect connection / coupling, electrically connection / coupling, communicatively connection / coupling, functionally connection / coupling, and physically connection / coupling. These terms should be interpreted appropriately according to the context in which they are used, but any form of connection / coupling that is not intentionally or naturally excluded should be interpreted as being included in these terms without limitation.

[0091] In this specification (including the claims), the expression "A configured to B" may include the physical structure of element A having a configuration capable of performing operation B, and the permanent or temporary setting / configuration of element A being configured / set to actually perform operation B. For example, if element A is a general-purpose processor, it is sufficient that the processor has a hardware configuration capable of performing operation B, and is configured to actually perform operation B by setting a permanent or temporary program (instruction). Also, if element A is a dedicated processor or dedicated arithmetic circuit, it is sufficient that the circuit structure of the processor is implemented to actually perform operation B, regardless of whether control instructions and data are actually attached.

[0092] When used in this specification (including the claims), terms implying containing or possessing (e.g., "comprising / including" and "having") are intended to be open-ended terms that include containing or possessing things other than the object designated by the object of the term. When the object of such a term implies no quantity or a singular number (e.g., an article such as "a" or "an"), the expression should be construed as not being limited to a specific number.

[0093] In this specification (including the claims), even if expressions such as "one or more" or "at least one" are used in some places and expressions that do not specify a quantity or that imply a singular number (expressions using the articles "a" or "an") are used in other places, the latter expressions are not intended to mean "one." In general, expressions that do not specify a quantity or that imply a singular number (expressions using the articles "a" or "an") should be interpreted as not necessarily being limited to a specific number.

[0094] In this specification, when a particular advantage / result is described as being obtained from a particular configuration of an embodiment, it should be understood that the same advantage / result can also be obtained from one or more other embodiments having the same configuration, unless otherwise stated. However, it should be understood that the presence or absence of the effect generally depends on various factors, conditions, and / or states, etc., and that the effect is not necessarily obtained by the configuration. The effect is merely obtained by the configuration described in the embodiment when various factors, conditions, and / or states, etc. are satisfied, and the effect does not necessarily occur in a claimed invention that defines the same or a similar configuration.

[0095] When used in this specification (including the claims), terms such as "maximize" include finding a global maximum, finding an approximation of a global maximum, finding a local maximum, and finding an approximation of a local maximum, and should be interpreted appropriately according to the context in which the term is used. It also includes finding approximations of these maxima probabilistically or heuristically. Similarly, when used in this specification (including the claims), terms such as "minimize" include finding a global minimum, finding an approximation of a global minimum, finding a local minimum, and finding an approximation of a local minimum, and should be interpreted appropriately according to the context in which the term is used. It also includes finding approximations of these minima probabilistically or heuristically. Similarly, when used in this specification (including the claims), terms such as "optimize" include finding a global optimum, finding an approximation of a global optimum, finding a local optimum, and finding an approximation of a local optimum, and should be interpreted appropriately according to the context in which the term is used. It also includes finding approximations of these optima probabilistically or heuristically.

[0096] In this specification (including claims), when multiple pieces of hardware perform a predetermined process, the pieces of hardware may cooperate to perform the predetermined process, or some of the hardware may perform all of the predetermined process. Furthermore, some of the hardware may perform part of the predetermined process, and other hardware may perform the rest of the predetermined process. In this specification (including claims), when an expression such as "one or more pieces of hardware perform a first process, and the one or more pieces of hardware perform a second process" is used, the hardware performing the first process and the hardware performing the second process may be the same or different. In other words, it is sufficient that the hardware performing the first process and the hardware performing the second process are included in the one or more pieces of hardware. Note that the hardware may include an electronic circuit, a device including an electronic circuit, or the like.

[0097] Although the embodiments of the present disclosure have been described in detail above, the present disclosure is not limited to the individual embodiments described above. Various additions, modifications, substitutions, partial deletions, etc. are possible within the scope of the conceptual idea and spirit of the present invention derived from the content defined in the claims and their equivalents. For example, in all of the above-described embodiments, when numerical values ​​or formulas are used in the explanation, they are shown as examples and are not limited to these. Furthermore, the order of each operation in the embodiments is shown as an example and is not limited to these. [Explanation of symbols]

[0098] 1: Evaluation device, 100: Input section, 102: Memory section, 104: Reasoning Department, 106: Evaluation section, 108: Output section, 2: Reasoning device, 200: Input section, 202: Memory section, 204: Reasoning Department; 206: Output section, 3: training equipment, 300: Input section, 302: Memory section, 304: Optimization section, 306: Output section

Claims

1. one or more processors; The processor: inputting each of a plurality of atomic structures, each of which belongs to one of a plurality of domains representing chemical regions related to atomic structures, into a trained model that is a neural network to obtain a first physical property value for each of the plurality of atomic structures; evaluating the trained model based on the second physical property value of each of the plurality of atomic structures and the first physical property value obtained from the trained model; Evaluation equipment.

2. the processor evaluates the trained model for each of the plurality of atomic structures containing a predetermined element. The evaluation device according to claim 1.

3. the second physical property value of each of the plurality of atomic structures is obtained by first-principles calculation; 3. The evaluation device according to claim 1 or 2.

4. The trained model is a neural network model used in NNP (Neural Network Potential), The evaluation device according to any one of claims 1 to 3.

5. The plurality of domains include at least one of a crystalline structure, an amorphous structure, a diatomic molecule, a molecular structure, a surface structure, or an adsorbed structure.

5. The evaluation device according to claim 1.

6. The processor calculates an evaluation value for each of the plurality of domains and performs the evaluation.

6. The evaluation device according to claim 1.

7. the processor calculates, as an evaluation value, at least one of a mean absolute error, a mean square error, a root mean square error, or a coefficient of determination between the second physical property value of each of the plurality of atomic structures and the first physical property value acquired from the trained model, and evaluates the trained model based on the evaluation value.

7. The evaluation device according to claim 1.

8. the processor calculates the mean absolute error for each domain as the evaluation value and performs the evaluation. The evaluation device according to claim 7.

9. the processor calculates the variance or standard deviation of the mean absolute error for each domain as the evaluation value and performs the evaluation. The evaluation device according to claim 7.

10. the processor makes the evaluation based on whether the evaluation value is less than a predetermined threshold. The evaluation device according to any one of claims 6 to 9.

11. The first physical property value obtained from the trained model includes at least one of energy, force, energy difference, adsorption energy, total interatomic distance, total atomic angle, or lattice constant; The evaluation device according to any one of claims 1 to 10.

12. 12. The evaluation device according to claim 1, wherein the data set of the atomic structure and the first physical property value is a data set that has not been used to train the trained model.

13. The evaluation is performed using the evaluation device according to any one of claims 1 to 12, and a physical property value is inferred from the atomic structure using a trained model in which a predetermined evaluation value is obtained by one or more processors. Reasoning device.

14. training the model by one or more processors based on evaluation of the model using the evaluation device according to any one of claims 1 to 12; training equipment.

15. The computer inputting each of a plurality of atomic structures, each of which belongs to one of a plurality of domains representing chemical regions related to atomic structures, into a trained model that is a neural network to obtain a first physical property value for each of the plurality of atomic structures; evaluating the trained model based on the second physical property value of each of the plurality of atomic structures and the first physical property value obtained from the trained model; Evaluation method.

16. On the computer, inputting each of a plurality of atomic structures, each of which belongs to one of a plurality of domains representing chemical regions related to the atomic structures, into a trained model that is a neural network, to obtain a first physical property value for each of the plurality of atomic structures; evaluating the trained model based on the second physical property value of each of the plurality of atomic structures and the first physical property value obtained from the trained model; A program that makes it happen.

17. On the computer, inputting each of a plurality of atomic structures, each of which belongs to one of a plurality of domains representing chemical regions related to the atomic structures, into a trained model that is a neural network, to obtain a first physical property value for each of the plurality of atomic structures; evaluating the trained model based on the second physical property value of each of the plurality of atomic structures and the first physical property value obtained from the trained model; A non-transitory computer-readable medium that stores a program that causes the computer to perform the following:

Images