Information processing program, information processing method, and information processing device

Abstract

To make it easier to train the model efficiently. [Solution] The information processing device 100 generates a first noise for each sample data 130, corresponding to the coordinates of each dimension of each atom in the system represented by the sample data 130. The information processing device 100 calculates a second noise for each sample data 130 by removing the average of the first noise from the first noise generated for each sample data 130, corresponding to the coordinates of each dimension of each atom in the system represented by the sample data 130. The information processing device 100 performs self-supervised learning on the model 110 based on training data that associates each sample data 130 after the addition of the second noise with the second noise corresponding to the sample data 130. This allows the information processing device 100 to efficiently train the model 110, which is used, for example, when performing first-principles calculations or molecular dynamics in fields such as drug discovery or materials development.

Description

[Technical Field]

[0001] The present invention relates to an information processing program, an information processing method, and an information processing apparatus. [Background technology]

[0002] Traditionally, there are techniques for training models that output labels corresponding to input data. These models are, for example, neural networks called machine learning interatomic potentials, used in fields such as first-principles calculations or molecular dynamics. The data, for example, represents the coordinates of each atom in a system of multiple atoms, used in fields such as first-principles calculations or molecular dynamics. For instance, there is a technique called "self-supervised learning" that automatically generates correct labels for data samples and trains the model based on training data that associates the data samples with these automatically generated correct labels. Specifically, one could generate random noise to serve as the correct labels, and then train the model based on training data that associates data samples with this noise with the corresponding correct labels.

[0003] Prior art, for example, involves obtaining a third feature relating to one or more predetermined types of energy of a molecule based on a graph containing a first feature relating to the molecule and a second feature relating to the energy assigned to the atoms that make up the molecule. [Prior art documents] [Patent Documents]

[0004] [Patent Document 1] Japanese Patent Publication No. 2024-072890 [Overview of the project] [Problems that the invention aims to solve]

[0005] However, conventional techniques can sometimes make it difficult to efficiently train models. For example, consider training a machine learning interatomic potential that takes into account the relative positions of atoms. In this case, if there are multiple training datasets with the same relative positions of atoms but different label content, it becomes impossible to efficiently train the machine learning interatomic potential.

[0006] In one aspect, the present invention aims to make it easier to efficiently train models. [Means for solving the problem]

[0007] According to one embodiment, an information processing program, information processing method, and information processing device are proposed to perform self-supervised learning on a model that generates a first noise for each of a plurality of sample data for each of the coordinates of each dimension of each atom in the system represented by the sample data, calculates the average of the first noise generated for each dimension of each of the sample data for each of the dimensions of each of the sample data for each of the dimensions of each of the sample data for each of the dimensions of each of the sample data for each of the sample data for each of the dimensions of each of the sample data for each of the sample data for each of the dimensions of each of the sample data for each of the sample data for each of the dimensions of each of the sample data for each of the sample data for each of the dimensions of each of the sample data for each of the sample data for each of the sample data for each of the dimensions of each of the sample data for each of the sample data for each of the sample data for each of the sample data for each of the input data representing the coordinates of each dimension of each of the atoms in the system. [Effects of the Invention]

[0008] According to one embodiment, it becomes possible to train the model more efficiently. [Brief explanation of the drawing]

[0009] [Figure 1] Figure 1 is an explanatory diagram showing one embodiment of the information processing method according to the embodiment. [Figure 2] FIG. 2 is an explanatory diagram showing an example of the information processing system 200. [Figure 3] FIG. 3 is a block diagram showing an example of the hardware configuration of the information processing apparatus 100. [Figure 4] FIG. 4 is a block diagram showing an example of the functional configuration of the information processing apparatus 100. [Figure 5] FIG. 5 is an explanatory diagram showing the flow of training the model 500. [Figure 6] FIG. 6 is an explanatory diagram (part 1) showing an example of the MLIAP 620. [Figure 7] FIG. 7 is an explanatory diagram (part 2) showing an example of the MLIAP 620. [Figure 8] FIG. 8 is an explanatory diagram showing an example of performing self-supervised learning. [Figure 9] FIG. 9 is an explanatory diagram (part 1) showing an example of an effect. [Figure 10] FIG. 10 is an explanatory diagram (part 2) showing an example of an effect. [Figure 11] FIG. 11 is an explanatory diagram (part 3) showing an example of an effect. [Figure 12] FIG. 12 is a flowchart showing an example of the overall processing procedure. [Figure 13] FIG. 13 is a flowchart showing an example of an individual processing procedure.

MODE FOR CARRYING OUT THE INVENTION

[0010] Hereinafter, embodiments of an information processing program, an information processing method, and an information processing apparatus according to the present invention will be described in detail with reference to the drawings.

[0011] (An example of an information processing method according to an embodiment) FIG. 1 is an explanatory diagram showing an example of an information processing method according to an embodiment. The information processing apparatus 100 is a computer for facilitating training of a model when performing self-supervised learning. The information processing apparatus 100 is, for example, a server or a PC (Personal Computer).

[0012] Self-supervised learning is one of the techniques called machine learning that trains a model to output labels corresponding to the input data. The model is a computational formula including parameters. The model is, for example, a neural network. Specifically, the model is a machine learning interatomic potential in fields such as first-principles calculations or molecular dynamics. In the following description, the machine learning interatomic potential may be denoted as "MLIAP (Machine Learning InterAtomic Potential)".

[0013] The label represents, for example, the features of the data. The label may represent, for example, the result of classifying the data. Specifically, when the model to be trained is MLIAP, the data represents a system containing atoms. And the label represents the energy of the system and the forces acting on the atoms. The system is, for example, the structure of a molecule. The force is, for example, a three-dimensional vector. Therefore, MLIAP, for example, when receiving an input of data representing a system containing atoms, outputs a label representing the energy of the system and the forces acting on the atoms corresponding to the data.

[0014] Specifically, MLIAP has an encoder that calculates the feature quantity of the system, an energy head that calculates the energy of the system from the feature quantity of the system, and a force head that calculates the forces acting on the atoms from the feature quantity of the system. Specifically, MLIAP calculates the feature quantity of the system by the encoder based on the element for each atom and the information around the atom. The element is the type of atom. The information around the atom is, for example, a graph representing the relative positional relationship between atoms. Specifically, MLIAP calculates the energy of the system based on the feature quantity of the system by the energy head. Specifically, MLIAP calculates the forces acting on the atoms based on the feature quantity of the system by the force head.

[0015] Training involves updating the model's parameters, for example. Training is also called learning. Training utilizes techniques such as mathematical optimization. Specifically, training employs methods like backpropagation and stochastic gradient descent. The process by which the model generates labels corresponding to the data is also called inference.

[0016] Self-supervised learning is applied in situations where ground truth labels corresponding to data samples are not provided in advance. A sample is also called a sample. Self-supervised learning, for example, automatically generates ground truth labels for data samples. Self-supervised learning trains a model based on a dataset of training data that associates data samples with the automatically generated ground truth labels. Because self-supervised learning does not require ground truth labels to be prepared in advance, it can reduce the time and cost associated with preparing labels compared to supervised learning. In the following explanation, self-supervised learning may be referred to as "SSL (Self-Supervised Learning)".

[0017] In self-supervised learning, specifically when the model being trained is MLIAP, random noise can be used as the ground truth for the labels. Specifically, self-supervised learning generates random noise for each atom, and then generates training data that includes a sample of data with the generated random noise added together, and the ground truth for the labels representing the generated random noise. Self-supervised learning trains the model based on a dataset of the generated training data.

[0018] Machine learning techniques, such as self-supervised learning, are used in methods like pre-training and fine-tuning.

[0019] Pre-training is a method of training a model, for example, based on a relatively large dataset. Specifically, pre-training involves training a model based on a dataset of training data that associates sample documents written in an arbitrary language with the corresponding labels, and then simulating data of documents written in various languages.

[0020] Fine-tuning is a technique for further training a model that has already been pre-trained to suit a specific task. A task is a problem of generating labels corresponding to samples, based on those samples. Fine-tuning can be done, for example, by training a model on a relatively small dataset related to a specific task.

[0021] Fine-tuning specifically involves training a model to fit a dataset of Japanese-language documents, based on training data that associates samples of documents written in Japanese with the corresponding labels.

[0022] The combination of pre-training and fine-tuning is expected to reduce the time and cost associated with improving the inference accuracy of a model. Specifically, pre-training preferably utilizes self-supervised learning to reduce the time and cost associated with preparing large datasets.

[0023] Fine-tuning, specifically, preferably utilizes supervised learning to make the model more adaptable to a particular task. Supervised learning is applicable when ground truth labels corresponding to data samples are provided in advance. For example, supervised learning trains a model based on a dataset containing training data that associates data samples with pre-prepared ground truth labels. In the following explanation, supervised learning may be referred to as "SL (Supervised Learning)".

[0024] However, conventional techniques sometimes make it difficult to implement self-supervised learning. For example, it is difficult to implement self-supervised learning for MLIAP. As mentioned above, MLIAP sometimes uses graphs that represent the relative positions of atoms. On the other hand, when adding random noise to data samples in self-supervised learning, it is possible to generate multiple inconsistent training data sets where the correct labels differ, even though the relative positions of atoms are the same. For this reason, it may not be possible to efficiently train MLIAP using self-supervised learning.

[0025] Therefore, this embodiment describes an information processing method that can facilitate efficient model training.

[0026] In Figure 1, the information processing device 100 stores model 110. Model 110 generates output data representing at least the forces acting on an atom, in response to input data representing a system including an atom. Model 110 is subject to training by self-supervised learning. Model 110 has parameters 111. Model 110 is, for example, MLIAP.

[0027] The input data represents, for example, the structure of a system containing atoms. The input data represents, for example, the coordinates of each atom in the system in each dimension. Specifically, the input data represents the positional relationships between atoms in a system containing atoms. The dimensions are, for example, the X-axis, Y-axis, and Z-axis. The output data may also represent the energy of the system. The output data is a label.

[0028] (1-1) The information processing device 100 acquires the dataset 120. The dataset 120 is a collection of sample data 130. The sample data 130 includes samples representing systems containing atoms. The sample data 130 does not include the correct labels representing the energy of the system and the forces acting on the atoms. In the example in Figure 1, the dataset 120 specifically includes sample data 131 to 133.

[0029] In the example in Figure 1, for the sake of simplicity, sample data 131-133 are assumed to relate to two atoms, 141 and 142, respectively. Atoms 141 and 142 are, for example, H atoms or O atoms. Specifically, sample data 131 represents the structure of system 150 containing atoms 141 and 142. Similarly, sample data 132 represents the structure of system 160 containing atoms 141 and 142. And specifically, sample data 133 represents the structure of system 170 containing atoms 141 and 142.

[0030] (1-2) The information processing device 100 generates a first noise for each sample data 130 contained in the acquired dataset 120, for each dimension of the coordinates of each atom in the system represented by the sample data 130. The first noise is, for example, a random number.

[0031] In the example in Figure 1, for the sake of simplicity, we assume that the first noise is 0 for the coordinates in the Y-axis direction and the coordinates in the Z-axis direction.

[0032] In the example in Figure 1, for sample data 131, the first noise for the X-axis coordinate of atom 141 in system 150 is specifically 0. Similarly, for sample data 131, the first noise for the X-axis coordinate of atom 142 in system 150 is specifically the value corresponding to vector 151.

[0033] Furthermore, in the example in Figure 1, with respect to sample data 132, the first noise for the X-axis coordinate of atom 141 in system 160 is specifically the value corresponding to vector 161. Also, with respect to sample data 132, the first noise for the X-axis coordinate of atom 142 in system 160 is specifically the value corresponding to vector 162.

[0034] Furthermore, in the example in Figure 1, for sample data 133, the first noise for the X-axis coordinate of atom 141 in system 170 is specifically the value corresponding to vector 171. Also, for sample data 133, the first noise for the X-axis coordinate of atom 142 in system 170 is specifically 0.

[0035] (1-3) For each sample data 130, the information processing device 100 calculates the average of the first noise generated for each dimension, with respect to the coordinates of each atom in the system represented by the sample data 130 in that dimension. The average is calculated, for example, for each sample data 130 and for each dimension.

[0036] In the example in Figure 1, for the sake of simplicity, the average of the first noise is 0 for both the Y-axis coordinate and the Z-axis coordinate.

[0037] In the example in Figure 1, for sample data 131, the average of the first noise for the X-axis coordinate in system 150 is, for example, half the sum of 0 and the value corresponding to vector 151, so specifically it is the value corresponding to half of vector 151. Also in the example in Figure 1, for sample data 132, the average of the first noise for the X-axis coordinate in system 160 is, for example, half the sum of the values ​​corresponding to vectors 161 and 162, so specifically it is the value 0, which is the result of vectors 161 and 162 canceling each other out. Also in the example in Figure 1, for sample data 133, the average of the first noise for the X-axis coordinate in system 170 is, for example, half the sum of the value corresponding to vector 171 and 0, so specifically it is the value corresponding to half of vector 171.

[0038] (1-4) For each sample data 130, the information processing device 100 calculates a second noise by removing the average calculated for each dimension from the first noise generated for each dimension coordinate of each atom in the system represented by the sample data 130.

[0039] In the example in Figure 1, for the sake of simplicity, the second noise is 0 for the coordinates in the Y-axis direction and the coordinates in the Z-axis direction.

[0040] In the example in Figure 1, for sample data 131, the second noise for the X-axis coordinate of atom 141 in system 150 corresponds to, for example, the value of vector 152. Also, for sample data 131, the second noise for the X-axis coordinate of atom 142 in system 150 corresponds to, for example, the value of vector 153.

[0041] Furthermore, in the example in Figure 1, for sample data 132, the second noise with respect to the X-axis coordinate of atom 141 in system 160 corresponds to, for example, the value of vector 163. Also, for sample data 132, the second noise with respect to the X-axis coordinate of atom 142 in system 160 corresponds to, for example, the value of vector 164.

[0042] Furthermore, in the example in Figure 1, for sample data 133, the second noise for the X-axis coordinate of atom 141 in system 170 corresponds to, for example, the value of vector 172. Also, for sample data 133, the second noise for the X-axis coordinate of atom 142 in system 170 corresponds to, for example, the value of vector 173.

[0043] (1-5) For each sample data 130, the information processing device 100 adds the second noise calculated for each dimension of the coordinates of each atom in the system represented by the sample data 130 to the coordinates of each dimension. In the example in Figure 1, the addition may change the coordinates of atoms 141 and 142 in the X-axis direction.

[0044] In the example shown in Figure 1, the addition causes the X-axis coordinate of atom 141 in system 150 to change in the direction of vector 152, for example, with respect to sample data 131. Furthermore, the addition causes the X-axis coordinate of atom 142 in system 150 to change in the direction of vector 153, for example, with respect to sample data 131.

[0045] Furthermore, in the example shown in Figure 1, with respect to sample data 132, the X-axis coordinate of atom 141 in system 160 changes, for example, in the direction of vector 163. Also, with respect to sample data 132, the X-axis coordinate of atom 142 in system 160 changes, for example, in the direction of vector 164.

[0046] Furthermore, in the example shown in Figure 1, with respect to sample data 133, the X-axis coordinate of atom 141 in system 170 changes, for example, in the direction of vector 172. Also, with respect to sample data 133, the X-axis coordinate of atom 142 in system 170 changes, for example, in the direction of vector 173.

[0047] (1-6) The information processing device 100 generates training data that associates each sample data 130 after adding the second noise with the second noise corresponding to that sample data 130. Here, the second noise is treated as the correct label for model 110. For example, the force acting on an atom is the force that causes the atom to return to a stable coordinate. On the other hand, the second noise is thought to act to move the atom's coordinate to an unstable coordinate. For this reason, the second noise can be used as an indicator that substitutes for the force acting on an atom. Accordingly, when training model 110, model 110 will be trained in the direction of removing the second noise from the atom's coordinate.

[0048] This allows the information processing device 100 to prepare multiple appropriate training data for use in self-supervised learning. For example, in conventional methods, for each sample data 130, a first noise calculated for each dimensional coordinate of each atom in the system represented by the sample data 130 is added to the coordinate of that dimensional coordinate. Therefore, in conventional methods, training data is prepared that associates each sample data 130 after the first noise has been added with the first noise corresponding to that sample data 130.

[0049] Therefore, conventional methods end up with multiple inconsistent training datasets, each associated with a different first noise as the correct label, even though they contain sample data 130 with common relative positional relationships between atoms. As a result, conventional methods have the problem of being difficult to properly train model 110 in the direction of removing the first noise when performing self-supervised learning.

[0050] In contrast, the information processing device 100 can prepare multiple mutually consistent training data sets, for example, including sample data 130 with common relative positional relationships between atoms, and to which a common second noise is associated as the correct label. Therefore, when performing self-supervised learning, for example, the information processing device 100 can easily train the model 110 in a way that removes the second noise.

[0051] (1-7) The information processing device 100 performs self-supervised learning on the model 110 based on the generated training data. The information processing device 100 calculates a loss 180 based on the training data using the model 110, for example. The information processing device 100 trains the model 110 by performing self-supervised learning on the model 110 based on the calculated loss 180.

[0052] As a result, the information processing device 100 can efficiently train the model 110 and reduce the processing time required for self-supervised learning. The information processing device 100 can improve the convergence speed of self-supervised learning. Specifically, the information processing device 100 can efficiently train MLIAP, which utilizes graphs representing the relative positional relationships between atoms.

[0053] The information processing device 100 can, for example, efficiently improve the inference accuracy of model 110. Specifically, the information processing device 100 can improve the inference accuracy of model 110 from sample data 130 that do not have pre-labeled data. In other words, the information processing device 100 can improve the inference accuracy of model 110 even if sample data with pre-associated labels is not available. Therefore, the information processing device 100 can reduce the time, cost, and human costs associated with preparing labels in advance.

[0054] Here, we have described the case where the functions of the information processing device 100 are realized by a single computer, but this is not the only case. For example, the functions of the information processing device 100 may be realized through the collaboration of multiple computers. For example, the functions of the information processing device 100 may be realized on the cloud.

[0055] Here, we have described a case in which the information processing device 100 generates first noise for each sample data 130, for each dimension of the atom in the system represented by the sample data 130, but it is not limited to this case. For example, another computer may generate first noise for each sample data 130, for each dimension of the atom in the system represented by the sample data 130. The information processing device 100 may also acquire first noise from the other computer.

[0056] (An example of information processing system 200) Next, using Figure 2, we will describe an example of an information processing system 200 to which the information processing device 100 shown in Figure 1 is applied.

[0057] Figure 2 is an explanatory diagram showing an example of an information processing system 200. In Figure 2, the information processing system 200 includes an information processing device 100 and one or more client devices 201.

[0058] In the information processing system 200, the information processing device 100 and the client device 201 are connected via a wired or wireless network 210. The network 210 can be, for example, a LAN (Local Area Network), a WAN (Wide Area Network), or the Internet.

[0059] The information processing device 100 is a computer for supporting self-supervised learning. The information processing device 100 stores a model, for example, MLIAP. The information processing device 100 stores a dataset containing sample data, which includes samples representing systems containing atoms. A sample is, for example, an example of input data for the model. The information processing device 100 stores a training program that uses the sample data to train the model.

[0060] The information processing device 100 receives a training request that requests self-supervised learning. The information processing device 100 receives a training request, for example, by receiving the training request from the client device 201. In response to the training request, the information processing device 100 uses a training program to train a model by performing self-supervised learning based on sample data, as in Figure 1. After training the model, the information processing device 100 may output the trained model. The information processing device 100 may, for example, provide the trained model to the client device 201.

[0061] The information processing device 100 may receive an inference request that requests the generation of labels corresponding to input data. The inference request includes, for example, the input data. The information processing device 100 receives the inference request, for example, by receiving it from the client device 201. In response to the inference request, the information processing device 100 uses a trained model to generate labels corresponding to the input data. The information processing device 100 sends the generated labels to the client device 201. The information processing device 100 is, for example, a server or a PC.

[0062] The client device 201 is a computer that sends training requests or inference requests, etc., to the information processing device 100. For example, the client device 201 generates a training request in response to user input and sends it to the information processing device 100. After sending the training request, the client device 201 may receive a trained model from the information processing device 100. This allows the client device 201 to make the trained model available on its own device.

[0063] The client device 201 acquires input data, for example, in response to user input. The client device 201 generates an inference request containing the acquired input data, for example, in response to user input, and sends it to the information processing device 100. After sending the inference request, the client device 201 may receive a label corresponding to the input data from the information processing device 100. The client device 201 outputs the label corresponding to the input data so that the user can refer to it. In this way, the client device 201 makes the label corresponding to the input data available to the user. The client device 201 may be, for example, a PC, a tablet terminal, a smartphone, or a wearable device.

[0064] This explanation describes a case where the information processing device 100 is a different device from the client device 201, but it is not limited to this case. For example, the information processing device 100 may have the functionality of a client device 201 and may operate as a client device 201. In this case, the information processing system 200 does not need to include a client device 201.

[0065] (Example of hardware configuration of information processing device 100) Next, an example of the hardware configuration of the information processing device 100 will be described using Figure 3.

[0066] Figure 3 is a block diagram showing an example of the hardware configuration of the information processing device 100. In Figure 3, the information processing device 100 includes a CPU (Central Processing Unit) 301, a memory 302, and a network interface 303. The information processing device 100 also includes a recording medium interface 304, a recording medium 305, a display 306, and an input device 307. Each component is connected by a bus 300.

[0067] Here, the CPU 301 is responsible for the overall control of the information processing device 100. The memory 302 includes, for example, ROM (Read Only Memory), RAM (Random Access Memory), and flash ROM. Specifically, for example, flash ROM and ROM store various programs, and RAM is used as the work area for the CPU 301. Programs stored in memory 302 are loaded into the CPU 301, causing the CPU 301 to execute the coded processes.

[0068] The network interface 303 is connected to network 210 via a communication line, and then connects to other computers via network 210. The network interface 303 manages the internal interface with network 210 and controls the input and output of data from other computers. The network interface 303 is, for example, a modem or a LAN adapter.

[0069] The recording medium interface (I / F) 304 controls the reading and writing of data to the recording medium 305 according to the control of the CPU 301. The recording medium interface (I / F) 304 is, for example, a disk drive, an SSD (Solid State Drive), or a USB (Universal Serial Bus) port. The recording medium 305 is a non-volatile memory that stores the data written under the control of the recording medium interface (I / F) 304. The recording medium 305 is, for example, a disk, semiconductor memory, or USB memory. The recording medium 305 may be detachable from the information processing device 100.

[0070] Display 306 displays data such as cursors, icons, toolboxes, documents, images, or functional information. Display 306 is, for example, a CRT (Cathode Ray Tube), a liquid crystal display, or an organic EL (Electroluminescence) display. Input device 307 has keys for inputting characters, numbers, or various instructions, and performs data input. Input device 307 is, for example, a keyboard or a mouse. Input device 307 may also be, for example, a touch panel input pad or a numeric keypad.

[0071] The information processing device 100 may have, in addition to the components described above, a camera, for example. Furthermore, the information processing device 100 may have, in addition to the components described above, a printer, scanner, microphone, or speaker, for example. Also, the information processing device 100 may have multiple recording medium interfaces 304 and recording mediums 305, for example. Furthermore, the information processing device 100 does not necessarily have, for example, a display 306 or an input device 307. Also, the information processing device 100 does not necessarily have, for example, recording medium interfaces 304 and recording mediums 305.

[0072] (Example hardware configuration for client device 201) The hardware configuration example for client device 201 is the same as the hardware configuration example for information processing device 100 shown in Figure 3, so the explanation is omitted.

[0073] (Example of the functional configuration of the information processing device 100) Next, an example of the functional configuration of the information processing device 100 will be described using Figure 4.

[0074] Figure 4 is a block diagram showing an example of the functional configuration of the information processing device 100. The information processing device 100 includes a storage unit 400, an acquisition unit 401, a first generation unit 402, a second generation unit 403, a third generation unit 404, a training unit 405, and an output unit 406.

[0075] The storage unit 400 is implemented by a storage area such as the memory 302 or recording medium 305 shown in Figure 3. The following description will focus on the case where the storage unit 400 is included in the information processing device 100, but is not limited to this case. For example, the storage unit 400 may be included in a device different from the information processing device 100, and the contents of the storage unit 400 may be accessible from the information processing device 100.

[0076] The acquisition unit 401 to the output unit 406 function as an example of a control unit. Specifically, the acquisition unit 401 to the output unit 406 realize their functions, for example, by having the CPU 301 execute a program stored in a storage area such as the memory 302 or recording medium 305 shown in Figure 3, or by using the network I / F 303. The processing results of each functional unit are stored in a storage area such as the memory 302 or recording medium 305 shown in Figure 3.

[0077] The memory unit 400 stores various information that is referenced or updated during the processing of each functional unit. For example, the memory unit 400 stores a model. Specifically, the memory unit 400 stores a calculation formula that serves as the model and the values ​​of the parameters included in the calculation formula. The model generates output data that represents at least the forces acting on an atom, in response to input data that represents a system including an atom. The model is subject to training by self-supervised learning. The model is, for example, MLIAP.

[0078] The input data represents, for example, the structure of a system containing atoms. The input data represents, for example, the coordinates of each atom in the system in each dimension. Specifically, the input data represents the positional relationships between atoms in a system containing atoms. The dimensions are, for example, the X-axis, Y-axis, and Z-axis. The output data may further represent the energy of the system. The output data is a label. The model is, for example, pre-configured by the user. The model may be acquired, for example, by the acquisition unit 401.

[0079] The memory unit 400 stores, for example, a dataset. The dataset is a collection of sample data. The sample data includes samples representing systems containing atoms. The sample data does not have, for example, correct labels representing the forces acting on the atoms of the system, which are not pre-prepared by the user. The dataset is pre-configured by the user, for example. The dataset may be acquired by, for example, the acquisition unit 401.

[0080] The acquisition unit 401 acquires various types of information used in the processing of each functional unit. The acquisition unit 401 stores the acquired information in the storage unit 400 or outputs it to each functional unit. The acquisition unit 401 may also output the information stored in the storage unit 400 to each functional unit. The acquisition unit 401 acquires various types of information, for example, based on user input. The acquisition unit 401 may also receive various types of information from a device other than the information processing device 100, for example.

[0081] The acquisition unit 401 acquires, for example, a training request that requests training a model. The training request may include, for example, a model. The training request may also include, for example, a dataset. Specifically, the acquisition unit 401 acquires a training request by accepting input of a training request. Specifically, the acquisition unit 401 may acquire a training request by receiving a training request from another computer. The other computer is, for example, a client device 201.

[0082] The acquisition unit 401 acquires a model, for example. Specifically, the acquisition unit 401 acquires a model by extracting it from a training request. Specifically, the acquisition unit 401 may acquire a model by accepting a model input. Specifically, the acquisition unit 401 may acquire a model by receiving a model from another computer. The other computer is, for example, a client device 201.

[0083] The acquisition unit 401 acquires a dataset, for example. Specifically, the acquisition unit 401 acquires a dataset by extracting it from a training request. Specifically, the acquisition unit 401 may acquire a dataset by accepting a dataset as input. Specifically, the acquisition unit 401 may acquire a dataset by receiving it from another computer. The other computer is, for example, a client device 201.

[0084] The acquisition unit 401 may receive a start trigger to initiate processing in any of the functional units. A start trigger may be, for example, a predetermined operation input by a user. A start trigger may also be, for example, the receipt of predetermined information from another computer. A start trigger may also be, for example, the output of predetermined information by any of the functional units. The acquisition unit 401 accepts, for example, the acquisition of a training request as a start trigger to initiate processing in the first generation unit 402, the second generation unit 403, the third generation unit 404, and the training unit 405.

[0085] The first generation unit 402 generates first noise for each dimensional coordinate of each atom in the system represented by each of the multiple sample data included in the dataset acquired by the acquisition unit 401. The first noise is, for example, a random number. The first noise may be generated according to a predetermined rule. For example, the first generation unit 402 generates first noise, which is a random number, for each dimensional coordinate of each atom in the system represented by each sample data. This allows the first generation unit 402 to generate first noise, which is the source for generating labels to be associated with the sample data.

[0086] The second generation unit 403 calculates the average of the first noise generated for each dimension of each sample data in the dataset acquired by the acquisition unit 401, for each of the multiple sample data in the dataset. The average is calculated, for example, for each sample data and for each dimension. The second generation unit 403 generates a second noise by removing the average calculated for each dimension from the first noise generated for each dimension of each atom in the system represented by the sample data. In this way, the second generation unit 403 can prepare a second noise that can be added to each dimension of each atom in the system represented by each sample data without causing inconsistencies between the sample data.

[0087] The third generation unit 404 adds the second noise generated for each dimension of the coordinates of each atom in the system represented by the sample data to the coordinates of each dimension of the sample data included in the dataset acquired by the acquisition unit 401. The third generation unit 404 generates training data that associates each sample data after the addition of the second noise with the second noise corresponding to that sample data. Here, the second noise is treated as the correct label for the sample data with respect to the model. As a result, the third generation unit 404 can prepare multiple training data sets containing each of the corresponding sample data, enabling efficient model training.

[0088] The training unit 405 performs self-supervised learning on a model that generates output data representing the force acting on each atom in a system, based on input data representing the coordinates of each dimension of each atom, based on the training data generated by the third generation unit 404. The training unit 405 calculates a loss on the model using a loss function, for example, based on the difference between the output data generated by the model when input sample data included in the training data and the second noise corresponding to the sample data included in the training data. The training unit 405 updates the model parameters in a way that reduces the calculated loss, for example.

[0089] This allows the training unit 405 to use sample data for which labels have not been pre-prepared by the user to train the model. The training unit 405 can efficiently train the model based on multiple training datasets, each containing matching sample data. The training unit 405 can reduce the processing time required when performing self-supervised learning.

[0090] The output unit 406 outputs the processing result of at least one of the functional units. The output format can be, for example, display on a screen, print to a printer, transmit to an external device via the network interface 303, or store in a storage area such as the memory 302 or recording medium 305. This allows the output unit 406 to notify the user of the processing result of at least one of the functional units, thereby improving the usability of the information processing device 100.

[0091] The output unit 406 outputs, for example, the model trained by the training unit 405. Specifically, the output unit 406 outputs the model so that it can be viewed by the user. Specifically, the output unit 406 may also send the model to another computer, such as the client device 201. This allows the output unit 406 to make the model available externally.

[0092] (Example of operation of the information processing device 100) Next, we will explain an example of the operation of the information processing device 100 using Figures 5 to 13. First, using Figure 5, we will explain the process by which the information processing device 100 trains the model 500 using self-supervised learning.

[0093] In the following explanation, for convenience, when a subscript is added to a specific character, it may be written as "(specific character)_(subscript)". Also, for convenience, when a superscript is added to a specific character, it may be written as "(specific character)^(superscript)". Also, for convenience, when a ~ is added directly above a specific character, it may be written as "(specific character)~". Also, for convenience, when a hyphen is added directly above a specific character, it may be written as "(specific character) - It is sometimes written as "".

[0094] Figure 5 is an explanatory diagram showing the flow of training model 500. In Figure 5, the information processing device 100 stores model 500. Model 500 is, for example, the MLIAP620, which will be described later in Figure 6. Model 500 has parameters 501. The information processing device 100 stores, for example, the calculation formula that becomes model 500 and the values ​​of the parameters 501 included in the calculation formula. Training is updating model 500. Specifically, training is updating the parameters 501.

[0095] The information processing device 100 stores dataset 510 and dataset 520. Dataset 510 is a data set that collects multiple primary data representing a relatively broad range of knowledge. Multiple primary data is a data set that collects multiple types of primary data, regardless of the actual use of model 500. Multiple primary data includes, for example, unlabeled data. Unlabeled data is sample data that represents a sample of data to be input to model 500. Unlabeled data does not contain the correct labels.

[0096] Dataset 520 is a collection of multiple secondary datasets that represent relatively narrow knowledge. These secondary datasets are, for example, collections of secondary datasets of types that match the actual elements of model 500. These secondary datasets include, for example, labeled data. Labeled data is training data that associates a sample of data to be input into model 500 with the correct label corresponding to that sample.

[0097] The information processing device 100 trains the model 500 by performing pre-training on the model 500 based on the dataset 510. Specifically, the training involves optimizing the parameters 501 based on the loss 511. Subsequently, the information processing device 100 further trains the model 500 by performing fine-tuning on the pre-trained model 500 based on the dataset 520. Specifically, the training involves optimizing the parameters 501 based on the loss 521. Next, we will move on to explaining Figures 6 and 7 and describe an example of MLIAP620.

[0098] Figures 6 and 7 are explanatory diagrams illustrating an example of MLIAP620. In Figure 6, MLIAP620 has a parameter 621, which corresponds to parameter 501. MLIAP620 has the function of generating a label 630 corresponding to a sample 610, in response to the input sample 610. In the example in Figure 6, the sample 610 represents the position of an atom forming the system and the element of that atom. The label 630 represents the energy of the system and the force acting on the atom. Next, we will move on to the explanation of Figure 7.

[0099] As shown in Figure 7, the MLIAP620 specifically comprises an encoder 710, an energy head 720, and a force head 730. The encoder 710 has parameter 711. The energy head 720 has parameter 721. The force head 730 has parameter 731.

[0100] Encoder 710 refers to sample 610 and calculates the feature quantity of atom 0 based on atom 0, information about the area surrounding atom 0, and parameter 711. Encoder 710 refers to sample 610 and calculates the feature quantity of atom 1 based on atom 1, information about the area surrounding atom 1, and parameter 711. Energy head 720 calculates the energy of the system based on the feature quantity of atom 0, the feature quantity of atom 1, and parameter 721.

[0101] The force head 730 calculates the forces acting on atom 0 and atom 1 based on the features of atom 0, the features of atom 1, and the parameter 731. The MLIAP620 generates labels 630 based on the energy of the system, the forces acting on atom 0, and the forces acting on atom 1. The forces acting on atoms may be calculated, for example, using the energy head 720.

[0102] Next, using Figure 8, we will explain an example in which the information processing device 100 performs self-supervised learning on the MLIAP620 during pre-training.

[0103] Figure 8 is an explanatory diagram illustrating an example of implementing self-supervised learning. In Figure 8, the dataset 510 specifically contains D first data points.

[0104] (8-1) The information processing device 100 randomly selects first data 810 from the dataset 510, which includes a sample representing a system containing N atoms, to be used when performing self-supervised learning. The first data 810 represents, for example, the coordinate x ∈ R^(N×3) of an atom in three-dimensional space. The first data 810 is unlabeled data and is sample data.

[0105] Data 1, 810, relates to an H2 molecule. An H2 molecule is formed from two H atoms. In Data 1, 810, indices 0 and 1 are assigned to each atom that makes up the H2 molecule. Data 1, 810, represents the coordinates of each atom that makes up the H2 molecule. The coordinates include, for example, x-axis coordinate values, y-axis coordinate values, and z-axis coordinate values.

[0106] (8-2) The information processing device 100 generates a random number Δx∈R^(N×3) representing a first noise corresponding to the coordinates of each atom for the first data 810. The first noise is a three-dimensional vector. The first noise includes, for example, an x-axis component, a y-axis component, and a z-axis component.

[0107] The information processing device 100 generates a random number Δx for the first data 810 that includes a component following a normal distribution N(0,σ^2) and represents a first noise corresponding to the coordinates of each atom. σ^2 is the variance. The information processing device 100 may also generate a random number Δx for the first data 810 that includes a component 0 in the range of probability 0 < α < 1, and a component following a normal distribution N(0,σ^2) outside the range of probability 0 < α < 1, representing a first noise corresponding to the coordinates of each atom.

[0108] In the example shown in Figure 8, the information processing device 100 specifically generates the random number Δx shown in Table 820 for the first data 810. Specifically, the X-axis component for the atom at index 0 is 0.1. Specifically, the Y-axis component for the atom at index 0 is -0.2. Specifically, the Z-axis component for the atom at index 0 is 0.3. Specifically, the X-axis component for the atom at index 1 is -0.4. Specifically, the Y-axis component for the atom at index 1 is 0.5. Specifically, the Z-axis component for the atom at index 1 is -0.6.

[0109] (8-3) The information processing device 100 calculates the average Δx of the first noise in each dimension for the entire set of N atoms for the first data 810 according to the following formula (1). - The coefficient ∈R^(N×3) is calculated. In the example in Figure 8, the information processing device 100 specifically calculates the average Δx for the first data 810. - Let's assume that the following was generated. Specifically, the average of the X-axis components is (0.1 + (-0.4)) / 2 = -0.15. Also, specifically, the average of the Y-axis components is ((-0.2) + 0.5) / 2 = 0.15. Also, specifically, the average of the Z-axis components is (0.3 + (-0.6)) = -0.15.

[0110]

number

[0111] (8-4) The information processing device 100 calculates, as the second noise to be added to the coordinates of each atom, (random number Δx) - (average Δx - ) for the first data 810. In the example of FIG. 8, it is assumed that the information processing device 100 calculates the second noise shown in Table 830 for the first data 810.

[0112] (8-5) The information processing device 100 associates and represents (x + Δx - Δx - ), which is the first data 810 with the second noise added to the coordinate x thereof, with the second noise (Δx - Δx - ), which is the correct label, to generate teacher data x - = (x + Δx - Δx - , Δx - Δx - ). (x + Δx - Δx - ) is shown in Table 840. The information processing device 100 trains the MLIAP 620 based on the generated teacher data. Thereby, the information processing device 100 can efficiently perform self-supervised learning.

[0113] Here, the case where the correct label includes at least the second noise has been described, but it is not limited to this. For example, the correct label of the label may further include a feature amount that substitutes the energy of a system including molecules. The feature amount is, for example, a truth value indicating whether or not the system contains a specific element.

[0114] Next, by comparing the case where the model 500 is trained by the proposed method by the information processing device 100 and the conventional method using FIGS. 9 to 11, an example of the effect obtained by the proposed method by the information processing device 100 will be described. Here, the conventional method trains the model 500 based on teacher data that associates and represents (x + Δx), which is the first noise added to the coordinate x of the first data 810, with the first noise Δx as the correct label of the label.

[0115] FIGS. 9 to 11 are explanatory diagrams showing an example of the effect. First, we will move on to the description of FIG. 9. Specifically, FIG. 9 shows the case where the model 500 is trained by the conventional method.

[0116] In conventional methods, specifically, training data is prepared for the first data 810 without considering the average of the first noise in each dimension. Therefore, conventional methods may end up with multiple inconsistent training data sets, where different first noises are associated as the correct labels, even though the relative positions between atoms are the same.

[0117] Specifically, in the example in Figure 9, let's assume that there are first data points 811 to 813. First data point 811 relates to system 910, which contains two atoms 901 and 902. First data point 812 relates to system 920, which contains two atoms 901 and 902. First data point 813 relates to system 930, which contains two atoms 901 and 902. Atoms 901 and 902 are hydrogen atoms.

[0118] In the conventional method, training data is generated by adding the noise component represented by vector 911 to the coordinates of atom 902 in the first data 811. Similarly, in the conventional method, training data is generated by adding the noise component represented by vector 921 to the coordinates of atom 901 in the first data 812, and the noise component represented by vector 922 to the coordinates of atom 902 in the first data 812. Similarly, in the conventional method, training data is generated by adding the noise component represented by vector 931 to the coordinates of atom 901 in the first data 813.

[0119] The training data is thought to represent how the coordinates of each atom in the system should be preferably corrected during inference by Model 500. Training is thought to correspond to updating the parameters 501 of Model 500 so that the coordinates of each atom in the system are appropriately corrected according to the meaning of the training data.

[0120] Here, we consider the case where model 500 is trained based on training data using a conventional method. In this case, it is preferable to remove noise components from the training data corresponding to the first data 811 during inference, so that the coordinates of atom 902 are represented by vector 941. On the other hand, it is preferable to remove noise components from the training data corresponding to the first data 812 during inference, so that the coordinates of atom 901 are represented by vector 951, and the coordinates of atom 902 are represented by vector 952.

[0121] Furthermore, the training data corresponding to the first data 813 preferably has noise components removed during inference so that the coordinates of atom 901 are represented by vector 961. In this way, conventional methods may result in inconsistent meanings between multiple training data. For this reason, conventional methods may find it difficult to learn how to handle the coordinates of atoms 901 and 902 based on each training data, and may not be able to properly train model 500. Next, we will move on to the explanation of Figure 10. Figure 10 specifically shows the case when model 500 is trained using the proposed method.

[0122] In the proposed method, specifically, training data is prepared for the first data 810 (811-813) by considering the average of the first noise in each dimension. Therefore, the proposed method can prepare multiple training data sets so that a common second noise is associated as the correct label for multiple training data sets that share the same relative positional relationship between atoms.

[0123] Specifically, in the example in Figure 10, we assume that there are first data points 811 to 813, similar to the example in Figure 9. First data point 811 relates to system 910, which contains two atoms 901 and 902. First data point 812 relates to system 920, which contains two atoms 901 and 902. First data point 813 relates to system 930, which contains two atoms 901 and 902.

[0124] Specifically, the information processing device 100 can generate a second noise for atoms 901 and 902 of the first data 811, taking into account the average of the first noise in each dimension. Specifically, the information processing device 100 can generate training data by adding the noise component represented by vector 1011 to the coordinates of atom 901 of the first data 811, and by adding the noise component represented by vector 1012 to the coordinates of atom 902 of the first data 811.

[0125] Similarly, the information processing device 100 can specifically generate a second noise for atoms 901 and 902 of the first data 812, taking into account the average of the first noise in each dimension. Specifically, the information processing device 100 can generate training data by adding the noise component represented by vector 1021 to the coordinates of atom 901 of the first data 812, and by adding the noise component represented by vector 1022 to the coordinates of atom 902 of the first data 812.

[0126] Similarly, the information processing device 100 can specifically generate a second noise for atoms 901 and 902 of the first data 813, taking into account the average of the first noise in each dimension. Specifically, the information processing device 100 can generate training data by adding the noise component represented by vector 1031 to the coordinates of atom 901 of the first data 813, and by adding the noise component represented by vector 1032 to the coordinates of atom 902 of the first data 813.

[0127] Here, it is preferable that, during inference, the training data corresponding to the first data 811 has noise components removed from the coordinates of atom 901 so that they are represented by vector 1041, and from the coordinates of atom 902 so that they are represented by vector 1042.

[0128] Similarly, it is preferable that, during inference, the training data corresponding to the first data 812 has noise components removed from the coordinates of atom 901 so that they are represented by vector 1051, and from the coordinates of atom 902 so that they are represented by vector 1052.

[0129] Similarly, it is preferable that, during inference, the training data corresponding to the first data 813 has noise components removed from the coordinates of atom 901 so as represented by vector 1061, and from the coordinates of atom 902 so as represented by vector 1062.

[0130] In this way, the information processing device 100 can reconcile the meanings of multiple training data sets that share a common relative positional relationship between atoms, and can prepare multiple appropriate training data sets for use in self-supervised learning. Therefore, the information processing device 100 can appropriately learn how to handle the coordinates of atoms 901 and 902 based on each training data set, and can appropriately train the model 500. Furthermore, the information processing device 100 can reduce the processing time required when training the model 500. Next, we will move on to the explanation of Figure 11.

[0131] In Figure 11, Model 500 is the PaiNN model. In the example in Figure 11, the conventional method specifically trains the PaiNN model based on the PubChem database, which contains data representing the three-dimensional structures of compounds treated as unlabeled data. The proposed method specifically trains the PaiNN model based on the PubChem database, which contains data representing the three-dimensional structures of compounds treated as unlabeled data.

[0132] In Figure 11, 95% of the data representing the three-dimensional structures of compounds with compound IDs between 1 and 1,000,000 in the PubChem database are randomly selected and treated as unlabeled training data. The remaining 5% of the data representing the three-dimensional structures of compounds with compound IDs between 1 and 1,000,000 in the PubChem database are treated as unlabeled evaluation data.

[0133] In the example shown in Figure 11, the information processing device 100 generates a first noise based on a normal distribution with mean 0 and variance 0.1 for each atom in unlabeled data. The information processing device 100 uses L2 MAE (Mean Average Error) × number of atoms as its loss function. The information processing device 100 sets the number of epochs to 2.

[0134] Graph 1100 shows the change in MAE of the noise when the PaiNN model is trained using the proposed method, and the change in MAE of the noise when the PaiNN model is evaluated using the proposed method. Graph 1100 also shows the change in MAE of the noise when the PaiNN model is trained using the conventional method, and the change in MAE of the noise when the PaiNN model is evaluated using the conventional method.

[0135] As shown in Graph 1100, the number of steps required for the noise MAE to reach 0.03 Å, depending on the unlabeled data used for evaluation, is, for example, 15,000 to 20,000 for the conventional method. On the other hand, as shown in Graph 1100, the number of steps required for the noise MAE to reach 0.03 Å, depending on the unlabeled data used for evaluation, is, for example, 5,000 to 10,000 for the proposed method. Therefore, the proposed method can accelerate the convergence speed of the noise MAE by 2.0 times compared to the conventional method.

[0136] Furthermore, the information processing device 100 can generate a second noise from a first noise through simple calculations. Therefore, the information processing device 100 can suppress the increase in processing load caused by generating a second noise from a first noise to a relatively small amount.

[0137] This explanation describes a case where the information processing device 100 performs self-supervised learning on the MLIAP620 during pre-training, but it is not limited to this. For example, the information processing device 100 may similarly perform self-supervised learning on the MLIAP620 during fine-tuning.

[0138] (An example of applying the information processing device 100) The information processing device 100 can be applied, for example, to fields such as drug discovery or materials development. Specifically, the information processing device 100 can efficiently train MLIAP, which is used for first-principles calculations or molecular dynamics. Therefore, the information processing device 100 can make it easier to analyze systems such as molecules by performing first-principles calculations or molecular dynamics in fields such as drug discovery or materials development.

[0139] (Overall processing procedure) Next, an example of the overall processing procedure executed by the information processing device 100 will be described using Figure 12. The overall processing is realized, for example, by the CPU 301 shown in Figure 3, storage areas such as memory 302 and recording medium 305, and network I / F 303.

[0140] Figure 12 is a flowchart showing an example of the overall processing procedure. In Figure 12, the information processing device 100 initializes the model parameter W (step S1201).

[0141] The information processing device 100 obtains Z, G, and y by performing the individual processing described later in Figure 13 (step S1202). The information processing device 100 uses the model to calculate the inferred label value y~ (step S1203).

[0142] The information processing device 100 calculates the label loss l = Loss(y, y~) using the loss function Loss() (step S1204). The information processing device 100 updates W based on l (step S1205).

[0143] The information processing device 100 determines whether or not the termination conditions are met (step S1206). The termination conditions are, for example, that W has been updated a predetermined number of times. The predetermined number of times is, for example, set in advance by the user. Another termination condition is that the loss is below a threshold. The threshold is, for example, set in advance by the user.

[0144] If the termination condition is not met (step S1206: No), the information processing device 100 returns to the process in step S1202. On the other hand, if the termination condition is met (step S1206: Yes), the information processing device 100 outputs W (step S1207) and terminates the entire process.

[0145] (Individual processing procedures) Next, an example of an individual processing procedure executed by the information processing device 100 will be described using Figure 13. Individual processing is realized, for example, by the CPU 301 shown in Figure 3, storage areas such as memory 302 and recording medium 305, and network I / F 303.

[0146] Figure 13 is a flowchart of an example of an individual processing procedure. In Figure 13, the information processing device 100 sets Z=Φ, x=Φ, y=Φ, i=0 (step S1301). The information processing device 100 then sets i=i+1 (step S1302).

[0147] The information processing device 100 obtains a sample {Z_i, x_i} for a system containing N_i atoms (step S1303). Z_i is a tensor of atomic numbers. x_i is the three-dimensional coordinate of the atom. The information processing device 100 randomly generates Δx_i (step S1304).

[0148] The information processing device 100 calculates Δx based on Δx_i. - The information processing device 100 calculates _i (step S1305). Δx ~ _i = x_i + Δx_i - Δx - The information processing device 100 calculates _i (step S1306). y_i = Δx_i - Δx - Calculate _i (step S1307).

[0149] The information processing device 100 sets Z ← concat(Z, Z_i) (step S1308). concat(x, y) represents the operation of concating tensor x and tensor y at the first dimension. concat(Φ, y) = y. The information processing device 100 sets x ← concat(x, x_i) (step S1309). The information processing device 100 sets y ← concat(y, y_i) (step S1310).

[0150] The information processing device 100 determines whether i > S or not (step S1311). S is, for example, pre-set by the user. If i ≤ S and not i > S (step S1311: No), the information processing device 100 returns to the process in step S1302. On the other hand, if i > S (step S1311: Yes), the information processing device 100 generates a graph G based on x (step S1312) and terminates the individual processing.

[0151] As explained above, the information processing device 100 can generate a first noise for each dimension of each atom in the system represented by multiple sample data representing the coordinates of each dimension of each atom in the system. The information processing device 100 can calculate the average of the first noise generated for each dimension of each sample data for each dimension of each atom in the system represented by that sample data. The information processing device 100 can generate a second noise for each sample data by removing the average calculated for each dimension from the first noise generated for each dimension of each atom in the system represented by that sample data. The information processing device 100 can add the second noise generated for each dimension of each sample data to the coordinates of each dimension of each atom in the system represented by that sample data. The information processing device 100 can prepare training data that associates each sample data after the addition with the second noise corresponding to that sample data. According to the information processing device 100, self-supervised learning can be performed on a model that generates output data representing the force acting on each atom in a system, based on input data representing the coordinates of each dimension of each atom, according to the prepared training data. This allows the information processing device 100 to prepare multiple training data sets that are consistent with each other, and to efficiently train the model. The information processing device 100 can also reduce the processing time required when performing self-supervised learning.

[0152] According to the information processing device 100, the loss for the model can be calculated using a loss function based on the difference between the output data generated by the model, which is input to sample data included in the training data, and the second noise corresponding to the sample data included in the training data. According to the information processing device 100, the model parameters can be updated in a direction that reduces the calculated loss. As a result, the information processing device 100 can appropriately train the model.

[0153] According to the information processing device 100, machine learning interatomic potentials can be used as the model to be trained. This allows the information processing device 100 to efficiently train machine learning interatomic potentials.

[0154] According to the information processing device 100, a first noise, which is a random number, can be generated for the coordinates of each dimension of each atom in the system represented by each sample data. This allows the information processing device 100 to generate appropriate first noise.

[0155] The information processing method described in this embodiment can be implemented by executing a pre-prepared program on a computer such as a PC or workstation. The information processing program described in this embodiment is recorded on a computer-readable recording medium and executed by being read from the recording medium by the computer. The recording medium can be a hard disk, flexible disk, CD (Compact Disc)-ROM, MO (Magneto Optical Disc), DVD (Digital Versatile Disc), etc. Furthermore, the information processing program described in this embodiment may be distributed via a network such as the Internet.

[0156] With regard to the embodiments described above, the following additional information is disclosed.

[0157] (Note 1) For each of the multiple sample data, a first noise is generated for the coordinates of each dimension of each atom in the system represented by the sample data. For each of the aforementioned sample data, the average of the first noise generated for the coordinates of each atom in the system represented by the sample data in that dimension is calculated. For each of the aforementioned sample data, a second noise is obtained by removing the average calculated for each dimension from the first noise generated for each dimension of the coordinates of each atom in the system represented by the sample data, and this second noise is added to the coordinates of each dimension. Self-supervised learning is performed on a model that generates output data representing the force acting on each atom in the system, based on input data representing the coordinates of each atom in each dimension, using training data that associates each of the added sample data with the corresponding second noise. An information processing program characterized by having a computer perform the processing.

[0158] (Note 2) The process to be carried out above is: The information processing program according to Appendix 1, characterized in that it calculates a loss for the model using a loss function according to the difference between output data generated by the model, which is input to sample data included in the training data, and the second noise corresponding to the sample data included in the training data, and updates the parameters of the model in a direction that reduces the calculated loss.

[0159] (Note 3) The information processing program according to Note 1 or 2, characterized in that the model is a machine learning interatomic potential.

[0160] (Note 4) The above generation process is: An information processing program according to any one of the appendices 1 to 3, characterized by generating a first noise, which is a random number, for the coordinates of each dimension of each atom in the system represented by each of the aforementioned sample data.

[0161] (Note 5) For each of the multiple sample data, a first noise is generated for the coordinates of each dimension of each atom in the system represented by the sample data. For each of the aforementioned sample data, the average of the first noise generated for the coordinates of each atom in the system represented by the sample data in that dimension is calculated. For each of the aforementioned sample data, a second noise is obtained by removing the average calculated for each dimension from the first noise generated for each dimension of the coordinates of each atom in the system represented by the sample data, and this second noise is added to the coordinates of each dimension. Self-supervised learning is performed on a model that generates output data representing the force acting on each atom in the system, based on input data representing the coordinates of each atom in each dimension, using training data that associates each of the added sample data with the corresponding second noise. An information processing method characterized in that the processing is performed by a computer.

[0162] (Note 6) For each of the multiple sample data, a first noise is generated for the coordinates of each dimension of each atom in the system represented by the sample data. For each of the aforementioned sample data, the average of the first noise generated for the coordinates of each atom in the system represented by the sample data in that dimension is calculated. For each of the aforementioned sample data, a second noise is obtained by removing the average calculated for each dimension from the first noise generated for each dimension of the coordinates of each atom in the system represented by the sample data, and this second noise is added to the coordinates of each dimension. Self-supervised learning is performed on a model that generates output data representing the force acting on each atom in the system, based on input data representing the coordinates of each atom in each dimension, using training data that associates each of the added sample data with the corresponding second noise. An information processing device characterized by having a control unit. [Explanation of Symbols]

[0163] 100 Information Processing Devices 110,500 models 111,501,621,711,721,731 parameters 120,510,520 datasets 130-133 Sample data 141,142,901,902 atoms 150, 160, 170, 910, 920, 930 series 151~153,161~164,171~173,911,921,922,931,941,951,952,961,1011,1012,1021,1022,1031,1032,1041,1042,1051,1052,1061,1062 vectors 180,511,521 loss 200 Information Processing Systems 201 Client Device 210 Network 300 bus 301 CPU 302 memory 303 Network I / F 304 Recording medium interface 305 Recording media 306 displays 307 Input device 400 Storage section 401 Acquisition Department 402 1st generation part 403 Second generation part 404 Third generation part 405 Training Department 406 Output section 610 specimens 620 MLIAP 630 labels 710 Encoder 720 Energy Head 730 Power Head 810-813 First Data 820,830,840 table 1100 Graph

Claims

1. For each of the multiple sample data, a first noise is generated for the coordinates of each dimension of each atom in the system represented by that sample data. For each of the aforementioned sample data, the average of the first noise generated for each dimension of the system represented by the sample data is calculated, For each of the aforementioned sample data, a second noise is obtained by removing the average calculated for each dimension from the first noise generated for each dimension of the atom in the system represented by the sample data, and this second noise is added to the coordinates of each dimension. Self-supervised learning is performed on a model that generates output data representing the force acting on each atom in the system, based on input data representing the coordinates of each dimension of each atom, using training data that associates each of the added sample data with the second noise corresponding to that sample data. An information processing program characterized by having a computer perform the processing.

2. The process to be carried out as described above is: The information processing program according to claim 1, characterized in that it calculates a loss for the model using a loss function according to the difference between output data generated by the model, which is input to sample data included in the training data, and the second noise corresponding to the sample data included in the training data, and updates the parameters of the model in a direction that reduces the calculated loss.

3. The information processing program according to claim 1 or 2, characterized in that the model is a machine learning interatomic potential.

4. For each of the multiple sample data, a first noise is generated for the coordinates of each dimension of each atom in the system represented by that sample data. For each of the aforementioned sample data, the average of the first noise generated for each dimension of the system represented by the sample data is calculated, For each of the aforementioned sample data, a second noise is obtained by removing the average calculated for each dimension from the first noise generated for each dimension of the atom in the system represented by the sample data, and this second noise is added to the coordinates of each dimension. Self-supervised learning is performed on a model that generates output data representing the force acting on each atom in the system, based on input data representing the coordinates of each dimension of each atom, using training data that associates each of the added sample data with the second noise corresponding to that sample data. An information processing method characterized in that the processing is performed by a computer.

5. For each of the multiple sample data, a first noise is generated for the coordinates of each dimension of each atom in the system represented by that sample data. For each of the aforementioned sample data, the average of the first noise generated for each dimension of the system represented by the sample data is calculated, For each of the aforementioned sample data, a second noise is obtained by removing the average calculated for each dimension from the first noise generated for each dimension of the atom in the system represented by the sample data, and this second noise is added to the coordinates of each dimension. Self-supervised learning is performed on a model that generates output data representing the force acting on each atom in the system, based on input data representing the coordinates of each dimension of each atom, using training data that associates each of the added sample data with the second noise corresponding to that sample data. An information processing device characterized by having a control unit.

Images