Information processing program, information processing method, and information processing device

Abstract

An information processing device (100) generates, with respect to each piece of sample data (130), a first noise for the coordinates of each dimension of each atom of a system represented by the sample data (130). The information processing device (100), with respect to each piece of the sample data (130), calculates a second noise by removing the average of the first noise from the first noise generated for the coordinates of each dimension of each atom of the system represented by the sample data (130). The information processing device (100) performs self-supervised learning on a model (110) on the basis of training data representing each piece of the sample data (130) to which the second noise has been added and the second noise corresponding to the sample data (130) in association with each other. This makes it possible for the information processing device (100) to efficiently train the model (110), which may be used when performing first principle calculation, molecular dynamics, or the like in the field of drug discovery, material development, or the like, for example.

Description

Information processing program, information processing method, and information processing device

[0001] The present invention relates to an information processing program, an information processing method, and an information processing apparatus.

[0002] Traditionally, there are techniques for training models that output labels corresponding to input data. These models are, for example, neural networks called machine learning interatomic potentials, used in fields such as first-principles calculations or molecular dynamics. The data, for example, represents the coordinates of each atom in a system of multiple atoms, used in fields such as first-principles calculations or molecular dynamics. For instance, there is a technique called "self-supervised learning" that automatically generates correct labels for data samples and trains the model based on training data that associates the data samples with these automatically generated correct labels. Specifically, one could generate random noise to serve as the correct labels, and then train the model based on training data that associates data samples with this noise with the corresponding correct labels.

[0003] Prior art, for example, involves obtaining a third feature relating to one or more predetermined types of energy of a molecule based on a graph containing a first feature relating to the molecule and a second feature relating to the energy assigned to the atoms constituting the molecule.

[0004] Japanese Patent Publication No. 2024-072890

[0005] However, conventional techniques can sometimes make it difficult to efficiently train models. For example, consider training a machine learning interatomic potential that takes into account the relative positions of atoms. In this case, if there are multiple training datasets with the same relative positions of atoms but different label content, it becomes impossible to efficiently train the machine learning interatomic potential.

[0006] In one aspect, the present invention aims to make it easier to efficiently train models.

[0007] According to one embodiment, an information processing program, information processing method, and information processing device are proposed to perform self-supervised learning on a model that generates a first noise for each of a plurality of sample data for each of the coordinates of each dimension of each atom in the system represented by the sample data, calculates the average of the first noise generated for each dimension of each of the sample data for each of the dimensions of each of the sample data for each of the dimensions of each of the sample data for each of the dimensions of each of the sample data, adds a second noise obtained by removing the average calculated for each dimension from the first noise generated for each of the coordinates of each dimension of each of the atoms in the system represented by the sample data for each of the dimensions of each of the sample data for each of the sample data, and generates output data representing the force acting on each atom according to input data representing the coordinates of each dimension of each atom in the system.

[0008] According to one embodiment, it becomes possible to train the model more efficiently.

[0009] Figure 1 is an explanatory diagram showing one embodiment of the information processing method according to the embodiment. Figure 2 is an explanatory diagram showing an example of an information processing system 200. Figure 3 is a block diagram showing an example of the hardware configuration of the information processing device 100. Figure 4 is a block diagram showing an example of the functional configuration of the information processing device 100. Figure 5 is an explanatory diagram showing the flow of training the model 500. Figure 6 is an explanatory diagram (part 1) showing an example of MLIAP 620. Figure 7 is an explanatory diagram (part 2) showing an example of MLIAP 620. Figure 8 is an explanatory diagram showing an example of implementing self-supervised learning. Figure 9 is an explanatory diagram (part 1) showing an example of the effect. Figure 10 is an explanatory diagram (part 2) showing an example of the effect. Figure 11 is an explanatory diagram (part 3) showing an example of the effect. Figure 12 is a flowchart showing an example of the overall processing procedure. Figure 13 is a flowchart showing an example of the individual processing procedure.

[0010] Embodiments of the information processing program, information processing method, and information processing apparatus according to the present invention will be described in detail below with reference to the drawings.

[0011] (An Embodiment of the Information Processing Method According to the Embodiment) Figure 1 is an explanatory diagram showing an embodiment of the information processing method according to the embodiment. The information processing device 100 is a computer that facilitates training a model when performing self-supervised learning. The information processing device 100 is, for example, a server or a PC (Personal Computer).

[0012] Self-supervised learning is a technique called machine learning, which trains a model to output labels corresponding to input data. A model is a computational formula containing parameters. An example of a model is a neural network. Specifically, a model is a machine learning interatomic potential in fields such as first-principles calculations or molecular dynamics. In the following explanation, the machine learning interatomic potential may be referred to as "MLIAP (Machine Learning InterAtomic Potential)".

[0013] Labels represent, for example, the characteristics of the data. Labels may also represent, for example, the result of classifying the data. Specifically, if the model being trained is MLIAP, the data represents a system containing atoms. The labels then represent the energy of the system and the forces acting on the atoms. The system is, for example, the structure of a molecule. The forces are, for example, three-dimensional vectors. Therefore, when MLIAP receives input data representing, for example, a system containing atoms, it outputs labels that represent the energy of the system and the forces acting on the atoms corresponding to that data.

[0014] Specifically, MLIAP comprises an encoder for calculating system features, an energy head for calculating the system's energy from these features, and a force head for calculating the forces acting on atoms from these features. Specifically, MLIAP uses the encoder to calculate system features based on the element of each atom and information about the atoms' surroundings. The element is the type of atom. Information about the atoms' surroundings is, for example, a graph representing the relative positional relationships between atoms. Specifically, MLIAP uses the energy head to calculate the system's energy based on the system's features. Specifically, MLIAP uses the force head to calculate the forces acting on atoms based on the system's features.

[0015] Training involves updating the model's parameters, for example. Training is also called learning. Training utilizes techniques such as mathematical optimization. Specifically, training employs methods like backpropagation and stochastic gradient descent. The process by which the model generates labels corresponding to the data is also called inference.

[0016] Self-supervised learning is applied in situations where ground truth labels corresponding to data samples are not pre-prepared. A sample is also called a sample. Self-supervised learning, for example, automatically generates ground truth labels for data samples. Self-supervised learning trains a model based on a dataset of training data that associates data samples with the automatically generated ground truth labels. Because self-supervised learning does not require the pre-preparation of ground truth labels, it can reduce the time and cost associated with preparing labels compared to supervised learning. In the following explanation, self-supervised learning may be referred to as "SSL (Self-Supervised Learning)".

[0017] In self-supervised learning, specifically when the model being trained is MLIAP, random noise can be used as the ground truth for the labels. Specifically, self-supervised learning generates random noise for each atom, and then generates training data that includes a sample of data with the generated random noise added together, and the ground truth for the labels representing the generated random noise. Self-supervised learning trains the model based on a dataset of the generated training data.

[0018] Machine learning techniques, such as self-supervised learning, are used in methods like pre-training and fine-tuning. Pre-training is the first step in the process. Fine-tuning is the second step in the process.

[0019] Pre-training is a method of training a model, for example, based on a relatively large dataset. Specifically, pre-training involves training a model based on a dataset of training data that associates sample documents written in an arbitrary language with the corresponding labels, and then simulating data of documents written in various languages.

[0020] Fine-tuning is a technique for further training a model that has already been pre-trained to suit a specific task. A task is a problem of generating labels corresponding to samples, based on those samples. Fine-tuning can be done, for example, by training a model on a relatively small dataset related to a specific task.

[0021] Fine-tuning specifically involves training a model to fit a dataset of Japanese-language documents, based on training data that associates samples of documents written in Japanese with the corresponding labels.

[0022] The combination of pre-training and fine-tuning is expected to reduce the time and cost associated with improving the inference accuracy of a model. Specifically, pre-training preferably utilizes self-supervised learning to reduce the time and cost associated with preparing large datasets.

[0023] Fine-tuning, specifically, preferably utilizes supervised learning to make the model more adaptable to a particular task. Supervised learning is applicable when ground truth labels corresponding to data samples are provided in advance. For example, supervised learning trains a model based on a dataset of training data that associates data samples with pre-prepared ground truth labels. In the following explanation, supervised learning may be referred to as "SL (Supervised Learning)".

[0024] However, conventional techniques sometimes make it difficult to implement self-supervised learning. For example, it is difficult to implement self-supervised learning for MLIAP. As mentioned above, MLIAP sometimes uses graphs that represent the relative positions of atoms. On the other hand, when adding random noise to data samples in self-supervised learning, it is possible to generate multiple inconsistent training data sets where the correct labels differ, even though the relative positions of atoms are the same for each set. For this reason, it may not be possible to efficiently train MLIAP using self-supervised learning.

[0025] Therefore, this embodiment describes an information processing method that can facilitate efficient model training.

[0026] In Figure 1, the information processing device 100 stores the model 110. The model 110 generates output data representing at least the forces acting on an atom, in response to input data representing a system including an atom. The model 110 is the subject of training by self-supervised learning. The model 110 has parameters 111. The model 110 is, for example, MLIAP.

[0027] The input data represents, for example, the structure of a system containing atoms. The input data represents, for example, the coordinates of each atom in the system in each dimension. Specifically, the input data represents the positional relationships between atoms in a system containing atoms. The dimensions are, for example, the X-axis, Y-axis, and Z-axis. The output data may also represent the energy of the system. The output data is a label.

[0028] (1-1) The information processing device 100 acquires a dataset 120. The dataset 120 is a collection of sample data 130. The sample data 130 includes samples representing systems containing atoms. The sample data 130 does not include the correct labels representing the energy of the system and the forces acting on the atoms. In the example in Figure 1, the dataset 120 specifically includes sample data 131 to 133.

[0029] In the example in Figure 1, for the sake of simplicity, sample data 131 to 133 are assumed to relate to two atoms 141 and 142, respectively. Atoms 141 and 142 are, for example, H atoms or O atoms. Specifically, sample data 131 represents the structure of system 150 containing atoms 141 and 142. Similarly, sample data 132 represents the structure of system 160 containing atoms 141 and 142. Furthermore, sample data 133 represents the structure of system 170 containing atoms 141 and 142.

[0030] (1-2) The information processing device 100 generates a first noise for each sample data 130 included in the acquired dataset 120, for each dimension coordinate of each atom in the system represented by the sample data 130. The first noise is, for example, a random number.

[0031] In the example in Figure 1, for the sake of simplicity, we assume that the first noise is 0 for the coordinates in the Y-axis direction and the coordinates in the Z-axis direction.

[0032] In the example in Figure 1, with respect to sample data 131, the first noise for the X-axis coordinate of atom 141 in system 150 is specifically 0. Also, with respect to sample data 131, the first noise for the X-axis coordinate of atom 142 in system 150 is specifically the value corresponding to vector 151.

[0033] Furthermore, in the example shown in Figure 1, with respect to the sample data 132, the first noise with respect to the X-axis coordinate of atom 141 in system 160 is specifically the value corresponding to vector 161. Also, with respect to the sample data 132, the first noise with respect to the X-axis coordinate of atom 142 in system 160 is specifically the value corresponding to vector 162.

[0034] Furthermore, in the example shown in Figure 1, with respect to sample data 133, the first noise for the X-axis coordinate of atom 141 in system 170 is specifically the value corresponding to vector 171. Also, with respect to sample data 133, the first noise for the X-axis coordinate of atom 142 in system 170 is specifically 0.

[0035] (1-3) For each sample data 130, the information processing apparatus 100 calculates the average of the first noise generated for the coordinates of each atom in the system represented by the sample data 130 for each dimension. The average is calculated, for example, for each sample data 130 and for each dimension.

[0036] In the example of FIG. 1, for the sake of simplifying the explanation, for the coordinates in the Y-axis direction and the coordinates in the Z-axis direction, the average of the first noise is 0.

[0037] In the example of FIG. 1, for the sample data 131, the average of the first noise for the coordinates in the X-axis direction in the system 150 is, for example, half of the sum of 0 and the value corresponding to the vector 151, and thus specifically, it is the value corresponding to half of the vector 151. Also, in the example of FIG. 1, for the sample data 132, the average of the first noise for the coordinates in the X-axis direction in the system 160 is, for example, half of the sum of the values corresponding to the vectors 161 and 162, and thus specifically, it is 0 where the vectors 161 and 162 cancel each other out. Also, in the example of FIG. 1, for the sample data 133, the average of the first noise for the coordinates in the X-axis direction in the system 170 is, for example, half of the sum of the value corresponding to the vector 171 and 0, and thus specifically, it is the value corresponding to half of the vector 171.

[0038] (1-4) For each sample data 130, the information processing apparatus 100 calculates the second noise by removing the average calculated for each dimension from the first noise generated for the coordinates of each atom in each dimension of the system represented by the sample data 130.

[0039] In the example of FIG. 1, for the sake of simplifying the explanation, for the coordinates in the Y-axis direction and the coordinates in the Z-axis direction, the second noise is 0.

[0040] In the example of FIG. 1, for the sample data 131, the second noise for the coordinates in the X-axis direction of the atom 141 in the system 150 is, for example, the value corresponding to the vector 152. Also, for the sample data 131, the second noise for the coordinates in the X-axis direction of the atom 142 in the system 150 is, for example, the value corresponding to the vector 153.

[0041] Also, in the example of FIG. 1, with respect to the sample data 132, the second noise with respect to the X-axis coordinate of the atom 141 in the system 160, for example, becomes a value corresponding to the vector 163. Also, with respect to the sample data 132, the second noise with respect to the X-axis coordinate of the atom 142 in the system 160, for example, becomes a value corresponding to the vector 164.

[0042] Also, in the example of FIG. 1, with respect to the sample data 133, the second noise with respect to the X-axis coordinate of the atom 141 in the system 170, for example, becomes a value corresponding to the vector 172. Also, with respect to the sample data 133, the second noise with respect to the X-axis coordinate of the atom 142 in the system 170, for example, becomes a value corresponding to the vector 173.

[0043] (1-5) The information processing device 100 adds the second noise calculated for each coordinate dimension of each atom in the system represented by the sample data 130 to the respective coordinate dimensions of the sample data 130. In the example of FIG. 1, the X-axis coordinates of the atoms 141 and 142 may change by the addition.

[0044] In the example of FIG. 1, by the addition, with respect to the sample data 131, the X-axis coordinate of the atom 141 in the system 150 changes, for example, in the direction of the vector 152. Also, by the addition, with respect to the sample data 131, the X-axis coordinate of the atom 142 in the system 150 changes, for example, in the direction of the vector 153.

[0045] Also, in the example of FIG. 1, with respect to the sample data 132, the X-axis coordinate of the atom 141 in the system 160 changes, for example, in the direction of the vector 163. Also, with respect to the sample data 132, the X-axis coordinate of the atom 142 in the system 160 changes, for example, in the direction of the vector 164.

[0046] Also, in the example of FIG. 1, with respect to the sample data 133, the X-axis coordinate of the atom 141 in the system 170 changes, for example, in the direction of the vector 172. Also, with respect to the sample data 133, the X-axis coordinate of the atom 142 in the system 170 changes, for example, in the direction of the vector 173.

[0047] (1-6) The information processing device 100 generates training data that associates each sample data 130 after adding the second noise with the second noise corresponding to that sample data 130. Here, the second noise is treated as the correct label for the model 110. For example, the force acting on an atom is the force that causes the atom to return to a stable coordinate. On the other hand, the second noise is thought to act to move the atom's coordinate to an unstable coordinate. For this reason, the second noise can be used as an indicator that substitutes for the force acting on an atom. Accordingly, when training the model 110, the model 110 will be trained in the direction of removing the second noise from the atom's coordinate.

[0048] This allows the information processing device 100 to prepare multiple appropriate training data for use in self-supervised learning. For example, in conventional methods, for each sample data 130, it is thought that a first noise calculated for each dimensional coordinate of each atom in the system represented by the sample data 130 is added to the coordinate of each dimensional. Therefore, in conventional methods, it is thought that training data is prepared that associates each sample data 130 after the first noise has been added with the first noise corresponding to that sample data 130.

[0049] Therefore, conventional methods end up with multiple inconsistent training datasets, each associated with a different first noise as the correct label, even though they contain sample data 130 with common relative positional relationships between atoms. As a result, conventional methods have the problem of being difficult to properly train the model 110 in the direction of removing the first noise when performing self-supervised learning.

[0050] In contrast, the information processing device 100 can prepare multiple mutually consistent training data sets, for example, including sample data 130 with common relative positional relationships between atoms, and to which a common second noise is associated as the correct label. Therefore, when performing self-supervised learning, for example, the information processing device 100 can make it easier to appropriately train the model 110 in a direction that removes the second noise.

[0051] (1-7) The information processing device 100 performs self-supervised learning on the model 110 based on the generated training data. The information processing device 100 calculates a loss 180 based on the training data using the model 110, for example. The information processing device 100 trains the model 110 by performing self-supervised learning on the model 110 based on the calculated loss 180.

[0052] As a result, the information processing device 100 can efficiently train the model 110 and reduce the processing time required for self-supervised learning. The information processing device 100 can improve the convergence speed of self-supervised learning. Specifically, the information processing device 100 can efficiently train MLIAP, which utilizes graphs representing the relative positional relationships between atoms.

[0053] The information processing device 100 can, for example, efficiently improve the inference accuracy of the model 110. Specifically, the information processing device 100 can improve the inference accuracy of the model 110 from sample data 130 that do not have pre-labeled data. In other words, the information processing device 100 can improve the inference accuracy of the model 110 even if sample data with pre-associated labels is not available. Therefore, the information processing device 100 can reduce the time, cost, and human costs associated with preparing labels in advance.

[0054] Here, we have described the case where the functions of the information processing device 100 are realized by a single computer, but this is not the only case. For example, the functions of the information processing device 100 may be realized through the collaboration of multiple computers. For example, the functions of the information processing device 100 may be realized on the cloud.

[0055] Here, we have described a case in which the information processing device 100 generates first noise for each sample data 130, for each dimension of the atom in the system represented by the sample data 130, but it is not limited to this. For example, another computer may generate first noise for each sample data 130, for each dimension of the atom in the system represented by the sample data 130. The information processing device 100 may also acquire first noise from the other computer.

[0056] (An example of an information processing system 200) Next, an example of an information processing system 200 that applies the information processing device 100 shown in Figure 1 will be described using Figure 2.

[0057] Figure 2 is an explanatory diagram showing an example of an information processing system 200. In Figure 2, the information processing system 200 includes an information processing device 100 and one or more client devices 201.

[0058] In the information processing system 200, the information processing device 100 and the client device 201 are connected via a wired or wireless network 210. The network 210 may be, for example, a LAN (Local Area Network), a WAN (Wide Area Network), or the Internet.

[0059] The information processing device 100 is a computer for supporting self-supervised learning. The information processing device 100 stores a model. The model is, for example, MLIAP. The information processing device 100 stores a dataset of sample data, which includes samples representing systems containing atoms. A sample is, for example, an example of input data for the model. The information processing device 100 stores a training program for training the model using the sample data.

[0060] The information processing device 100 receives a training request that requests self-supervised learning. The information processing device 100 receives a training request, for example, by receiving the training request from the client device 201. In response to the training request, the information processing device 100 uses a training program to train a model by performing self-supervised learning based on sample data, as in Figure 1. After training the model, the information processing device 100 may output the trained model. The information processing device 100 may, for example, provide the trained model to the client device 201.

[0061] The information processing device 100 may receive an inference request that requests the generation of labels corresponding to input data. The inference request includes, for example, input data. The information processing device 100 receives the inference request, for example, by receiving it from the client device 201. In response to the inference request, the information processing device 100 generates labels corresponding to the input data using a trained model. The information processing device 100 sends the generated labels to the client device 201. The information processing device 100 is, for example, a server or a PC.

[0062] The client device 201 is a computer that sends training requests or inference requests, etc., to the information processing device 100. For example, the client device 201 generates a training request in response to user input and sends it to the information processing device 100. After sending the training request, the client device 201 may receive a trained model from the information processing device 100. This allows the client device 201 to make the trained model available on its own device.

[0063] The client device 201 acquires input data in response to user input, for example. The client device 201 generates an inference request including the acquired input data in response to user input and transmits it to the information processing device 100. After transmitting the inference request, the client device 201 may receive a label corresponding to the input data from the information processing device 100. The client device 201 outputs the label corresponding to the input data so that the user can refer to it. This allows the client device 201 to make the label corresponding to the input data available to the user. The client device 201 may be, for example, a PC, a tablet terminal, a smartphone, or a wearable device.

[0064] Here, we have described a case where the information processing device 100 is a different device from the client device 201, but this is not limited to this case. For example, the information processing device 100 may have the functionality of a client device 201 and may operate as a client device 201. In this case, the information processing system 200 does not need to include a client device 201.

[0065] (Example of hardware configuration of the information processing device 100) Next, an example of the hardware configuration of the information processing device 100 will be described using Figure 3.

[0066] Figure 3 is a block diagram showing an example of the hardware configuration of the information processing device 100. In Figure 3, the information processing device 100 includes a CPU (Central Processing Unit) 301, a memory 302, and a network interface 303. The information processing device 100 also includes a recording medium interface 304, a recording medium 305, a display 306, and an input device 307. Each component is connected by a bus 300.

[0067] Here, the CPU 301 is responsible for the overall control of the information processing device 100. The memory 302 includes, for example, ROM (Read Only Memory), RAM (Random Access Memory), and flash ROM. Specifically, for example, flash ROM and ROM store various programs, and RAM is used as the work area for the CPU 301. Programs stored in memory 302 are loaded into the CPU 301, causing the CPU 301 to execute the coded processes.

[0068] The network interface 303 is connected to the network 210 via a communication line, and then connected to other computers via the network 210. The network interface 303 manages the internal interface with the network 210 and controls the input and output of data from other computers. The network interface 303 is, for example, a modem or a LAN adapter.

[0069] The recording medium interface 304 controls the reading and writing of data to the recording medium 305 according to the control of the CPU 301. The recording medium interface 304 is, for example, a disk drive, an SSD (Solid State Drive), or a USB (Universal Serial Bus) port. The recording medium 305 is a non-volatile memory that stores the data written under the control of the recording medium interface 304. The recording medium 305 is, for example, a disk, semiconductor memory, or USB memory. The recording medium 305 may be detachable from the information processing device 100.

[0070] Display 306 displays data such as cursors, icons, toolboxes, documents, images, or functional information. Display 306 is, for example, a CRT (Cathode Ray Tube), a liquid crystal display, or an organic EL (Electroluminescence) display. Input device 307 has keys for inputting characters, numbers, or various instructions, and performs data input. Input device 307 is, for example, a keyboard or a mouse. Input device 307 may also be, for example, a touch panel input pad or a numeric keypad.

[0071] The information processing device 100 may have, in addition to the components described above, a camera, for example. Furthermore, the information processing device 100 may have, in addition to the components described above, a printer, scanner, microphone, or speaker, for example. Also, the information processing device 100 may have, for example, multiple recording medium interfaces 304 and recording media 305. Furthermore, the information processing device 100 does not necessarily have, for example, a display 306 or an input device 307. Also, the information processing device 100 does not necessarily have, for example, recording medium interfaces 304 and recording media 305.

[0072] (Example of hardware configuration of client device 201) The example of hardware configuration of client device 201 is the same as the example of hardware configuration of information processing device 100 shown in Figure 3, so the explanation is omitted.

[0073] (Example of Functional Configuration of Information Processing Device 100) Next, an example of the functional configuration of the information processing device 100 will be described using Figure 4.

[0074] Figure 4 is a block diagram showing an example of the functional configuration of the information processing device 100. The information processing device 100 includes a storage unit 400, an acquisition unit 401, a first generation unit 402, a second generation unit 403, a third generation unit 404, a training unit 405, and an output unit 406.

[0075] The storage unit 400 is implemented by a storage area such as the memory 302 or recording medium 305 shown in Figure 3. The following description will focus on the case where the storage unit 400 is included in the information processing device 100, but is not limited to this case. For example, the storage unit 400 may be included in a device different from the information processing device 100, and the contents of the storage unit 400 may be accessible from the information processing device 100.

[0076] The acquisition unit 401 to the output unit 406 function as an example of a control unit. Specifically, the acquisition unit 401 to the output unit 406 realize their functions, for example, by causing the CPU 301 to execute a program stored in a storage area such as the memory 302 or recording medium 305 shown in Figure 3, or by using the network interface 303. The processing results of each functional unit are stored in a storage area such as the memory 302 or recording medium 305 shown in Figure 3.

[0077] The memory unit 400 stores various information that is referenced or updated in the processing of each functional unit. For example, the memory unit 400 stores a model. Specifically, the memory unit 400 stores a calculation formula that serves as the model and the values ​​of the parameters included in the calculation formula. The model generates output data that represents at least the forces acting on the atoms, in response to input data that represents a system including atoms. The model is subject to training by self-supervised learning. The model is, for example, MLIAP.

[0078] The input data represents, for example, the structure of a system containing atoms. The input data represents, for example, the coordinates of each atom in the system in each dimension. Specifically, the input data represents the positional relationships between atoms in a system containing atoms. The dimensions are, for example, the X-axis, Y-axis, and Z-axis. The output data may further represent the energy of the system. The output data is a label. The model is, for example, pre-set by the user. The model may be acquired, for example, by the acquisition unit 401.

[0079] The storage unit 400 stores, for example, a dataset. The dataset is a collection of sample data. The sample data includes samples representing systems containing atoms. The sample data does not have, for example, correct labels representing the forces acting on the atoms of the system, which are not pre-prepared by the user. The dataset is pre-set by the user, for example. The dataset may be acquired, for example, by the acquisition unit 401.

[0080] The acquisition unit 401 acquires various types of information used in the processing of each functional unit. The acquisition unit 401 stores the acquired information in the storage unit 400 or outputs it to each functional unit. The acquisition unit 401 may also output the information stored in the storage unit 400 to each functional unit. The acquisition unit 401 acquires various types of information, for example, based on user input. The acquisition unit 401 may also receive various types of information from a device other than the information processing device 100, for example.

[0081] The acquisition unit 401 acquires, for example, a training request that requests training a model. The training request may include, for example, a model. The training request may include, for example, a dataset. Specifically, the acquisition unit 401 acquires a training request by accepting input of a training request. Specifically, the acquisition unit 401 may acquire a training request by receiving a training request from another computer. The other computer is, for example, a client device 201.

[0082] The acquisition unit 401 acquires a model, for example. Specifically, the acquisition unit 401 acquires a model by extracting it from a training request. Specifically, the acquisition unit 401 may acquire a model by accepting a model input. Specifically, the acquisition unit 401 may acquire a model by receiving a model from another computer. The other computer is, for example, a client device 201.

[0083] The acquisition unit 401 acquires a dataset, for example. Specifically, the acquisition unit 401 acquires a dataset by extracting it from a training request. Specifically, the acquisition unit 401 may acquire a dataset by accepting a dataset as input. Specifically, the acquisition unit 401 may acquire a dataset by receiving a dataset from another computer. The other computer is, for example, a client device 201.

[0084] The acquisition unit 401 may receive a start trigger to initiate processing in any of the functional units. A start trigger may be, for example, a predetermined operation input by a user. A start trigger may also be, for example, the reception of predetermined information from another computer. A start trigger may also be, for example, the output of predetermined information by any of the functional units. The acquisition unit 401 accepts, for example, the acquisition of a training request as a start trigger to initiate processing in the first generation unit 402, the second generation unit 403, the third generation unit 404, and the training unit 405.

[0085] The first generation unit 402 generates first noise for each dimension coordinate of each atom in the system represented by each of the multiple sample data included in the dataset acquired by the acquisition unit 401. The first noise is, for example, a random number. The first noise may be generated according to a predetermined rule. For example, the first generation unit 402 generates first noise, which is a random number, for each dimension coordinate of each atom in the system represented by each sample data. This allows the first generation unit 402 to generate first noise, which is the source for generating labels to be associated with the sample data.

[0086] The second generation unit 403 calculates the average of the first noise generated for each dimension of each sample data in the dataset acquired by the acquisition unit 401, for each of the multiple sample data in the dataset. The average is calculated, for example, for each sample data and for each dimension. The second generation unit 403 generates a second noise by removing the average calculated for each dimension from the first noise generated for each dimension of each atom in the system represented by the sample data. In this way, the second generation unit 403 can prepare a second noise that can be added to each dimension of each atom in the system represented by each sample data without causing inconsistencies between the sample data.

[0087] The third generation unit 404 adds the second noise generated for each dimension of the coordinates of each atom in the system represented by the sample data to the coordinates of each dimension of the sample data included in the dataset acquired by the acquisition unit 401. The third generation unit 404 generates training data that associates each sample data after the addition of the second noise with the second noise corresponding to that sample data. Here, the second noise is treated as the correct label for the sample data with respect to the model. As a result, the third generation unit 404 can prepare multiple training data sets containing each of the corresponding sample data, enabling efficient model training.

[0088] The training unit 405 performs self-supervised learning on a model that generates output data representing the force acting on each atom in a system, based on input data representing the coordinates of each dimension of each atom, based on the training data generated by the third generation unit 404. The training unit 405 calculates a loss on the model using a loss function, for example, based on the difference between the output data generated by the model when input sample data included in the training data and the second noise corresponding to the sample data included in the training data. The training unit 405 updates the model parameters in a way that reduces the calculated loss, for example.

[0089] This allows the training unit 405 to use sample data for which labels have not been pre-prepared by the user to train the model. The training unit 405 can efficiently train the model based on multiple training datasets, each containing matching sample data. The training unit 405 can reduce the processing time required when performing self-supervised learning.

[0090] The output unit 406 outputs the processing result of at least one of the functional units. The output format can be, for example, display on a screen, print to a printer, transmit to an external device via the network interface 303, or store in a storage area such as the memory 302 or recording medium 305. This allows the output unit 406 to notify the user of the processing result of at least one of the functional units, thereby improving the usability of the information processing device 100.

[0091] The output unit 406 outputs, for example, the model trained by the training unit 405. Specifically, the output unit 406 outputs the model so that it can be viewed by a user. Specifically, the output unit 406 may also transmit the model to another computer, such as a client device 201. This allows the output unit 406 to make the model available externally.

[0092] (Example of operation of the information processing device 100) Next, an example of operation of the information processing device 100 will be explained using Figures 5 to 13. First, using Figure 5, the process by which the information processing device 100 trains the model 500 through self-supervised learning will be explained.

[0093] In the following explanation, for convenience, when a subscript is added to a specific character, it may be written as "(specific character) _(subscript)". Also, for convenience, when a superscript is added to a specific character, it may be written as "(specific character) ^(superscript)". Also, for convenience, when a tilde (~) is added directly above a specific character, it may be written as "(specific character) ~". Also, for convenience, when a hyphen (-) is added directly above a specific character, it may be written as "(specific character) - It is sometimes written as "".

[0094] Figure 5 is an explanatory diagram showing the flow of training model 500. In Figure 5, the information processing device 100 stores model 500. Model 500 is, for example, the MLIAP620, which will be described later in Figure 6. Model 500 has parameters 501. The information processing device 100 stores, for example, the calculation formula that makes up model 500 and the values ​​of the parameters 501 included in the calculation formula. Training is updating model 500. Specifically, training is updating the parameters 501.

[0095] The information processing device 100 stores dataset 510 and dataset 520. Dataset 510 is a data set that collects multiple first data sets that represent a relatively broad range of knowledge. The multiple first data sets are, for example, a data set that collects multiple types of first data sets, regardless of the actual use of model 500. The multiple first data sets include, for example, unlabeled data. The unlabeled data is sample data that represents a sample of data to be input to model 500. The unlabeled data does not contain the correct labels.

[0096] Dataset 520 is a collection of secondary data sets that represent relatively narrow knowledge. These secondary data sets are, for example, collections of secondary data of types that match the actual elements of model 500. These secondary data sets include, for example, labeled data. Labeled data is training data that associates a sample of data to be input into model 500 with the correct label corresponding to that sample.

[0097] The information processing device 100 trains the model 500 by performing pre-training on the model 500 based on the dataset 510. Specifically, the training involves optimizing the parameters 501 based on the loss 511. Subsequently, the information processing device 100 further trains the model 500 by performing fine-tuning on the pre-trained model 500 based on the dataset 520. Specifically, the training involves optimizing the parameters 501 based on the loss 521. Next, we will move on to explaining Figures 6 and 7 and describe an example of MLIAP620.

[0098] Figures 6 and 7 are explanatory diagrams showing an example of MLIAP620. In Figure 6, MLIAP620 has a parameter 621. Parameter 621 corresponds to parameter 501. MLIAP620 has the function of generating a label 630 corresponding to a sample 610 when a sample 610 is input. In the example in Figure 6, the sample 610 represents the position of an atom forming the system and the element of that atom. The label 630 represents the energy of the system and the force acting on the atom. Next, we will move on to the explanation of Figure 7.

[0099] As shown in Figure 7, the MLIAP620 specifically includes an encoder 710, an energy head 720, and a force head 730. The encoder 710 has parameter 711. The energy head 720 has parameter 721. The force head 730 has parameter 731.

[0100] The encoder 710 refers to sample 610 and calculates the feature quantity of atom 0 based on atom 0, information about the area around atom 0, and parameter 711. The encoder 710 refers to sample 610 and calculates the feature quantity of atom 1 based on atom 1, information about the area around atom 1, and parameter 711. The energy head 720 calculates the energy of the system based on the feature quantity of atom 0, the feature quantity of atom 1, and parameter 721.

[0101] The force head 730 calculates the force acting on atom 0 and the force acting on atom 1 based on the feature quantities of atom 0, the feature quantities of atom 1, and the parameter 731. The MLIAP 620 generates labels 630 based on the energy of the system, the force acting on atom 0, and the force acting on atom 1. The force acting on the atoms may be calculated, for example, using the energy head 720.

[0102] Next, using Figure 8, we will explain an example in which the information processing device 100 performs self-supervised learning on the MLIAP 620 during pre-training.

[0103] Figure 8 is an explanatory diagram illustrating an example of implementing self-supervised learning. In Figure 8, the dataset 510 specifically includes D first data points.

[0104] (8-1) The information processing device 100 randomly selects first data 810 from the dataset 510, which includes a sample representing a system containing N atoms, to be used when performing self-supervised learning. The first data 810 represents, for example, the coordinate x ∈ R^(N × 3) of an atom in three-dimensional space. The first data 810 is unlabeled data and is sample data.

[0105] The first data 810 relates to an H2 molecule. An H2 molecule is formed from two H atoms. In the first data 810, indices 0 and 1 are assigned to each atom that makes up the H2 molecule. The first data 810 represents the coordinates of each atom that makes up the H2 molecule. The coordinates include, for example, x-axis coordinate values, y-axis coordinate values, and z-axis coordinate values.

[0106] (8-2) The information processing device 100 generates a random number Δx ∈ R^(N × 3) representing a first noise corresponding to the coordinates of each atom for the first data 810. The first noise is a three-dimensional vector. The first noise includes, for example, an x-axis component, a y-axis component, and a z-axis component.

[0107] The information processing device 100 generates a random number Δx for the first data 810 that includes a component following a normal distribution N(0, σ^2) and represents a first noise corresponding to the coordinates of each atom. σ^2 is the variance. The information processing device 100 may also generate a random number Δx for the first data 810 that includes a component 0 in the range of probability 0 < α < 1, and a component following a normal distribution N(0, σ^2) outside the range of probability 0 < α < 1, representing a first noise corresponding to the coordinates of each atom.

[0108] In the example of FIG. 8, it is assumed that the information processing apparatus 100 specifically generates a random number Δx shown in Table 820 for the first data 810. Specifically, the X-axis component related to the atom at index 0 is 0.1. Also, specifically, the Y-axis component related to the atom at index 0 is -0.2. Also, specifically, the Z-axis component related to the atom at index 0 is 0.3. Also, specifically, the X-axis component related to the atom at index 1 is -0.4. Also, the Y-axis component related to the atom at index 1 is 0.5. Also, the Z-axis component related to the atom at index 1 is -0.6.

[0109] (8-3) The information processing apparatus 100 calculates the average Δx of the first noise in each dimension for the entire N atoms with respect to the first data 810 according to the following formula (1). - ∈R^(N×3). In the example of FIG. 8, it is assumed that the information processing apparatus 100 specifically generates the average Δx - for the first data 810. Specifically, the average of the X-axis components is (0.1 + (-0.4)) / 2 = -0.15. Also, specifically, the average of the Y-axis components is ((-0.2) + 0.5) / 2 = 0.15. Also, specifically, the average of the Z-axis components is (0.3 + (-0.6)) = -0.15.

[0110]

[0111] (8-4) The information processing apparatus 100 calculates (random number Δx) - (average Δx - ) as the second noise to be added to the coordinates of each atom with respect to the first data 810. In the example of FIG. 8, it is assumed that the information processing apparatus 100 calculates the second noise shown in Table 830 for the first data 810.

[0112] (8-5) The information processing apparatus 100 associates and represents the teacher data x - = (x + Δx - Δx - ) obtained by adding the second noise to the coordinate x of the first data 810 with the second noise (Δx - Δx - ) which is the correct label. (x + Δx - Δx - ), Δx - Δx - ). (x + Δx - Δx -The data is shown in Table 840. The information processing device 100 trains the MLIAP 620 based on the generated training data. This allows the information processing device 100 to efficiently perform self-supervised learning.

[0113] Here, we have described the case where the correct label includes at least a second noise, but it is not limited to this. For example, the correct label may also include a feature that substitutes for the energy of the system containing the molecules. Such a feature could be, for example, a truth value indicating whether or not the system contains a particular element.

[0114] Next, using Figures 9 to 11, we will explain an example of the effects obtained by the proposed method using the information processing device 100 by comparing the cases in which the model 500 is trained using the proposed method using the information processing device 100 and the conventional method. Here, the conventional method trains the model 500 based on training data that, for example, associates (x + Δx), which is obtained by adding a first noise to the coordinate x of the first data 810, with the first noise Δx which is considered the correct label.

[0115] Figures 9 to 11 are explanatory diagrams illustrating an example of the effect. First, let's move on to the explanation of Figure 9. Specifically, Figure 9 shows the case where model 500 is trained using a conventional method.

[0116] In conventional methods, specifically, training data is prepared for the first data 810 without considering the average of the first noise in each dimension. Therefore, conventional methods may end up with multiple inconsistent training data sets, where different first noises are associated as the correct labels, even though the relative positions between atoms are the same.

[0117] Specifically, in the example in Figure 9, we assume that there are first data points 811 to 813. First data point 811 relates to system 910 containing two atoms 901 and 902. First data point 812 relates to system 920 containing two atoms 901 and 902. First data point 813 relates to system 930 containing two atoms 901 and 902. Atoms 901 and 902 are H atoms.

[0118] In the conventional method, training data is generated by adding the noise component represented by vector 911 to the coordinates of atom 902 in the first data 811. Similarly, in the conventional method, training data is generated by adding the noise component represented by vector 921 to the coordinates of atom 901 in the first data 812, and the noise component represented by vector 922 to the coordinates of atom 902 in the first data 812. Similarly, in the conventional method, training data is generated by adding the noise component represented by vector 931 to the coordinates of atom 901 in the first data 813.

[0119] The training data is thought to represent how the coordinates of each atom in the target system should be preferably corrected during inference by model 500. Training is thought to correspond to updating the parameters 501 of model 500 so that the coordinates of each atom in the target system are appropriately corrected according to the meaning of the training data.

[0120] Here, we consider the case where the model 500 is trained based on training data using a conventional method. In this case, it is preferable to remove noise components from the training data corresponding to the first data 811 during inference, so that the coordinates of atom 902 are represented by vector 941. On the other hand, it is preferable to remove noise components from the training data corresponding to the first data 812 during inference, so that the coordinates of atom 901 are represented by vector 951, and the coordinates of atom 902 are represented by vector 952.

[0121] Furthermore, the training data corresponding to the first data 813 preferably has noise components removed during inference so that the coordinates of atom 901 are represented by vector 961. In this way, the conventional method may result in inconsistent meanings between multiple training data. For this reason, the conventional method may find it difficult to learn how to handle the coordinates of atoms 901 and 902 based on each training data, and may not be able to properly train the model 500. Next, we will move on to the explanation of Figure 10. Figure 10 specifically shows the case where the model 500 is trained using the proposed method.

[0122] In the proposed method, specifically, training data is prepared for the first data 810 (811-813) by considering the average of the first noise in each dimension. Therefore, the proposed method can prepare multiple training data so that a common second noise can be associated as the correct label for multiple training data where the relative positional relationships between atoms are common.

[0123] Specifically, in the example of Figure 10, we assume that there are first data 811 to 813, similar to the example of Figure 9. First data 811 relates to system 910 containing two atoms 901 and 902. First data 812 relates to system 920 containing two atoms 901 and 902. First data 813 relates to system 930 containing two atoms 901 and 902.

[0124] Specifically, the information processing device 100 can generate a second noise for atoms 901 and 902 of the first data 811, taking into account the average of the first noise in each dimension. Specifically, the information processing device 100 can generate training data by adding the noise component represented by vector 1011 to the coordinates of atom 901 of the first data 811, and by adding the noise component represented by vector 1012 to the coordinates of atom 902 of the first data 811.

[0125] Similarly, the information processing device 100 can specifically generate a second noise for atoms 901 and 902 of the first data 812, taking into account the average of the first noise in each dimension. Specifically, the information processing device 100 can generate training data by adding the noise component represented by vector 1021 to the coordinates of atom 901 of the first data 812, and by adding the noise component represented by vector 1022 to the coordinates of atom 902 of the first data 812.

[0126] Similarly, the information processing device 100 can specifically generate a second noise for atoms 901 and 902 of the first data 813, taking into account the average of the first noise in each dimension. Specifically, the information processing device 100 can generate training data by adding the noise component represented by vector 1031 to the coordinates of atom 901 of the first data 813, and by adding the noise component represented by vector 1032 to the coordinates of atom 902 of the first data 813.

[0127] Here, it is preferable that, during inference, the training data corresponding to the first data 811 has noise components removed from the coordinates of atom 901 so as represented by vector 1041, and from the coordinates of atom 902 so as represented by vector 1042.

[0128] Similarly, it is preferable that, during inference, the training data corresponding to the first data 812 has noise components removed from the coordinates of atom 901 so as represented by vector 1051, and from the coordinates of atom 902 so as represented by vector 1052.

[0129] Similarly, it is preferable that, during inference, the training data corresponding to the first data 813 has noise components removed from the coordinates of atom 901 so as represented by vector 1061, and from the coordinates of atom 902 so as represented by vector 1062.

[0130] In this way, the information processing device 100 can reconcile the meanings of multiple training data sets that share a common relative positional relationship between atoms, and can prepare multiple appropriate training data sets for use in self-supervised learning. Therefore, the information processing device 100 can appropriately learn how to handle the coordinates of atoms 901 and 902 based on each training data set, and can appropriately train the model 500. Furthermore, the information processing device 100 can reduce the processing time required when training the model 500. Next, we will move on to the explanation of Figure 11.

[0131] In Figure 11, Model 500 is a PaiNN model. In the example in Figure 11, the conventional method specifically trains the PaiNN model based on the PubChem database, which contains data representing the three-dimensional structures of compounds treated as unlabeled data. The proposed method specifically trains the PaiNN model based on the PubChem database, which contains data representing the three-dimensional structures of compounds treated as unlabeled data.

[0132] In Figure 11, 95% of the data representing the three-dimensional structures of compounds with compound IDs between 1 and 1,000,000 in the PubChem database are randomly selected and treated as unlabeled training data. The remaining 5% of the data representing the three-dimensional structures of compounds with compound IDs between 1 and 1,000,000 in the PubChem database are treated as unlabeled evaluation data.

[0133] In the example shown in Figure 11, the information processing device 100 generates a first noise based on a normal distribution with a mean of 0 and a variance of 0.1 for each atom in unlabeled data. The information processing device 100 uses L2 MAE (Mean Average Error) × number of atoms as its loss function. The information processing device 100 sets the number of Epochs to 2.

[0134] Graph 1100 shows the change in MAE of the noise when the PaiNN model is trained using the proposed method, and the change in MAE of the noise when the PaiNN model is evaluated using the proposed method. Graph 1100 also shows the change in MAE of the noise when the PaiNN model is trained using the conventional method, and the change in MAE of the noise when the PaiNN model is evaluated using the conventional method.

[0135] As shown in Graph 1100, the number of steps required for the noise MAE to reach 0.03 Å, depending on the unlabeled data used for evaluation, is, for example, 15,000 to 20,000 in the conventional method. On the other hand, as shown in Graph 1100, the number of steps required for the noise MAE to reach 0.03 Å, depending on the unlabeled data used for evaluation, is, for example, 5,000 to 10,000 in the proposed method. Therefore, the proposed method can accelerate the convergence speed of the noise MAE by 2.0 times compared to the conventional method.

[0136] Furthermore, the information processing device 100 can generate a second noise from a first noise through simple calculations. Therefore, the information processing device 100 can suppress the increase in processing load caused by generating a second noise from a first noise to a relatively small amount.

[0137] Here, we have described a case in which the information processing device 100 performs self-supervised learning on the MLIAP 620 during pre-training, but it is not limited to this. For example, the information processing device 100 may similarly perform self-supervised learning on the MLIAP 620 during fine-tuning.

[0138] (An example of applying the information processing device 100) The information processing device 100 can be applied to fields such as drug discovery or materials development. Specifically, the information processing device 100 can efficiently train MLIAP, which is used for first-principles calculations or molecular dynamics. Therefore, the information processing device 100 can make it easier to analyze systems such as molecules by performing first-principles calculations or molecular dynamics in fields such as drug discovery or materials development.

[0139] (Overall Processing Procedure) Next, an example of the overall processing procedure executed by the information processing device 100 will be described using Figure 12. The overall processing is realized, for example, by the CPU 301 shown in Figure 3, storage areas such as memory 302 and recording medium 305, and network I / F 303.

[0140] Figure 12 is a flowchart showing an example of the overall processing procedure. In Figure 12, the information processing device 100 initializes the model parameters W (step S1201).

[0141] The information processing device 100 obtains Z, G, and y by performing individual processing described later in Figure 13 (step S1202). The information processing device 100 uses the model to calculate the inferred label value y~ (step S1203).

[0142] The information processing device 100 calculates the label loss l = Loss(y, y~) using the loss function Loss() (step S1204). The information processing device 100 updates W based on l (step S1205).

[0143] The information processing device 100 determines whether or not the termination conditions are met (step S1206). The termination conditions are, for example, that W has been updated a predetermined number of times. The predetermined number of times is, for example, set in advance by the user. Another termination condition is that the loss is below a threshold. The threshold is, for example, set in advance by the user.

[0144] If the termination condition is not met (step S1206: No), the information processing device 100 returns to the process in step S1202. On the other hand, if the termination condition is met (step S1206: Yes), the information processing device 100 outputs W (step S1207) and terminates the entire process.

[0145] (Individual Processing Procedure) Next, an example of an individual processing procedure executed by the information processing device 100 will be described using Figure 13. Individual processing is realized, for example, by the CPU 301 shown in Figure 3, a storage area such as the memory 302 and recording medium 305, and the network I / F 303.

[0146] Figure 13 is a flowchart showing an example of an individual processing procedure. In Figure 13, the information processing device 100 sets Z = Φ, x = Φ, y = Φ, and i = 0 (step S1301). The information processing device 100 then sets i = i + 1 (step S1302).

[0147] The information processing device 100 obtains a sample {Z_i, x_i} relating to a system containing N_i atoms (step S1303). Z_i is a tensor of atomic numbers. x_i is the three-dimensional coordinate of the atom. The information processing device 100 randomly generates Δx_i (step S1304).

[0148] The information processing device 100 calculates Δx based on Δx_i. - The information processing device 100 calculates _i (step S1305). Δx ~ _i = x_i + Δx_i - Δx - The information processing device 100 calculates _i (step S1306). The information processing device 100 calculates y_i = Δx_i - Δx - Calculate _i (step S1307).

[0149] The information processing device 100 sets Z ← concat(Z, Z_i) (step S1308). concat(x, y) represents the operation of joining tensor x and tensor y at the first dimension. concat(Φ, y) = y. The information processing device 100 sets x ← concat(x, x_i) (step S1309). The information processing device 100 sets y ← concat(y, y_i) (step S1310).

[0150] The information processing device 100 determines whether i > S (step S1311). S is, for example, set in advance by the user. If i ≤ S and not i > S (step S1311: No), the information processing device 100 returns to the process in step S1302. On the other hand, if i > S (step S1311: Yes), the information processing device 100 generates a graph G based on x (step S1312) and terminates the individual processing.

[0151] As explained above, the information processing device 100 can generate a first noise for each dimension of each atom in the system represented by each of several sample data representing the coordinates of each dimension of each atom in the system. The information processing device 100 can calculate the average of the first noise generated for each dimension of each sample data representing the coordinates of each atom in the system represented by that sample data. The information processing device 100 can generate a second noise for each sample data by removing the average calculated for each dimension from the first noise generated for each dimension of each atom in the system represented by that sample data. The information processing device 100 can add the second noise generated for each dimension of each sample data representing the coordinates of each atom in the system represented by that sample data to the coordinates of each dimension. The information processing device 100 can prepare training data that associates each sample data after the addition with the second noise corresponding to that sample data. According to the information processing device 100, self-supervised learning can be performed on a model that generates output data representing the force acting on each atom in a system, based on input data representing the coordinates of each dimension of each atom, according to the prepared training data. This allows the information processing device 100 to prepare multiple training data sets that are consistent with each other, and to train the model efficiently. The information processing device 100 can also reduce the processing time required when performing self-supervised learning.

[0152] According to the information processing device 100, a loss related to the model can be calculated using a loss function based on the difference between the output data generated by the model, which is input to sample data included in the training data, and the second noise corresponding to the sample data included in the training data. According to the information processing device 100, the model parameters can be updated in a direction that reduces the calculated loss. As a result, the information processing device 100 can appropriately train the model.

[0153] According to the information processing device 100, machine learning interatomic potentials can be used as the model to be trained. This allows the information processing device 100 to efficiently train machine learning interatomic potentials.

[0154] According to the information processing device 100, a first noise, which is a random number, can be generated for the coordinates of each dimension of each atom in the system represented by each sample data. This allows the information processing device 100 to generate appropriate first noise.

[0155] The information processing method described in this embodiment can be implemented by executing a pre-prepared program on a computer such as a PC or workstation. The information processing program described in this embodiment is recorded on a computer-readable recording medium and executed by being read from the recording medium by the computer. The recording medium can be a hard disk, flexible disk, CD (Compact Disc)-ROM, MO (Magneto Optical Disc), DVD (Digital Versatile Disc), etc. Furthermore, the information processing program described in this embodiment may be distributed via a network such as the Internet.

[0156] 100 Information Processing Device 110, 500 Model 111, 501, 621, 711, 721, 731 Parameters 120, 510, 520 Dataset 130-133 Sample Data 141, 142, 901, 902 Atoms 150, 160, 170, 910, 920, 930 Systems 151-153, 161-164, 171-173, 911, 921, 922, 931, 941, 951, 952, 961, 1011, 1012, 1021, 1022, 1031, 1032, 1041, 1042, 1051, 1052, 1061, 1062 Vectors 180, 511, 521 Loss 200 Information Processing System 201 Client Device 210 Network 300 Bus 301 CPU 302 Memory 303 Network I / F 304 Recording Medium I / F 305 Recording Medium 306 Display 307 Input Device 400 Storage Unit 401 Acquisition Unit 402 First Generation Unit 403 Second Generation Unit 404 Third Generation Unit 405 Training Unit 406 Output Unit 610 Sample 620 MLIAP 630 Label 710 Encoder 720 Energy Head 730 Force Head 810-813 First Data 820, 830, 840 Table 1100 Graph

Claims

1. An information processing program characterized by causing a computer to execute a process that involves generating a first noise for each of the coordinates of each dimension of each atom in the system represented by the sample data for each of a plurality of sample data; calculating the average of the first noise generated for each dimension of each of the sample data for each of the coordinates of each dimension of each atom in the system represented by the sample data for each of the sample data; adding a second noise, obtained by removing the average calculated for each dimension from the first noise generated for each dimension of each of the coordinates of each dimension of each atom in the system represented by the sample data for each of the sample data, to the coordinates of each dimension; and performing self-supervised learning on a model that generates output data representing the force acting on each atom in the system according to input data representing the coordinates of each dimension of each atom in the system, based on training data that associates each of the sample data after the addition with the second noise corresponding to that sample data.

2. The information processing program according to claim 1, characterized in that the process performed is to calculate a loss for the model using a loss function according to the difference between the output data generated by the model that has been input with sample data included in the training data and the second noise corresponding to the sample data included in the training data, and update the parameters of the model in a direction that reduces the calculated loss.

3. The information processing program according to claim 2, characterized in that the model is a machine learning interatomic potential.

4. The information processing program according to any one of claims 1 to 3, characterized in that the generation process generates a first noise, which is a random number, for the coordinates of each dimension of each atom in the system represented by each of the sample data.

5. An information processing method characterized in that a computer performs the following steps: for each of a plurality of sample data, a first noise is generated for each dimension of the coordinates of each atom in the system represented by the sample data; for each of the sample data, the average of the first noise generated for each dimension of the first noise generated for each dimension of the first noise is calculated for each dimension of the first noise generated for each dimension of the first noise generated for each dimension of the first noise generated for each dimension of the first noise of the system represented by the sample data is added to the coordinates of each dimension; and self-supervised learning is performed on a model that generates output data representing the force acting on each atom in the system according to input data representing the coordinates of each dimension of each atom, based on training data that associates each of the sample data after the addition with the second noise corresponding to the sample data.

6. An information processing device comprising a control unit, which generates a first noise for each of the coordinates of each dimension of each atom in the system represented by the sample data for each of a plurality of sample data; calculates the average of the first noise generated for each dimension of each of the sample data for each of the coordinates of each dimension of each atom in the system represented by the sample data; adds a second noise obtained by removing the average calculated for each dimension from the first noise generated for each dimension of each of the sample data for each of the coordinates of each dimension of each atom in the system represented by the sample data, to the coordinates of each dimension; and performs self-supervised learning on a model that generates output data representing the force acting on each atom in the system according to input data representing the coordinates of each dimension of each atom, based on training data that associates each of the sample data after the addition with the second noise corresponding to that sample data.

Images