Self-supervised learning programs, methods, and apparatus

Abstract

High-precision machine learning models are generated through self-supervised learning. [Solution] The self-supervised learning device generates data representing a second molecule in which one or more atoms, which are groups of atoms arranged consecutively from among the multiple atoms contained in the first molecule, are replaced with other atoms, with each atom in which the number of atoms in the one or more atom groups is in a predetermined proportion of the multiple atoms. The data representing the second molecule is input into a machine learning model that makes predictions about molecular structure to obtain prediction results, and the parameters of the machine learning model are updated based on the comparison result between the correct data corresponding to the first molecule and the prediction results.

Description

[Technical Field]

[0001] The disclosed technologies relate to self-supervised learning programs, self-supervised learning methods, and self-supervised learning devices. [Background technology]

[0002] Materials discovery, which involves predicting the properties of material structures and discovering or optimizing new materials suitable for various applications, is conducted in a wide range of fields and industries and is essential for technological advancement. Specific examples of the objectives of materials discovery include the search for high-performance materials for aerospace vehicles, materials for herovskite solar cells, new catalysts that efficiently promote chemical reactions, materials for biodegradable plastics, and highly biocompatible medical alloys.

[0003] In materials discovery, the properties of material structures are predicted through experiments and simulations. For example, molecular dynamics (MD) simulations, which use Newton's equations of motion to determine the forces and energies of atoms and molecules, are used. MD simulations have the advantage of being able to calculate the time evolution of the system and having relatively low computational costs. Density functional theory (DFT) calculations, which use first-principles calculations based on quantum mechanics to determine the forces and energies of atoms and molecules, are also used. DFT calculations generally calculate the ground state and have relatively high computational costs.

[0004] Recently, material discovery is sometimes carried out using Materials Informatics (MI), which utilizes large amounts of data. MI is a field in materials science that uses AI (Artificial Intelligence) and other technologies to clarify the properties of materials and discover new materials. MI provides methods for analyzing large amounts of data and predicting the properties and behavior of materials. One example of an MI technique is high-throughput screening, which rapidly selects materials that meet certain criteria for a given problem setting. High-throughput screening is used to quickly evaluate a large number of material candidates using AI and other technologies to identify the optimal material. There is also a method that uses AI surrogate models to directly predict the results obtained from MD simulations and DFT calculations. Methods using AI surrogate models are used to replace computationally expensive simulations and can significantly reduce computation time.

[0005] Thus, in recent years, methods for rapidly predicting material properties using AI models have been developed. Various AI models have been proposed for predicting material properties, such as SchNet, PaiNN, GemNet-OC, and EquiformerV2.

[0006] To train an AI model, correct label data is necessary. For example, in machine learning in the field of materials, the input data is a graph of the chemical structure considering the atomic species and the position of each atom, and the labels are force, energy, etc. This label data related to material structures can be obtained through experiments or simulations. However, experiments require advanced equipment and expertise, and are time-consuming and expensive. Simulations, especially DFT, are computationally expensive and require large computing resources. In other words, both experiments and simulations are costly to conduct, making it difficult to collect large amounts of label data.

[0007] Therefore, there is a method called self-supervised learning, in which an AI model is pre-trained using a large amount of unlabeled data, and then trained using a small amount of labeled data. This method is also used in the field of materials discovery.

[0008] For example, an information processing system has been proposed that generates a trained model by performing machine learning based on sequence information representing sequences that include part or all of the sequences of multiple antigen-binding molecules or proteins. Based on the trained model, this system generates virtual sequence information in which at least one of the constituent units that make up the sequence represented by the sequence information input to the trained model has been mutated.

[0009] Furthermore, a method has been proposed for pre-training a graph neural network using, for example, an untagged compound dataset. In this method, one of the tagged compound datasets tags one of the compound's properties. This method further obtains a compound property prediction model by performing multi-task training on a graph neural network pre-trained using multiple tagged compound datasets. The compound property prediction model is used to predict various properties of a compound. [Prior art documents] [Patent Documents]

[0010] [Patent Document 1] International Publication No. 2020 / 246617 [Patent Document 2] Japanese Patent Publication No. 2022-163051 [Overview of the Initiative] [Problems that the invention aims to solve]

[0011] Traditional self-supervised learning methods for training machine learning models still have room for improvement in terms of accuracy.

[0012] As one aspect, the disclosed technology aims to generate a high-precision machine learning model through self-supervised learning.

Means for Solving the Problem

[0013] In one aspect, the disclosed technology replaces each of one or more atomic groups, which are sets of consecutively arranged atoms among a plurality of atoms included in a first molecule and the number of atoms included in the one or more atomic groups is a predetermined ratio of the plurality of atoms, with other atoms. Thereby, data representing a second molecule is generated. Further, the disclosed technology inputs the data representing the second molecule into a machine learning model for making predictions regarding molecular structures to obtain a prediction result. Then, the disclosed technology updates the parameters of the machine learning model based on the comparison result between the correct data corresponding to the first molecule and the prediction result.

Advantages of the Invention

[0014] As one aspect, it has the effect of being able to generate a high-precision machine learning model through self-supervised learning.

Brief Description of the Drawings

[0015] [Figure 1] It is a diagram for explaining the NN for predicting the energy of a molecule. [Figure 2] It is a diagram for explaining transfer learning. [Figure 3] It is a diagram for explaining Reference Method 1. [Figure 4] It is a diagram for explaining Reference Method 2. [Figure 5] It is a diagram for explaining fine-tuning. [Figure 6] It is a diagram for explaining the problems of the reference method. [Figure 7] It is a diagram for explaining the outline of this embodiment. [Figure 8] It is a functional block diagram of the self-supervised learning device according to this embodiment. [Figure 9]This is a diagram illustrating the processing of the generation unit. [Figure 10] This diagram illustrates the atoms that can be substituted according to the length of the atomic group. [Figure 11] This is a block diagram illustrating the schematic configuration of a computer that functions as a self-supervised learning device according to this embodiment. [Figure 12] This flowchart shows an example of a self-supervised learning process. [Modes for carrying out the invention]

[0016] An example of an embodiment relating to the disclosed technology will be described below with reference to the drawings.

[0017] Before describing the details of the embodiments, we will explain self-supervised learning for machine learning models that predict material properties.

[0018] For example, as shown in Figure 1, a neural network (hereinafter also referred to as "NN") that predicts the energy of a molecule based on its structural data is trained using supervised learning with labeled data consisting of "structure" and "energy". That is, molecular structural data is input to the NN, and the parameters of the NN are updated by backpropagating the difference between the predicted molecular energy (output from the NN) and the correct energy. This correct energy is calculated, for example, by simulation such as DFT. Since the computation time for molecular energy using DFT is very long, it is difficult to collect a large amount of labeled data for supervised learning.

[0019] Therefore, there is a technique called self-supervised learning, which generates correct answers (labels) for a task from unlabeled data and performs supervised learning using the generated labels. Generally, self-supervised learning is applied to pre-training in transfer learning. As shown in Figure 2, transfer learning pre-trains a neural network (NN) using self-supervised learning with unlabeled data. The NN includes a feature extraction unit that extracts features from the input data and an output unit that corresponds to the task. The output unit used during pre-training is a pre-training unit. In pre-training, machine learning is performed using a large amount of data similar to the data domain that is actually to be specialized. By using a large amount of data to train the NN on the essential features of the input molecular data, the initial state of the NN can be improved.

[0020] Furthermore, if the tasks for pre-training and fine-tuning (FT) are different, the output layer is modified for FT. For example, if the pre-training task is 3-class classification but the fine-tuning task is 4-class classification, the output layer is changed to a fully connected layer for 4-class classification. After that, the neural network (NN) is trained using supervised learning with labeled data for fine-tuning. The feature extraction unit at this stage is the same as the one pre-trained. That is, the weights of the pre-trained NN are used as initial values, and machine learning is performed on the actual data. By applying self-supervised learning as pre-training, the knowledge gained from pre-training can be utilized, and even when there is little labeled data needed for fine-tuning (supervised learning) after pre-training, the accuracy of the NN can be improved.

[0021] Two example methods are presented for self-supervised learning of neural networks (NNs) used to predict material properties. Method 1, as shown in Figure 3, prepares molecular data containing information on the atomic species and 3D position coordinates of each of the multiple atoms constituting the molecule. In the example in Figure 3, each atom is represented by a circle, with α, β, etc., inside the circle representing the atomic species.

[0022] In Reference Method 1, a predetermined proportion of atoms in the prepared molecular data are replaced with other atoms. In the example in Figure 3, the shaded circles represent the atoms that have been replaced from the original molecular data. Then, the neural network (NN) is asked to predict whether each atom in the substituted molecular data has been replaced from each atom in the original molecular data. In other words, Reference Method 1 sets up an output unit (for pre-training) for performing a binary classification task. Since Reference Method 1 can generate ground truth data from the difference between the original molecular data and the substituted molecular data, self-supervised learning is possible. In the example in Figure 3, atoms that have not been replaced are represented by "0" and atoms that have been replaced are represented by "1" in both the ground truth data and the prediction results.

[0023] Reference Method 2, as shown in Figure 4, replaces a predetermined proportion of atoms in the prepared molecular data with other atoms, and then has the neural network predict the original atom species before the substitution for each atom included in the substituted molecular data. In other words, Reference Method 2 sets up an output unit (for pre-training) for performing a multi-class classification task. In Reference Method 2, the original molecular data becomes the ground truth data, making self-supervised learning possible.

[0024] In the examples in Figures 3 and 4, when referring to the ground truth data, the predictions made by the neural network (NN) are incorrect for the atoms indicated by the shaded areas. The NN parameters are updated so that these predictions match the ground truth data.

[0025] In both Reference Methods 1 and 2, as shown in Figure 5, the feature extraction unit of the NN, pre-trained by self-supervised learning, is used as the initial value, and fine-tuning is performed with the actual training data. During the fine-tuning stage, it is assumed that the feature extraction unit of the NN has learned essential features such as the structure of atoms and molecules. Furthermore, since the output unit of the NN during pre-training is for predicting binary or multi-class classification tasks, in fine-tuning, an output unit is set according to the actual task, such as predicting force or energy.

[0026] In reference methods 1 and 2, as shown in Figure 6, the atomic species of randomly selected atoms (shaded circles in Figure 6) are replaced in the original molecular data. In this case, it is easy to determine which atom has been replaced and what its original atomic species is from the atomic species of the surrounding atoms that have not been replaced (thick-lined circles in Figure 6). Therefore, the neural network can predict the correct answer without learning the essential characteristics of the entire molecular structure, which reduces the accuracy of the neural network as a tool for predicting material properties.

[0027] Therefore, in this embodiment, as shown in Figure 7, in the original molecular data, each atom contained in a cluster of a certain length, which is arranged consecutively, is replaced with another atom. The self-supervised learning device according to this embodiment will be described in detail below.

[0028] As shown in Figure 8, the self-supervised learning device 10 functionally includes a generation unit 12, an acquisition unit 14, and an update unit 16. Furthermore, a predetermined memory area of ​​the self-supervised learning device 10 stores an atomic library 22 and a neural network (NN) 24. The NN 24 is an example of a “machine learning model” in the disclosed technology.

[0029] As shown in Figure 8, the self-supervised learning device 10 receives first molecular data. The first molecular data includes, for example, information on the atomic species and three-dimensional position coordinates of each atom contained in the first molecule. The first molecular data may also include information indicating the bonding relationships between atoms.

[0030] The generation unit 12 acquires the first molecular data input to the self-supervised learning device 10. The generation unit 12 replaces each atom in the first molecule that is part of one or more atom groups, which are sets of atoms arranged consecutively, with other atoms, where the number of atoms in the one or more atom groups is in a predetermined proportion of the total number of atoms. As a result, the generation unit 12 generates second molecular data representing the second molecule.

[0031] Specifically, the generating unit 12 selects a group of atoms to be substituted, which includes an atom randomly selected from a plurality of atoms contained in the first molecule, and a predetermined number of atoms selected in order of proximity to that atom.

[0032] More specifically, the generation unit 12 determines the number of atom groups to be replaced by dividing a predetermined proportion of multiple atoms contained in the first molecule by a predetermined number representing the length of one atom group, i.e., the number of atoms contained in the atom group. The generation unit 12 also randomly selects a number of atoms equal to the number of atom groups determined from the multiple atoms contained in the first molecule, and selects each of a predetermined number of atom groups based on each of the selected atoms. Then, the generation unit 12 generates second molecular data by replacing each atom contained in each of the selected atom groups with any atom from the atom library 22.

[0033] The atom library 22 stores values ​​representing a predetermined number of types of atoms. The atoms whose values ​​are stored in the atom library 22 may include atoms contained in the first molecule, or atoms not contained in the first molecule, as long as they are atoms that actually exist.

[0034] Referring to Figure 9, an example of the processing in the generation unit 12 will be explained. Here, the number of atoms in the first molecule is n, and the number of atoms to be substituted is n. replace n group Let r be the substitution rate and L be the length of one group of atoms. The substitution rate is a value between 0 and 1 (0 to 100%), where a substitution rate of 0% means no substitution, and a substitution rate of 100% means all atoms in the first molecule are substituted.

[0035] The generating unit 12 sets the number of atoms to be substituted to n replace It is calculated as =n × r. Here, n replace Since it is an integer, it may be rounded up or otherwise as appropriate. Also, the generating unit 12 determines the number of atoms in a group of atoms of length L as n group =n replace It is calculated using / L. Here, n groupSince it is an integer, rounding up or the like may be performed as appropriate. And the generation unit 12 randomly extracts n group atoms from the first molecule. For example, when the number of atoms n = 25, the substitution rate r = 0.4, and the length L of the atomic group = 5, the generation unit 12 calculates the number of atoms to be substituted n replace = n×r = 25×0.4 = 10, and calculates the number of atomic groups n group = n replace / L = 10 / 5 = 2.

[0036] The generation unit 12 randomly selects two atoms from the first molecule, for example, as shown in A of FIG. 9. When the generation unit 12 selects n group atoms, it may be selected so that the positions of the selected atoms in the first molecule are not biased as much as possible. For example, the atoms in the first molecule may be divided into n group blocks based on the three-dimensional position coordinates of each atom, and one atom may be randomly selected from each block.

[0037] The generation unit 12 selects L - 1 atoms near the selected atoms. For example, the generation unit 12 calculates the distance between the three-dimensional position coordinates of the selected atoms and the three-dimensional position coordinates of each of the other atoms. Then, as shown in B of FIG. 9, the generation unit 12 selects L - 1 atoms as neighboring atoms in ascending order of the distance from the atom first selected. In FIG. 9, the shaded circles represent the atoms first selected, and the thick-lined circles represent the neighboring atoms. In the above example, since L = 5, four neighboring atoms are selected for each of the atoms first selected. As a result, n replace atoms are selected as atomic groups of length L with n group atoms.

[0038] In addition, if there are overlapping atoms among the selected atomic groups, the neighboring atoms may be reselected so that there is no overlap, or the selection may be made while allowing overlap, that is, the number of atoms less than the number of overlapping atoms by n replace may be selected. Also, when the molecular data includes information indicating the bonds between atoms, the atoms that are consecutive by following the bond relationship from the atom first selected may be selected as neighboring atoms.

[0039] Here, as shown in Figure 10, when the length L of one group of atoms is sufficiently large, the selected atoms form a single group of atoms. That is, L ≥ n replace In this case, only one group of atoms will be selected. If L=1, the same applies as in Reference Methods 1 and 2.

[0040] The generation unit 12 generates second molecular data by replacing the values ​​of the atoms to be substituted in the first molecular data with the values ​​of atoms randomly selected from the atom library 22. In this case, the generation unit 12 may replace the atoms to be substituted with the values ​​of different atoms, or some of the atoms to be substituted may be converted to the same value.

[0041] NN24 includes a feature extraction unit that extracts features representing the molecular structure of a second molecule from second molecular data, and an output unit that outputs prediction results according to the pre-training task based on the features extracted by the feature extraction unit. The pre-training task may be a binary classification task, as shown in Figure 3, which predicts whether each atom in the second molecule has been replaced by each atom in the first molecule. Alternatively, the pre-training task may be a multi-class classification task, as shown in Figure 4, which predicts the original atom before replacement for each atom in the second molecular data.

[0042] The acquisition unit 14 inputs the second molecular data generated by the generation unit 12 to the NN24 and acquires the prediction results output from the NN24. In the case of the binary classification task described above, the prediction results are information indicating whether or not each atom in the second molecule has been substituted from the first molecule, and in the case of the multi-class classification task, they are information indicating the original atomic species of each atom in the second molecule before substitution.

[0043] The update unit 16 updates the parameters of the NN24 based on the comparison result between the ground truth data corresponding to the first molecule and the prediction result obtained by the acquisition unit 14. Specifically, in the case of the binary classification task described above, the update unit 16 generates ground truth data in the first molecular data and the second molecular data where corresponding atoms with the same 3D position coordinates are [0] if they are the same atomic species, and [1] if they are different. In the case of the multi-class classification task described above, the update unit 16 uses the first molecular data as the ground truth data. The update unit 16 then backpropagates the difference between the prediction result and the ground truth data to the NN24 and updates the parameters of the NN24.

[0044] The update unit 16 repeatedly updates the parameters of NN24 until a predetermined termination condition is met. The predetermined termination condition may be, for example, when the number of iterations reaches a predetermined number, when the difference between the prediction result and the ground truth data falls below a predetermined value, or when the difference between the previous difference and the current difference falls below a predetermined value (when the difference converges).

[0045] The self-supervised learning device 10 may be implemented, for example, by the computer 40 shown in Figure 11. The computer 40 comprises a CPU (Central Processing Unit) 41, a GPU (Graphics Processing Unit) 42, a memory 43 as a temporary storage area, and a non-volatile storage device 44. The computer 40 also comprises input / output devices 45 such as input devices and display devices, and an R / W (Read / Write) device 46 that controls the reading and writing of data to and from the storage medium 49. The computer 40 also comprises a communication interface 47 that connects to a network such as the Internet. The CPU 41, GPU 42, memory 43, storage device 44, input / output devices 45, R / W device 46, and communication interface 47 are connected to each other via a bus 48.

[0046] The storage device 44 is, for example, an HDD (Hard Disk Drive), an SSD (Solid State Drive), or flash memory. The storage device 44 stores a self-supervised learning program 50 that causes the computer 40 to function as a self-supervised learning device 10. The self-supervised learning program 50 includes a generation process control instruction 52, an acquisition process control instruction 54, and an update process control instruction 56. The storage device 44 also has an information storage area 60 in which information constituting the atomic library 22 and NN24 is stored.

[0047] The CPU 41 reads the self-supervised learning program 50 from the storage device 44, loads it into memory 43, and sequentially executes the control instructions contained in the self-supervised learning program 50. The CPU 41 operates as the generation unit 12 shown in Figure 8 by executing the generation process control instruction 52. The CPU 41 also operates as the acquisition unit 14 shown in Figure 8 by executing the acquisition process control instruction 54. The CPU 41 also operates as the update unit 16 shown in Figure 8 by executing the update process control instruction 56. The CPU 41 also reads information from the information storage area 60 and loads the atomic library 22 and NN24 into memory 43. As a result, the computer 40 that executed the self-supervised learning program 50 functions as a self-supervised learning device 10. The CPU 41 that executes the program is hardware. Part of the program may also be executed by the GPU 42.

[0048] Furthermore, the functions realized by the self-supervised learning program 50 may be implemented, for example, by semiconductor integrated circuits, more specifically by ASICs (Application Specific Integrated Circuits), FPGAs (Field-Programmable Gate Arrays), etc.

[0049] Next, the operation of the self-supervised learning device 10 according to this embodiment will be described. When the first molecular data is input to the self-supervised learning device 10 and the execution of self-supervised learning of NN24 is instructed, the self-supervised learning process shown in Figure 12 is executed in the self-supervised learning device 10. Note that the self-supervised learning process is an example of the self-supervised learning method of the disclosed technology.

[0050] In step S10, the generation unit 12 acquires the first molecular data input to the self-supervised learning device 10 and creates copies, using one copy for generating the second molecular data and the other for generating the correct answer data.

[0051] Next, in step S12, the generating unit 12 selects one or more atom groups from among the multiple atoms contained in the first molecule, which are groups of atoms arranged consecutively, and the number of atoms in these groups is in a predetermined proportion of the total number of atoms, as the atom to be substituted.

[0052] Next, in step S14, the generation unit 12 generates second molecular data by replacing the values ​​of the atoms to be substituted in the first molecular data with the values ​​of atoms randomly selected from the atom library 22.

[0053] Next, in step S16, the acquisition unit 14 inputs the generated second molecular data into the NN24 to obtain prediction results regarding the molecular structure. Next, in step S18, the update unit 16 generates ground truth data based on the first molecular data. Next, in step S20, the update unit 16 backpropagates the difference between the prediction results and the ground truth data to the NN24 to update the parameters of the NN24.

[0054] Next, in step S22, the update unit 16 determines whether the termination condition for parameter updates has been met. If the termination condition is met, the process proceeds to step S24; otherwise, it returns to step S10. In step S24, the update unit 16 stores the NN24 with the final parameters set in a predetermined memory area, and the self-supervised learning process ends.

[0055] As described above, the self-supervised learning device according to this embodiment generates second molecular data in which a predetermined proportion of the atoms in the first molecule are replaced with one of the atoms in the atom library. The self-supervised learning device then inputs the second molecular data into a neural network (NN) that makes predictions about the molecular structure, obtains the prediction results, and updates the NN parameters based on the comparison result between the ground truth data corresponding to the first molecule and the prediction results. In this way, instead of selecting atoms to be replaced individually and randomly, the atoms to be replaced are selected as a group of atoms containing a predetermined number of atoms, making it difficult to make predictions from neighboring atom species in the pre-training task, and creating a challenging problem that requires predictions from a larger number of surrounding atoms. This difficult prediction problem allows for the generation of highly accurate machine learning models through self-supervised learning, as it accelerates the learning of essential features of the molecular structure.

[0056] Here, we will explain the results of verifying the effects of the above embodiment. In this verification, the following methods were tested.

[0057] Comparison Method 1: A method for supervised learning of a neural network using only labeled data (without self-supervised learning) Comparison Method 2: A method that fine-tunes a neural network (NN) pre-trained using self-supervised learning (predicting whether or not atoms have been substituted) from Reference Method 1, using labeled data. Method 1: A method for fine-tuning a neural network (NN) pre-trained (predicting the original atoms) using the method of this embodiment with labeled data.

[0058] In this validation, assuming that a small amount of data is used for fine-tuning, evaluation was performed using a portion of the publicly available dataset. For self-supervised learning, 460,328 training data points and 24,943 validation data points were used. For fine-tuning, 1,120 training data points and 2,624 validation data points were extracted from the total of 45,890 data points. Furthermore, a pre-trained model was generated by performing self-supervised learning for 4 epochs, and a machine learning model that predicts energy from molecular structure was generated by performing fine-tuning for 200 epochs. Note that in comparison method 1, only machine learning equivalent to fine-tuning was performed. The hyperparameters were set to substitution rate r=0.3 and atomic group length L=5.

[0059] The mean absolute error between the energy predicted for the validation data by the machine learning model generated through fine tuning and the energy label was used as an evaluation value for comparison. In other words, a lower evaluation value indicates higher accuracy. The table below shows the evaluation values ​​after fine tuning. As shown in the table below, an improvement in accuracy was observed with this method.

[0060] [Table 1]

[0061] In the above embodiment, the self-supervised learning program is pre-stored (installed) in the storage device, but the invention is not limited to this. The program relating to the disclosed technology may be provided in a form stored on a storage medium such as a CD-ROM, DVD-ROM, or USB memory.

[0062] The following additional information is disclosed regarding the embodiments described above.

[0063] (Note 1) Data is generated to represent a second molecule in which one or more atoms, which are groups of atoms arranged consecutively among the multiple atoms contained in the first molecule, are replaced by other atoms, with each atom in which the number of atoms in the one or more atom groups is in a predetermined proportion of the multiple atoms. The data representing the second molecule is input into a machine learning model that makes predictions about the molecular structure to obtain prediction results. Based on the comparison result between the ground truth data corresponding to the first molecule and the prediction result, the parameters of the machine learning model are updated. A self-supervised learning program that causes a computer to perform a process that includes the following.

[0064] (Note 2) The aforementioned group of atoms is a self-supervised learning program as described in Appendix 1, which includes atoms randomly selected from the plurality of atoms and a predetermined number of atoms selected in order of proximity to the selected atoms.

[0065] (Note 3) The process for generating data representing the second molecule is a self-supervised learning program as described in Appendix 2, which includes determining the number of atom groups by dividing the predetermined proportion of multiple atoms contained in the first molecule by the predetermined number, and randomly selecting a number of atoms equal to the number of atom groups from the multiple atoms contained in the first molecule.

[0066] (Note 4) The machine learning model is a self-supervised learning program as described in any one of the appendices 1 to 3, which predicts whether each atom in the second molecule is substituted for each atom in the first molecule.

[0067] (Note 5) The machine learning model is a self-supervised learning program as described in any one of the appendices 1 to 3, which predicts the original atoms of the atoms in the second molecule that have been substituted for atoms in the first molecule.

[0068] (Note 6) The process for generating data representing the second molecule is a self-supervised learning program as described in any one of the appendices 1 to 5, wherein the other atoms are selected from a predetermined set of atoms other than the types of atoms contained in the first molecule.

[0069] (Note 7) Data is generated to represent a second molecule in which one or more atoms, which are groups of atoms arranged consecutively among the multiple atoms contained in the first molecule, are replaced by other atoms, with each atom in which the number of atoms in the one or more atom groups is in a predetermined proportion of the multiple atoms. The data representing the second molecule is input into a machine learning model that makes predictions about the molecular structure to obtain prediction results. Based on the comparison result between the ground truth data corresponding to the first molecule and the prediction result, the parameters of the machine learning model are updated. A self-supervised learning method in which a computer performs processes that include the following.

[0070] (Note 8) The self-supervised learning method described in Appendix 7, wherein the group of atoms includes atoms randomly selected from the plurality of atoms and a predetermined number of atoms selected in order of proximity to the selected atoms.

[0071] (Note 9) The self-supervised learning method described in Appendix 8, wherein the process for generating data representing the second molecule includes determining the number of atom groups by dividing the predetermined proportion of multiple atoms contained in the first molecule by the predetermined number, and randomly selecting a number of atoms equal to the number of atom groups from the multiple atoms contained in the first molecule.

[0072] (Note 10) The machine learning model predicts whether each atom in the second molecule is substituted for each atom in the first molecule, according to the self-supervised learning method described in any one of the appendices 7 to 9.

[0073] (Note 11) The machine learning model predicts the original atoms of the atoms in the second molecule that have been substituted for atoms in the first molecule, according to the self-supervised learning method described in any one of the appendices 7 to 9.

[0074] (Note 12) The process for generating data representing the second molecule is a self-supervised learning method as described in any one of the appendices 7 to 11, wherein the other atoms are selected from a predetermined set of multiple types of atoms other than the types of atoms contained in the first molecule.

[0075] (Note 13) A generation unit generates data indicating a second molecule in which one or more atoms, which are groups of atoms arranged consecutively among the multiple atoms contained in the first molecule, are replaced by other atoms, with each atom in which the number of atoms in the one or more atom groups is in a predetermined proportion of the multiple atoms. An acquisition unit that inputs data representing the second molecule into a machine learning model that makes predictions about the molecular structure and obtains prediction results, An update unit updates the parameters of the machine learning model based on a comparison result between the ground truth data corresponding to the first molecule and the prediction result, A self-supervising learning device that includes [this].

[0076] (Note 14) The self-supervised learning device according to Appendix 13, wherein the group of atoms includes atoms randomly selected from the plurality of atoms and a predetermined number of atoms selected in order of proximity to the selected atoms.

[0077] (Note 15) The self-supervised learning device according to Appendix 14, wherein the generating unit determines the number of atom groups by dividing the predetermined proportion of multiple atoms contained in the first molecule by the predetermined number, and randomly selects a number of atoms equal to the number of atom groups from the multiple atoms contained in the first molecule.

[0078] (Note 16) The machine learning model predicts whether each atom in the second molecule is substituted for each atom in the first molecule, according to any one of the appendices 13 to 15.

[0079] (Note 17) The machine learning model predicts the original atoms of the atoms in the second molecule that have been substituted from the atoms in the first molecule, as described in any one of the appendices 13 to 15.

[0080] (Note 18) The generating unit is a self-supervised learning device according to any one of the appendices 13 to 17, wherein the generating unit selects an atom from a predetermined set of multiple types of atoms other than the types of atoms contained in the first molecule to form the other atoms.

[0081] (Note 19) Data is generated to represent a second molecule in which one or more atoms, which are groups of atoms arranged consecutively among the multiple atoms contained in the first molecule, are replaced by other atoms, with each atom in which the number of atoms in the one or more atom groups is in a predetermined proportion of the multiple atoms. The data representing the second molecule is input into a machine learning model that makes predictions about the molecular structure to obtain prediction results. Based on the comparison result between the ground truth data corresponding to the first molecule and the prediction result, the parameters of the machine learning model are updated. A non-temporary storage medium that stores a self-supervised learning program for causing a computer to perform a process that includes the following. [Explanation of Symbols]

[0082] 10 Self-supervised learning device 12 Generation part 14 Acquisition Department 16 Update section 22 Atomic Library 24 NN 40 Computers 41 CPU 42 GPU 43 memory 44 Storage device 45 Input / Output Devices 46 R / W device 47 Communication I / F 48 bus 49 Storage medium 50 Self-supervised learning programs 52 Generation Process Control Instructions 54 Acquisition process control instruction 56 Update process control instructions 60 Information storage area

Claims

1. Data is generated to represent a second molecule in which one or more atoms, which are groups of atoms arranged consecutively among the multiple atoms contained in the first molecule, are replaced by other atoms, with each atom in the one or more atom groups having a predetermined ratio of the number of atoms in that group being replaced by another atom. The data representing the second molecule is input into a machine learning model that makes predictions about the molecular structure to obtain prediction results. Based on the comparison result between the ground truth data corresponding to the first molecule and the prediction result, the parameters of the machine learning model are updated. A self-supervised learning program that causes a computer to perform a process that includes the following.

2. The self-supervised learning program according to claim 1, wherein the group of atoms includes atoms randomly selected from the plurality of atoms and a predetermined number of atoms selected in order of proximity to the selected atoms.

3. The self-supervised learning program according to claim 2, wherein the process for generating data representing the second molecule includes determining the number of atom groups by dividing the predetermined proportion of multiple atoms contained in the first molecule by the predetermined number, and randomly selecting a number of atoms equal to the number of atom groups from the multiple atoms contained in the first molecule.

4. The self-supervised learning program according to any one of claims 1 to 3, wherein the machine learning model predicts whether each of the atoms contained in the second molecule is substituted for each of the atoms contained in the first molecule.

5. The self-supervised learning program according to any one of claims 1 to 3, wherein the machine learning model predicts the original atom of the atom that was replaced from the atom in the first molecule among the atoms in the second molecule.

6. The self-supervised learning program according to any one of claims 1 to 3, wherein the process for generating data representing the second molecule is to select an atom from a predetermined set of multiple types of atoms other than the types of atoms contained in the first molecule to be the other atom.

7. Data is generated to represent a second molecule in which one or more atoms, which are groups of atoms arranged consecutively among the multiple atoms contained in the first molecule, are replaced by other atoms, with each atom in the one or more atom groups having a predetermined ratio of the number of atoms in that group being replaced by another atom. The data representing the second molecule is input into a machine learning model that makes predictions about the molecular structure to obtain prediction results. Based on the comparison result between the ground truth data corresponding to the first molecule and the prediction result, the parameters of the machine learning model are updated. A self-supervised learning method in which a computer performs processes that include the following.

8. A generation unit generates data indicating a second molecule in which one or more atoms, which are groups of atoms arranged consecutively among the multiple atoms contained in the first molecule, are replaced by other atoms, with each atom in the one or more atom groups having a predetermined ratio of the number of atoms in each group being replaced by another atom. An acquisition unit that inputs the data representing the second molecule into a machine learning model that makes predictions about the molecular structure and obtains prediction results, An update unit updates the parameters of the machine learning model based on a comparison result between the correct data corresponding to the first molecule and the prediction result, A self-supervising learning device that includes [this].

Images