Learning program, method, and device with self-teacher

By generating data with replaced atoms and updating neural network parameters based on correct predictions, the method enhances the accuracy of molecular energy prediction in self-supervised learning by avoiding unnecessary learning of non-existent 'mask' information.

JP2024168781APending Publication Date: 2024-12-05FUJITSU LTD
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
JP2023085718
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-05-24
Publication Date
2024-12-05

AI Technical Summary

Technical Problem

Conventional self-supervised learning methods for neural networks predicting molecular energies lead to unnecessary learning of non-existent 'mask' information, reducing prediction accuracy.

Method used

Generate data for a second molecule by replacing a predetermined proportion of atoms in a first molecule with 0, input this data into a neural network, and update its parameters based on the comparison between the correct and predicted results to suppress unnecessary learning.

Benefits of technology

Improves the prediction accuracy of neural networks trained by self-supervised learning by preventing the learning of irrelevant information.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024168781000001_ABST
    Figure 2024168781000001_ABST
Patent Text Reader

Abstract

To improve the accuracy of prediction of a neural network trained by a learning with a self-teacher.SOLUTION: A learning device with a self-teacher generates data showing a second molecule obtained by replacing the value showing each of a predetermined ratio of atoms of the atoms in a first molecule with 0, inputs data showing the second molecule into a machine learning model which makes a prediction about a molecular structure and acquires the result of the prediction, and updates a parameter of the machine learning model on the basis of correct data for the first molecule and the result of comparison with the result of prediction.SELECTED DRAWING: Figure 6
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The disclosed technology relates to a self-supervised learning program, a self-supervised learning method, and a self-supervised learning apparatus. [Background technology]

[0002] Conventionally, neural networks have been proposed that predict the energy of a molecule based on its structural data. These neural networks are trained by supervised learning using labeled data consisting of "structure" and "energy." This "energy" is calculated, for example, by a numerical calculation method called density functional theory (DFT). The calculation time for molecular energy using DFT is very long, and it can take half a day to three days to calculate the energy of one structure. Therefore, it is difficult to collect a large amount of labeled data for supervised learning.

[0003] Therefore, there is a method called self-supervised learning, in which a correct answer (label) for a task is generated from unlabeled data, and supervised learning is performed using the generated label. For example, self-supervised learning is performed by masking some words in a sentence and having a neural network predict the masked words. A self-supervised learning method has also been proposed for neural networks that predict the energy of molecules. In this method, some of the atoms and bonds between atoms that make up the molecule are masked. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Special Publication No. 2021-518024 [Non-patent literature]

[0005] [Non-Patent Document 1] Kristof T. Sch▲u▼tt, Oliver T. Unke, Michael Gastegger, "Equivariant Message Passing for the Prediction of Tensorial Properties and Molecular Spectra," PMLR, 2021. [Non-Patent Document 2] Johannes Gasteiger, Muhammed Shuaibi, Anuroop Sriram, Stephan Gunnemann, Zachary Ulissi, C. Lawrence Zitnick, Abhishek Das, "GemNet-OC: Developing Graph Neural Networks for Large and Diverse Molecular Simulation Datasets," Transactions on Machine Learning Research, 2022. [Non-Patent Document 3] Zaixi Zhang, Qi Liu, Shengyu Zhang, Chang-Yu Hsieh, Liang Shi, Chee-Kong Lee, "Graph Self-Supervised Learning for Optoelectronic Properties of Organic Semiconductors," ICML, 2022. Summary of the Invention [Problem to be solved by the invention]

[0006] However, in conventional technology, the neural network ends up learning "mask" information that does not exist as actual atoms, which results in unnecessary learning and a decrease in the prediction accuracy of the neural network.

[0007] In one aspect, the disclosed technology aims to improve the predictive accuracy of neural networks trained through self-supervised learning. [Means for solving the problem]

[0008] In one embodiment, the disclosed technology generates data representing a second molecule in which values ​​representing a predetermined percentage of atoms contained in a first molecule are replaced with 0. The disclosed technology also inputs the data representing the second molecule into a machine learning model that predicts a molecular structure to obtain a prediction result. The disclosed technology then updates parameters of the machine learning model based on a comparison result between ground truth data corresponding to the first molecule and the prediction result. Effect of the Invention

[0009] One aspect is that it has the effect of improving the prediction accuracy of a neural network trained by self-supervised learning. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a diagram for explaining a NN for predicting molecular energy. [Diagram 2] FIG. 1 is a diagram for explaining transfer learning. [Diagram 3] FIG. 1 is a diagram for explaining self-supervised learning for NN for molecular energy prediction. [Figure 4] FIG. 1 is a diagram for explaining the challenges of self-supervised learning for NN for molecular energy prediction. [Diagram 5] FIG. 1 is a functional block diagram of a self-supervised learning device. [Figure 6] FIG. 1 is a diagram for explaining the processing of a self-supervised learning device. [Figure 7] FIG. 1 is a block diagram showing a schematic configuration of a computer that functions as a self-supervised learning device. [Figure 8] 1 is a flowchart illustrating an example of a self-supervised learning process. [Figure 9] FIG. 13 is a diagram showing the results of verification of the effect of the present method. [Figure 10] FIG. 13 is a diagram showing the results of verification of the effect of the present method. [Figure 11] FIG. 2 is a diagram for explaining an example of the configuration of a NN. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, an example of an embodiment of the disclosed technology will be described with reference to the drawings.

[0012] Before describing the details of the embodiment, we will explain the challenges in self-supervised learning towards neural networks for molecular energy prediction.

[0013] As shown in Figure 1, a neural network (hereafter also referred to as "NN") that predicts the energy of a molecule based on its structural data is trained by supervised learning using labeled data consisting of "structure" and "energy." That is, the structural data of the molecule is input to the NN, and the parameters of the NN are updated by backpropagating the difference between the predicted molecular energy, which is the output from the NN, and the correct energy. This correct energy is calculated, for example, by a numerical calculation method called DFT. Because it takes a very long time to calculate the molecular energy using DFT, it is difficult to collect a large amount of labeled data for supervised learning.

[0014] Therefore, there is a method called self-supervised learning, in which a correct answer (label) for a task is generated from unlabeled data, and supervised learning is performed using the generated label. Generally, self-supervised learning is applied to pre-learning in transfer learning. As shown in Figure 2, in transfer learning, a NN is pre-trained by self-supervised learning using unlabeled data. The NN includes a feature extraction unit that extracts features of input data, and an output unit according to the task. The output unit used during pre-learning is the one for pre-learning.

[0015] Then, if the tasks are different between pre-learning and fine-tuning, the output section is changed for fine-tuning. For example, if the pre-learning task is 3-class classification but the fine-tuning task is 4-class classification, the output section is changed to a fully connected layer for 4-class classification. The NN is then trained by supervised learning using labeled data for fine-tuning. The feature extraction section at this time applies the pre-trained data as is. By applying self-supervised learning as pre-learning, it is possible to achieve high accuracy in the NN even in situations where there is little labeled data required for fine-tuning (supervised learning) after pre-learning.

[0016] A self-supervised learning method has also been proposed for neural networks that predict the energy of molecules. In this method, as shown in Figure 3, a molecular structure is prepared that is represented by the atoms that make up the molecule and the bonds between the atoms. In the example in Figure 3, each atom is represented by a circle, and different patterns of the circles represent different types of atoms. A certain percentage of the atoms or bonds in the molecular structure are masked, and the NN is made to predict the atoms or bonds in the masked areas. Since the structure of the molecule is known, the correct answers (actual atoms or bonds) for the masked areas are known, making self-supervised learning possible.

[0017] As shown in Fig. 4, the structure of a molecule is represented by input data in which each atom constituting the molecule is assigned a value (vector) according to the atomic attributes such as the element. In the input data, atoms that are the subject of masking are replaced with a value according to the mask (indicated as "x" in Fig. 4), for example, a value obtained by normalizing a random initial value using the Xavier method. Then, the NN performs a multi-value class classification prediction to predict which atom in the atom library (A, B, C, D in Fig. 4) the masked atom belongs to.

[0018] However, in this case of self-supervised learning that masks atoms, the NN is trained using molecular structures containing "mask" information that does not actually exist as input data. The values ​​of the masked parts in the input data are input to realize the mask, and there is no atom called "mask" among the atoms that make up the molecule. Therefore, the NN learns "mask" information that does not actually exist, which results in unnecessary learning and a decrease in the prediction accuracy of the trained NN.

[0019] Therefore, in the following embodiment, when performing self-supervised learning of a NN by masking some of the atoms contained in a molecule, unnecessary learning is suppressed and the prediction accuracy of the NN is improved. The following describes this embodiment in detail.

[0020] As shown in Fig. 5, the self-supervised learning device 10 functionally includes a generating unit 12, an acquiring unit 14, and an updating unit 16. Furthermore, an atom library 22 and a neural network (NN) 24 are stored in a predetermined storage area of ​​the self-supervised learning device 10. The NN 24 is an example of a "machine learning model" of the disclosed technology. Furthermore, a case will be described in which the NN 24 in this embodiment is a graph neural network (GNN).

[0021] The generation unit 12 acquires first molecule data indicating a first molecule to be input to the self-supervised learning device 10. The first molecule data is, for example, graph data in which atoms included in the first molecule are represented as nodes and bonds between atoms are represented as edges connecting the nodes. Each node holds a value (vector) indicating the atom corresponding to that node. The generation unit 12 generates second molecule data indicating a second molecule in which values ​​indicating a predetermined percentage of atoms included in the first molecule have been replaced with 0 (a vector with all elements being 0).

[0022] A specific description will be given with reference to FIG. 6. The generation unit 12 specifies, among the atoms included in the first molecule, a number of atoms according to a specified replacement rate as replacement targets. The replacement rate is a value from 0 to 100%, where a replacement rate of 0% means no replacement and a replacement rate of 100% means that all atoms included in the molecule are replaced. The example in FIG. 6 shows a case where the replacement rate is 40%, and two out of five atoms included in the molecule are specified as replacement targets (shaded circles). The generation unit 12 also generates second molecule data by replacing the node values ​​of the atoms to be replaced in the first molecule data with 0.

[0023] The NN 24 includes a feature extraction unit that extracts features indicating the molecular structure of the second molecule from the second molecule data, and an output unit that outputs a prediction result corresponding to a pre-learning task based on the features extracted by the feature extraction unit. In this embodiment, the pre-learning task is a multi-value classification task that predicts which atom in the atom library 22 each of the atoms included in the second molecule whose value has been replaced with 0 corresponds to. The atom library 22 stores values ​​indicating each atom of a predetermined group of multiple types of atoms.

[0024] The acquisition unit 14 inputs the second molecular data generated by the generation unit 12 to the NN 24, and acquires information indicating which atoms in the atomic library 22 each of the atoms whose values ​​have been replaced with 0, which are the prediction results output from the NN 24, corresponds to.

[0025] The update unit 16 updates the parameters of the NN 24 based on a comparison result between the supervised data corresponding to the first molecule and the prediction result acquired by the acquisition unit 14. Specifically, the update unit 16 generates the value in the first molecule data of the atom whose value has been replaced with 0 as supervised data, and back-propagates the difference between the prediction result and the supervised data to the NN 24 to update the parameters of the NN 24.

[0026] The update unit 16 repeats updating the parameters of the NN 24 until a predetermined termination condition is satisfied. The predetermined termination condition may be, for example, when the number of repetitions reaches a predetermined number, when the difference between the prediction result and the correct answer data becomes a predetermined value or less, when the difference between the previous difference and the current difference becomes a predetermined value or less (when the difference converges), etc.

[0027] The self-supervised learning device 10 may be realized by, for example, a computer 40 shown in Fig. 7. The computer 40 includes a CPU (Central Processing Unit) 41, a GPU (Graphics Processing Unit) 42, a memory 43 as a temporary storage area, and a non-volatile storage device 44. The computer 40 also includes an input / output device 45 such as an input device and a display device, and an R / W (Read / Write) device 46 that controls reading and writing of data from and to a storage medium 49. The computer 40 also includes a communication I / F (Interface) 47 that is connected to a network such as the Internet. The CPU 41, the GPU 42, the memory 43, the storage device 44, the input / output device 45, the R / W device 46, and the communication I / F 47 are connected to one another via a bus 48.

[0028] The storage device 44 is, for example, a hard disk drive (HDD), a solid state drive (SSD), a flash memory, etc. A self-supervised learning program 50 for causing the computer 40 to function as the self-supervised learning device 10 is stored in the storage device 44 as a storage medium. The self-supervised learning program 50 has a generation process control instruction 52, an acquisition process control instruction 54, and an update process control instruction 56. The storage device 44 also has an information storage area 60 in which information constituting each of the atomic library 22 and the NN 24 is stored.

[0029] The CPU 41 reads the self-supervised learning program 50 from the storage device 44, expands it in the memory 43, and sequentially executes the control instructions of the self-supervised learning program 50. The CPU 41 operates as the generation unit 12 shown in FIG. 5 by executing the generation process control instruction 52. The CPU 41 also operates as the acquisition unit 14 shown in FIG. 5 by executing the acquisition process control instruction 54. The CPU 41 also operates as the update unit 16 shown in FIG. 5 by executing the update process control instruction 56. The CPU 41 also reads information from the information storage area 60 and expands each of the atomic library 22 and the NN 24 in the memory 43. As a result, the computer 40 that has executed the self-supervised learning program 50 functions as the self-supervised learning device 10. The CPU 41 that executes the program is hardware. Also, a part of the program may be executed by the GPU 42.

[0030] The functions realized by the self-supervised learning program 50 may be realized, for example, by a semiconductor integrated circuit, more specifically, an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or the like.

[0031] Next, the operation of the self-supervised learning device 10 according to the first embodiment will be described. When first molecular data is input to the self-supervised learning device 10 and execution of self-supervised learning of the NN 24 is instructed, the self-supervised learning device 10 executes the self-supervised learning process shown in Fig. 8. Note that the self-supervised learning process is an example of a self-supervised learning method of the disclosed technology.

[0032] In step S10, the generation unit 12 acquires the first molecule data input to the self-supervised learning device 10. Next, in step S12, the generation unit 12 generates second molecule data indicating a second molecule in which values ​​indicating a predetermined ratio of atoms included in the first molecule are replaced with 0. Next, in step S14, the acquisition unit 14 inputs the generated second molecule data to the NN 24 and acquires information indicating which atoms in the atom library 22 each of the atoms whose values ​​have been replaced with 0 corresponds to, which is the prediction result output from the NN 24.

[0033] Next, in step S16, the update unit 16 generates, as correct data, the value in the first molecule data of the atom whose value has been replaced with 0. Next, in step S18, the update unit 16 back-propagates the difference between the prediction result and the correct data to the NN 24, and updates the parameters of the NN 24.

[0034] Next, in step S20, the update unit 16 determines whether or not a termination condition for parameter update is satisfied. If the termination condition is satisfied, the process proceeds to step S22, and if not, the process returns to step S10. In step S22, the update unit 16 stores the NN 24 in which the final parameters are set in a predetermined storage area, and the self-supervised learning process ends.

[0035] As described above, the self-supervised learning device according to this embodiment generates second molecular data by replacing the values ​​indicating each of a predetermined percentage of atoms contained in the first molecule with 0. The self-supervised learning device then inputs the second molecular data into a NN that predicts the molecular structure to obtain a prediction result, and updates the parameters of the NN based on a comparison result between the ground truth data corresponding to the first molecule and the prediction result. In this way, by replacing the atoms to be replaced with 0 instead of a random value, the learning of unnecessary information is suppressed, and the prediction accuracy of the neural network trained by self-supervised learning can be improved.

[0036] Furthermore, the NN trained by the self-supervised learning device according to the above embodiment can be applied to transfer learning, as described in Fig. 2. That is, the feature extraction unit of the NN trained by self-supervised learning is used as is, and the output unit is changed to suit the task of molecular energy prediction. Then, for example, the NN is trained by supervised learning with labeled data in which the energy calculated by DFT is the correct answer, thereby constructing a NN for the task of energy prediction.

[0037] Here, the results of verification of the effects of the above embodiment will be described. In this verification, the following methods were verified. Comparison method 1: A method of supervised learning of NN using only labeled data (without self-supervised learning) Comparison method 2: A method of supervising a NN trained by existing self-supervised learning (masking random values) with labeled data This method: Method of the first embodiment

[0038] In this validation, 3770 labeled data, which are combinations of molecular structures and the energies corresponding to those structures, were prepared, of which 2870 were used as training data and the remaining 900 were used as validation data that were not used in training. In addition, all of the 2870 training data structures were used for self-supervised learning, and all of the 2870 training data structures and energies were used for supervised learning. In addition, 5-fold cross validation was used as the validation method.

[0039] Figure 9 shows the results of comparing the prediction error of the NN in this method with comparison methods 1 and 2. The smaller the prediction error value, the better the accuracy of the NN. It can be seen that the prediction error of this method is smaller than that of comparison methods 1 and 2, and the accuracy is improved.

[0040] In addition, by applying the self-supervised learning of this method using 1660 labeled data, we verified whether the accuracy of the NN improves compared to the comparison method even when the amount of labeled data used during fine tuning is reduced. In this verification, we measured the prediction error of the NN when the amount of labeled data used during fine tuning is changed. As the above-mentioned labeled data, 1660 pieces of data of combinations of molecular structures and the energy corresponding to the structures were prepared, and of the 1660 pieces of data, 1360 pieces were used as training data and the remaining 300 pieces were used as validation data that were not used for training. In addition, all the structure data (unlabeled) of the 1360 training data were used for self-supervised learning.

[0041] Figure 10 shows the results of comparing the prediction error versus the number of labeled data used for fine-tuning between our method and comparative method 1. It can be seen that our method has a reduced prediction error compared to comparative method 1. In particular, the smaller the amount of labeled data, the greater the effect of our method in improving the accuracy of the NN.

[0042] In the above embodiment, a case has been described in which prediction results are obtained for atoms contained in the second molecule whose values ​​have been replaced with 0, but prediction results may be obtained for all atoms contained in the second molecule as prediction targets. The more prediction targets there are, the more difficult the task becomes, and therefore the more accurate the prediction of the trained neural network becomes.

[0043] Furthermore, when all atoms contained in the second molecule are to be predicted, the NN may have an output section configured with different fully connected layers that are connected to the outputs of the feature extraction sections corresponding to the nodes that make up the second molecule data that is input to the NN, as shown in Fig. 11. This allows prediction of all atoms contained in the second molecule at once.

[0044] In the above embodiment, the self-supervised learning program is stored (installed) in advance in the storage device, but this is not limiting. The program according to the disclosed technology may be provided in a form stored in a storage medium such as a CD-ROM, a DVD-ROM, or a USB memory.

[0045] Regarding the above embodiment, the following supplementary notes are further disclosed.

[0046] (Appendix 1) generating data representing a second molecule in which values ​​representing a predetermined proportion of atoms in the first molecule are replaced with 0; inputting the data representing the second molecule into a machine learning model that makes predictions regarding molecular structures to obtain prediction results; updating parameters of the machine learning model based on a comparison result between correct answer data corresponding to the first molecule and the prediction result; A self-supervised learning program for causing a computer to perform a process including:

[0047] (Appendix 2) The self-supervised learning program according to claim 1, wherein the prediction result is information indicating which atom, among the atoms contained in the second molecule, at least an atom substituted for an atom contained in the first molecule is an atom contained in a predetermined group of multiple types of atoms.

[0048] (Appendix 3) 3. The self-supervised learning program of claim 1, wherein the prediction result includes information about all atoms contained in the second molecule.

[0049] (Appendix 4) The self-supervised learning program according to any one of Supplementary Note 1 to Supplementary Note 3, wherein the machine learning model includes a first part that extracts features indicating a molecular structure of the second molecule from data indicating the second molecule, and a second part that outputs a prediction result according to a specific task based on the features extracted by the first part, and the first part of the trained machine learning model is used for transfer learning.

[0050] (Appendix 5) the second section includes different outputs corresponding to the outputs of the first section for each of the atoms in the second molecule; The process of obtaining the prediction result includes obtaining a prediction result for each atom included in the second molecule at a time. A self-supervised learning program as described in Appendix 4.

[0051] (Appendix 6) generating data representing a second molecule in which values ​​representing a predetermined proportion of atoms in the first molecule are replaced with 0; inputting the data representing the second molecule into a machine learning model that makes predictions regarding molecular structures to obtain prediction results; updating parameters of the machine learning model based on a comparison result between correct answer data corresponding to the first molecule and the prediction result; The self-supervised learning method includes a process in which a computer performs the process.

[0052] (Appendix 7) The self-supervised learning method according to claim 6, wherein the prediction result is information indicating which atom, among atoms contained in the second molecule, at least an atom substituted for an atom contained in the first molecule belongs to a group of predetermined types of atoms.

[0053] (Appendix 8) 8. The self-supervised learning method of claim 6, wherein the prediction result includes information about all atoms contained in the second molecule.

[0054] (Appendix 9) The self-supervised learning method according to any one of Supplementary Note 6 to Supplementary Note 8, wherein the machine learning model includes a first part that extracts features indicating a molecular structure of the second molecule from data indicating the second molecule, and a second part that outputs a prediction result according to a specific task based on the features extracted in the first part, and the first part of the trained machine learning model is used for transfer learning.

[0055] (Appendix 10) the second section includes different outputs corresponding to the outputs of the first section for each of the atoms in the second molecule; The process of obtaining the prediction result includes obtaining a prediction result for each atom included in the second molecule at a time. The self-supervised learning method according to claim 9.

[0056] (Appendix 11) a generating unit that generates data representing a second molecule in which values ​​representing atoms at a predetermined ratio among atoms included in the first molecule are replaced with 0; an acquisition unit that inputs data representing the second molecule into a machine learning model that makes predictions regarding molecular structures to acquire prediction results; an update unit that updates parameters of the machine learning model based on a comparison result between correct answer data corresponding to the first molecule and the prediction result; A self-supervised learning device including

[0057] (Appendix 12) The self-supervised learning apparatus according to claim 11, wherein the prediction result is information indicating which atom, among atoms contained in the second molecule, at least an atom substituted for an atom contained in the first molecule is an atom contained in a predetermined group of multiple types of atoms.

[0058] (Appendix 13) 13. The self-supervised learning apparatus of claim 11 or 12, wherein the prediction result includes information about all atoms contained in the second molecule.

[0059] (Appendix 14) The self-supervised learning device according to any one of Supplementary Note 11 to Supplementary Note 13, wherein the machine learning model includes a first part that extracts features indicating a molecular structure of the second molecule from data indicating the second molecule, and a second part that outputs a prediction result according to a specific task based on the features extracted by the first part, and the first part of the trained machine learning model is used for transfer learning.

[0060] (Appendix 15) the second section includes different outputs corresponding to the outputs of the first section for each of the atoms in the second molecule; The process of obtaining the prediction result includes obtaining a prediction result for each atom included in the second molecule at a time. 15. The self-supervised learning apparatus of claim 14.

[0061] (Appendix 16) generating data representing a second molecule in which values ​​representing a predetermined proportion of atoms in the first molecule are replaced with 0; inputting the data representing the second molecule into a machine learning model that makes predictions regarding molecular structures to obtain prediction results; updating parameters of the machine learning model based on a comparison result between correct answer data corresponding to the first molecule and the prediction result; A non-transitory storage medium storing a self-supervised learning program for causing a computer to execute a process including the steps of: [Explanation of symbols]

[0062] 10 Self-supervised learning device 12 Generation part 14 Acquisition Department 16 Update section 22 Atom Library 24 Neural Networks (NN) 40 Computer 41 CPU 42 GPU 43 Memory 44 Storage device 45 I / O Devices 46 R / W device 47 Communication I / F 48 Bus 49 Storage medium 50 Self-supervised learning programs 52 Generation process control instructions 54 Acquisition Process Control Instructions 56 Update process control instructions 60 Information storage area

Claims

1. generating data representing a second molecule in which values ​​representing a predetermined proportion of atoms among the atoms included in the first molecule are replaced with 0; inputting data representing the second molecule into a machine learning model that makes predictions regarding molecular structures to obtain prediction results; updating parameters of the machine learning model based on a comparison result between correct answer data corresponding to the first molecule and the prediction result; A self-supervised learning program for causing a computer to perform a process including:

2. 2. The self-supervised learning program according to claim 1, wherein the prediction result is information indicating which atom, among the atoms contained in the second molecule, an atom that has had its value replaced with 0 from at least an atom contained in the first molecule belongs to, among a group of atoms of multiple predetermined types of atoms.

3. The self-supervised learning program of claim 2 , wherein the prediction result includes information about all atoms contained in the second molecule.

4. The self-supervised learning program according to any one of claims 1 to 3, wherein the machine learning model includes a first part that extracts features indicating a molecular structure of the second molecule from data indicating the second molecule, and a second part that outputs a prediction result according to a specific task based on the features extracted in the first part, and the first part of the trained machine learning model is used for transfer learning.

5. the second section includes different outputs corresponding to the outputs of the first section for each of the atoms included in the second molecule; The process of obtaining the prediction result includes obtaining a prediction result for each atom included in the second molecule at a time. The self-supervised learning program of claim 4.

6. generating data representing a second molecule in which values ​​representing a predetermined proportion of atoms among the atoms included in the first molecule are replaced with 0; inputting data representing the second molecule into a machine learning model that makes predictions regarding molecular structures to obtain prediction results; updating parameters of the machine learning model based on a comparison result between correct answer data corresponding to the first molecule and the prediction result; The self-supervised learning method includes a process in which a computer performs the process.

7. a generating unit that generates data representing a second molecule in which values ​​representing atoms at a predetermined ratio among atoms included in the first molecule are replaced with 0; an acquisition unit that inputs data representing the second molecule into a machine learning model that makes predictions regarding molecular structures and acquires prediction results; an update unit that updates parameters of the machine learning model based on a comparison result between correct answer data corresponding to the first molecule and the prediction result; A self-supervised learning device including

Citation Information

Patent Citations

  • Method for pre-training graph neural network, program, and device

    JP2021197159A

  • Methods and systems for training a graph neural network using supervised contrastive learning

    US20220383127A1

  • Learning device, estimation device, method, and program

    WO2021111499A1

  • Method and system for generating data for machine learning algorithms

    JP2021518024A