Self-Supervised Molecular Training via Atom Masking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural networks trained for molecular energy prediction face challenges in collecting large labeled data due to the long calculation time of density functional theory (DFT), and self-supervised training methods sometimes perform useless training by masking non-existent atoms, which deteriorates prediction accuracy.
Innovation Solution
A self-supervised training program that generates data by replacing a predetermined percentage of atoms in a molecule with zero, allowing a machine learning model to predict the masked atoms and update its parameters based on comparison with correct answer data, thereby avoiding useless training and enhancing prediction accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If self-supervised training is performed by masking atoms in molecular structures, then training efficiency is improved, but prediction accuracy deteriorates due to training on non-existent atoms
Solution Approach 1:
The patent extracts and removes the harmful masking information from the training data. Instead of masking atoms with placeholder values that create non-existent atomic information, the patent completely removes these masked atoms from the molecular structure representation, ensuring the neural network trains only on actual, existing atoms and their interactions.
Solution Approach 2:
The patent inverts the conventional masking approach by not replacing masked atoms with placeholder values, but rather completely removing them from the data structure. This inversion transforms the masking operation from creating artificial atomic entities to eliminating non-existent ones, thereby preventing the neural network from learning spurious patterns.
2Measurement precision
If density functional theory is used to calculate molecular energy, then prediction accuracy is improved, but data collection efficiency deteriorates due to long calculation time
Solution Approach 1:
The patent performs preliminary self-supervised training on large amounts of unlabeled molecular data to pre-train the neural network on fundamental molecular representation and interaction patterns. This preliminary action enables the model to achieve good prediction accuracy on the final task with much fewer DFT-calculated labeled examples, effectively decoupling the accuracy requirement from the data collection burden.
Solution Approach 2:
The patent employs self-supervised learning where the neural network generates its own training labels from unlabeled molecular structures through masked atom prediction. This self-service mechanism allows the model to learn from vast amounts of molecular data without requiring expensive DFT energy calculations for each training example, dramatically improving data collection efficiency while maintaining accuracy.
Data Source
AI summary
A non-transitory computer-readable recording medium stores a self-supervised training program for causing a computer to execute a process including: generating data that indicates a second molecule obtained by replacing a value that indicates each of a predetermined percentage of atoms among the atoms contained in a first molecule, with zero; acquiring a prediction result by inputting the data that indicates the second molecule to a machine learning model that performs prediction regarding a molecular structure; and updating a parameter of the machine learning model, based on a comparison result between correct answer data that corresponds to the first molecule and the prediction result.