Multi-Task Pre-Training With Geometric Alignment for Small Molecular Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing transfer learning techniques struggle with small-scale, complex data sets such as molecular data sets, particularly in fields like chemistry and pharmacy, due to their inability to effectively handle non-Euclidean spaces and complex structures, leading to limitations in prediction performance and generalization.
Innovation Solution
A pre-training method and system for a multi-tasking model that performs transfer learning through geometric alignment in an integrated latent space, allowing simultaneous training of multiple tasks across domains and aligning geometric properties between tasks, thereby enhancing prediction performance and stability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If existing Euclidean space-based transfer learning techniques are applied to molecular structure data, then the model can be trained on available data, but the prediction performance deteriorates due to inability to effectively handle complex non-Euclidean structures
Solution Approach 1:
The patent changes the mathematical space parameter from Euclidean space to Riemannian manifold space. This transformation allows the model to properly represent and process molecular structure data that inherently exists in non-Euclidean space, thereby improving prediction performance while maintaining adaptability to complex data structures
Solution Approach 2:
The patent replaces the traditional Euclidean distance-based similarity measurement mechanism with a Riemannian geometric alignment mechanism. This substitution enables the model to capture complex relationships in molecular data by utilizing the curved geometry of the Riemannian manifold, leading to better transfer learning performance
2Quantity of substance
If transfer learning is applied to small-scale molecular data sets, then the need for extensive experimental data is reduced, but the generalization performance deteriorates due to insufficient data characteristics
Solution Approach 1:
The patent creates a universal Riemannian manifold-based transfer learning framework that can handle multiple molecular property prediction tasks simultaneously. By learning a unified geometric representation that captures fundamental molecular structure characteristics, the model achieves better generalization performance across different tasks even with limited data for each specific task
Solution Approach 2:
The patent performs preliminary pre-training on source domains with sufficient data to learn robust Riemannian geometric representations of molecular structures. These pre-learned representations are then transferred to target domains with small-scale data, enabling the model to generalize effectively without requiring extensive task-specific experimental data
3Measurement precision
If multiple physical properties are predicted separately using traditional methods, then each property can be optimized individually, but the training time and computational resources increase significantly
Solution Approach 1:
The patent merges multiple physical property prediction tasks into a unified multi-task learning framework based on Riemannian manifold. By sharing the underlying geometric representation and alignment mechanism across tasks, the model can predict multiple properties simultaneously while maintaining individual optimization, thereby reducing training time and computational resources
Solution Approach 2:
The patent establishes continuous geometric alignment across multiple tasks in the Riemannian manifold space. This continuous framework allows the model to learn transferable representations that benefit all tasks simultaneously, enabling parallel training of multiple properties without the need for separate training processes, thus reducing overall training time
Data Source
AI summary
A method for performing pre-training for a multi-tasking model includes: acquiring experimental data including material-specific characteristic information, which is information specifying unique characteristics of a predetermined material, and material-physical property specific information, which is information specifying characteristic values for a plurality of physical properties of the material; simultaneously training a plurality of tasks for predicting the characteristic values for the plurality of physical properties based on the acquired experimental data in the multi-tasking model; and providing the trained multi-tasking model. The simultaneous training of the plurality of tasks includes simultaneously training the plurality of tasks based on a plurality of task processing units, each including a task processing unit configured to process a plurality of sub-tasks for predicting a characteristic value for each physical property.