Large Language Model Data Augmentation for Protein Activity Prediction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing protein engineering techniques rely on labor-intensive and costly experimental methods to identify beneficial amino acid mutations, and supervised machine learning models lack sufficient training data to predict protein activity effectively due to the scarcity and high cost of experimentally validated protein sequence datasets.
Innovation Solution
Leverage large language models (LLMs) for negative data augmentation to generate synthetic protein sequences with amino acid mutations, creating a high-contrast training dataset that includes negative examples with low activity levels, using a masked marginal approach to train a machine learning model with a supervised loss function and contrastive objective.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If experimental methods are used to identify beneficial amino acid mutations, then accurate protein activity prediction is achieved, but time consumption and cost increase significantly
Solution Approach 1:
The patent creates synthetic copies of protein sequences with mutated amino acids using large language models. These synthetic sequences replicate the functional properties of real proteins without requiring physical experimentation, enabling rapid screening of mutations while maintaining prediction accuracy
Solution Approach 2:
The patent performs preliminary computational screening of amino acid mutations using trained machine learning models before conducting actual experiments. This preliminary action filters out unlikely beneficial mutations, reducing the number of experiments needed and accelerating the identification process
2Reliability
If experimental validation is performed for each protein sequence, then reliable activity data is obtained, but resource consumption increases
Solution Approach 1:
The patent develops a universal machine learning model that can predict activity across multiple protein sequences and mutation types simultaneously. Once trained on diverse data, the model serves multiple functions for screening different mutations, reducing repetitive experimental validation needs
Solution Approach 2:
Synthetic protein sequences with predicted low activity are generated as copies to serve as negative training examples. These copies provide reliable activity information computationally, replacing the need for expensive experimental validation of each individual sequence
3Measurement precision
If comprehensive training data is used to train machine learning models, then prediction accuracy improves, but data availability is limited due to experimental constraints
Solution Approach 1:
The patent generates synthetic training data by creating copies of known protein sequences with computationally predicted mutations. These synthetic examples expand the training dataset significantly, providing sufficient data for accurate model training without proportional increases in experimental work
Solution Approach 2:
Instead of experimenting to find beneficial mutations, the patent inverts the approach by using language models to identify and generate sequences with low probability of benefit (negative examples). This inverted strategy efficiently creates comprehensive training data by focusing computational resources on generating diverse negative examples
Data Source
AI summary
A method for training a machine learning model (MLM) to predict the activity of a protein is described herein. In an example, a method involves accessing a set of training data comprising labeled examples with known activity levels. A large language model is used to generate synthetic examples of each labeled example by incorporating each possible amino acid (AA) mutation at each AA position in the labeled example and predicting the probability each AA mutation has of replacing the original AA. Based on a predetermined cutoff, a subset of negative synthetic examples that comprises at least one AA mutation with the lowest probability of being incorporated are selected. An augmented training dataset is generated and a MLM is trained, using the training data and the augmented training data set, by performing iterative operations to find a set of parameters that jointly minimize the sum of at least two loss functions.