Protein Sequence Design via Memory-Efficient Global Attention
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for protein structure modeling and prediction face limitations in accurately learning complex residue inter-dependencies due to independence assumptions and noise propagation in residue interactions, leading to suboptimal sequence design and prediction performance.
Innovation Solution
A machine-learning based method that employs an entropy-based prediction-selection method and memory-efficient global graph attention layers to learn and propagate high-quality residue interactions, allowing each residue node to gather information from the entire sequence while maintaining memory efficiency, and generating sequences non-iteratively.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If proximity graphs are used to model residue interactions with autoregressive decoding, then complex residue inter-dependencies can be learned, but error accumulation occurs due to dependency on previous predictions
Solution Approach 1:
The patent inverts the conventional autoregressive decoding approach by using non-autoregressive decoding with entropy-based selection. Instead of predicting residues sequentially where each prediction depends on previous ones, the model predicts all residues simultaneously and then selects the most confident predictions based on entropy values, eliminating error accumulation from sequential dependency.
Solution Approach 2:
The entropy-based prediction-selection mechanism allows the model to self-evaluate its own prediction confidence and selectively accept or reject predictions based on their entropy values. High-confidence predictions (low entropy) are retained while low-confidence predictions (high entropy) are rejected and replaced, enabling the system to self-correct without external intervention.
2Measurement precision
If global attention is used to learn residue interactions from entire sequence, then accurate global residue interactions can be captured, but memory consumption increases significantly
Solution Approach 1:
The patent segments the global attention computation into two parts: (1) local attention within K-nearest neighbor graphs that captures spatially proximal residue interactions, and (2) global information propagation through the graph structure that indirectly captures long-range interactions. This segmentation reduces the computational complexity from O(N²) to O(N×K) while maintaining the ability to model global residue interactions.
3Manufacturing precision
If iterative decoding is used to update residues based on neighborhood, then residue interactions can be refined, but noisy residue information is propagated through the graph structure
Solution Approach 1:
The patent converts the potentially harmful noise propagation in iterative decoding into a beneficial filtering process. By computing entropy values for all predictions simultaneously and using these entropy values to selectively accept or reject predictions, the method transforms the noise issue into a confidence-based selection mechanism that eliminates noisy predictions while retaining high-confidence ones.
Data Source
AI summary
A machine-learning based method and systems for protein sequence design are provided. The method includes generating a portion of a sequence and removing noise in input residue context, encoding and processing the portion of the sequence and backbone structure to obtain graph features, performing memory-efficient global graph attention layers to propagate the graph features and learn global residue interactions; and generating an entire sequence non-iteratively. The performing memory-efficient global graph attention layers includes enabling each residue node to learn residue interactions and gather information from the entire sequence while maintaining memory efficiency. The edge features of the memory-efficient global graph attention layers include interatomic distances and direction vectors.