Anisotropic Pooling for Protein Sequence Contextual Embedding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current deep learning models for processing protein and genome sequences require sequences as input data structures, limiting the effectiveness of information learned from grouping evolutionary related sequences, such as those in multiple sequence alignments (MSA) matrices.
Innovation Solution
The generation of contextual embedding sequences through anisotropic pooling and convolutional operations on MSA matrices, allowing for the conversion of arrangement-based information into sequence format without significant loss, enhancing the robustness of sequence processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If deep learning models use sequence format as input data structure, then the models can process protein and genome sequences effectively, but the models cannot fully utilize information from grouping evolutionary related sequences in MSA matrices
Solution Approach 1:
The patent transforms the input data from a one-dimensional sequence format to a two-dimensional MSA matrix format, enabling the model to capture evolutionary relationships between sequences. The MSA matrix represents sequences in a grid where rows are sequences and columns are positions, allowing simultaneous processing of multiple sequence dimensions.
Solution Approach 2:
The patent introduces an intermediary representation layer that converts MSA matrices into contextual embedding sequences. This intermediary transformation preserves evolutionary information while maintaining compatibility with standard sequence-processing deep learning models, acting as a bridge between MSA data and model input requirements.
2Loss of information
If the patent converts MSA matrices to contextual embedding sequences through anisotropic pooling and convolutional operations, then evolutionary information is preserved, but the computational complexity increases
Solution Approach 1:
The patent divides the MSA matrix processing into discrete operational steps: convolutional operations for feature extraction, anisotropic pooling for dimensionality reduction, and contextual embedding generation. This segmentation allows each operation to be optimized independently and facilitates parallel computation.
Solution Approach 2:
The anisotropic pooling operation dynamically adjusts pooling parameters (kernel sizes, stride values) based on the input MSA matrix dimensions and evolutionary information content. This adaptive parameter adjustment optimizes the balance between information preservation and computational efficiency for different input scenarios.
Data Source
AI summary
Techniques are described and relate to anisotropic pooling for contextual embedding of a protein sequence. In an example, a system receives a first biological sequence and determines a sequence arrangement that comprises a component of the first biological sequence and a second biological sequence of components. By using an artificial intelligence (AI) model, the system determines a third sequence that comprises a contextual embedding vector corresponding to the component of the first biological sequence. The AI model generates the third sequence based at least in part on the sequence arrangement and by at least using a convolution and anisotropic pooling.